RESTful Content

One of the stories from the early days of the web is how search engines wiped out entire websites. When dynamic web sites were still a new concept, developers didn’t appreciate the difference between a GET and POST request. As a result, they created pages- accessed with the GET method- that would delete pages. When search engines started crawling these sites, they could wipe out all the content.

If these web developers had followed the HTTP spec properly, this would not have happened. A GET request is supposed to cause no side effects (you know, like wiping out a site). Recently, there has been a move in web development to properly embrace Representational State Transfer, also known as REST. This chapter describes the RESTful features in Yesod and how you can use them to create more robust web applications.

Request methods

In many web frameworks, you write one handler function per resource. In Yesod, the default is to have a separate handler function for each request method. The two most common request methods you will deal with in creating web sites are GET and POST. These are the most well-supported methods in HTML, since they are the only ones supported by web forms. However, when creating RESTful APIs, the other methods are very useful.

Technically speaking, you can create whichever request methods you like, but it is strongly recommended to stick to the ones spelled out in the HTTP spec. The most common of these are:

GET

Read-only requests. Assuming no other changes occur on the server, calling a GET request multiple times should result in the same response, barring such things as “current time” or randomly assigned results.

POST

A general mutating request. A POST request should never be submitted twice by the user. A common example of this would be to transfer funds from one bank account to another.

PUT

Create a new resource on the server, or replace an existing one. This method is safe to be called multiple times.

PATCH

Updates the resource partially on the server. When you want to update one or more field of the resource, this method should be preferred.

DELETE

Just like it sounds: wipe out a resource on the server. Calling multiple times should be OK.

To a certain extent, this fits in very well with Haskell philosophy: a GET request is similar to a pure function, which cannot have side effects. In practice, your GET functions will probably perform IO, such as reading information from a database, logging user actions, and so on.

See the routing and handlers chapter for more information on the syntax of defining handler functions for each request method.

Representations

Suppose we have a Haskell datatype and value:

  1. data Person = Person { name :: String, age :: Int }
  2. michael = Person "Michael" 25

We could represent that data as HTML:

  1. <table>
  2.     <tr>
  3.         <th>Name</th>
  4.         <td>Michael</td>
  5.     </tr>
  6.     <tr>
  7.         <th>Age</th>
  8.         <td>25</td>
  9.     </tr>
  10. </table>

or we could represent it as JSON:

  1. {"name":"Michael","age":25}

or as XML:

  1. <person>
  2.     <name>Michael</name>
  3.     <age>25</age>
  4. </person>

Often times, web applications will use a different URL to get each of these representations; perhaps /person/michael.html, /person/michael.json, etc. Yesod follows the RESTful principle of a single URL for each resource. So in Yesod, all of these would be accessed from /person/michael.

Then the question becomes how do we determine which representation to serve. The answer is the HTTP Accept header: it gives a prioritized list of content types the client is expecting. Yesod provides a pair of functions to abstract away the details of parsing that header directly, and instead allows you to talk at a much higher level of representations. Let’s make that last sentence a bit more concrete with some code:

  1. {-# LANGUAGE OverloadedStrings #-}
  2. {-# LANGUAGE QuasiQuotes       #-}
  3. {-# LANGUAGE TemplateHaskell   #-}
  4. {-# LANGUAGE TypeFamilies      #-}
  5. import           Data.Text (Text)
  6. import           Yesod
  8. data App = App
  10. mkYesod "App" [parseRoutes|
  11. / HomeR GET
  12. |]
  14. instance Yesod App
  16. getHomeR :: Handler TypedContent
  17. getHomeR = selectRep $ do
  18.     provideRep $ return
  19.         [shamlet|
  20.             <p>Hello, my name is #{name} and I am #{age} years old.
  21.         |]
  22.     provideRep $ return $ object
  23.         [ "name" .= name
  24.         , "age" .= age
  25.         ]
  26.   where
  27.     name = "Michael" :: Text
  28.     age = 28 :: Int
  30. main :: IO ()
  31. main = warp 3000 App

The selectRep function says “I’m about to give you some possible representations”. Each provideRep call provides an alternate representation. Yesod uses the Haskell types to determine the mime type for each representation. Since shamlet (a.k.a. simple Hamlet) produces an Html value, Yesod can determine that the relevant mime type is text/html. Similarly, object generates a JSON value, which implies the mime type application/json. TypedContent is a data type provided by Yesod for some raw content with an attached mime type. We’ll cover it in more detail in a little bit.

To test this out, start up the server and then try running the following different curl commands:

  1. curl http://localhost:3000 --header "accept: application/json"
  2. curl http://localhost:3000 --header "accept: text/html"
  3. curl http://localhost:3000

Notice how the response changes based on the accept header value. Also, when you leave off the header, the HTML response is displayed by default. The rule here is that if there is no accept header, the first representation is displayed. If an accept header is present, but we have no matches, then a 406 “not acceptable” response is returned.

By default, Yesod provides a convenience middleware that lets you set the accept header via a query string parameter. This can make it easier to test from your browser. To try this out, you can visit http://localhost:3000/?_accept=application/json.

JSON conveniences

Since JSON is such a commonly used data format in web applications today, we have some built-in helper functions for providing JSON representations. These are built off of the wonderful aeson library, so let’s start off with a quick explanation of how that library works.

aeson has a core datatype, Value, which represents any valid JSON value. It also provides two typeclasses- ToJSON and FromJSON- to automate marshaling to and from JSON values, respectively. For our purposes, we’re currently interested in ToJSON. Let’s look at a quick example of creating a ToJSON instance for our ever-recurring Person data type examples.

  1. {-# LANGUAGE OverloadedStrings #-}
  2. {-# LANGUAGE RecordWildCards   #-}
  3. import           Data.Aeson
  4. import qualified Data.ByteString.Lazy.Char8 as L
  5. import           Data.Text                  (Text)
  7. data Person = Person
  8.     { name :: Text
  9.     , age  :: Int
  10.     }
  12. instance ToJSON Person where
  13.     toJSON Person {..} = object
  14.         [ "name" .= name
  15.         , "age"  .= age
  16.         ]
  18. main :: IO ()
  19. main = L.putStrLn $ encode $ Person "Michael" 28

I won’t go into further detail on aeson, as the Haddock documentation already provides a great introduction to the library. What I’ve described so far is enough to understand our convenience functions.

Let’s suppose that you have such a Person datatype, with a corresponding value, and you’d like to use it as the representation for your current page. For that, you can use the returnJson function.

  1. {-# LANGUAGE OverloadedStrings #-}
  2. {-# LANGUAGE QuasiQuotes       #-}
  3. {-# LANGUAGE RecordWildCards   #-}
  4. {-# LANGUAGE TemplateHaskell   #-}
  5. {-# LANGUAGE TypeFamilies      #-}
  6. import           Data.Text (Text)
  7. import           Yesod
  9. data Person = Person
  10.     { name :: Text
  11.     , age  :: Int
  12.     }
  14. instance ToJSON Person where
  15.     toJSON Person {..} = object
  16.         [ "name" .= name
  17.         , "age"  .= age
  18.         ]
  20. data App = App
  22. mkYesod "App" [parseRoutes|
  23. / HomeR GET
  24. |]
  26. instance Yesod App
  28. getHomeR :: Handler Value
  29. getHomeR = returnJson $ Person "Michael" 28
  31. main :: IO ()
  32. main = warp 3000 App

returnJson is actually a trivial function; it is implemented as return . toJSON. However, it makes things just a bit more convenient. Similarly, if you would like to provide a JSON value as a representation inside a selectRep, you can use provideJson.

  1. {-# LANGUAGE OverloadedStrings #-}
  2. {-# LANGUAGE QuasiQuotes       #-}
  3. {-# LANGUAGE RecordWildCards   #-}
  4. {-# LANGUAGE TemplateHaskell   #-}
  5. {-# LANGUAGE TypeFamilies      #-}
  6. import           Data.Text (Text)
  7. import           Yesod
  9. data Person = Person
  10.     { name :: Text
  11.     , age  :: Int
  12.     }
  14. instance ToJSON Person where
  15.     toJSON Person {..} = object
  16.         [ "name" .= name
  17.         , "age"  .= age
  18.         ]
  20. data App = App
  22. mkYesod "App" [parseRoutes|
  23. / HomeR GET
  24. |]
  26. instance Yesod App
  28. getHomeR :: Handler TypedContent
  29. getHomeR = selectRep $ do
  30.     provideRep $ return
  31.         [shamlet|
  32.             <p>Hello, my name is #{name} and I am #{age} years old.
  33.         |]
  34.     provideJson person
  35.   where
  36.     person@Person {..} = Person "Michael" 28
  38. main :: IO ()
  39. main = warp 3000 App

provideJson is similarly trivial, in this case provideRep . returnJson.

New datatypes

Let’s say I’ve come up with some new data format based on using Haskell’s Show instance; I’ll call it “Haskell Show”, and give it a mime type of text/haskell-show. And let’s say that I decide to include this representation from my web app. How do I do it? For a first attempt, let’s use the TypedContent datatype directly.

  1. {-# LANGUAGE OverloadedStrings #-}
  2. {-# LANGUAGE QuasiQuotes       #-}
  3. {-# LANGUAGE TemplateHaskell   #-}
  4. {-# LANGUAGE TypeFamilies      #-}
  5. import           Data.Text (Text)
  6. import           Yesod
  8. data Person = Person
  9.     { name :: Text
  10.     , age  :: Int
  11.     }
  12.     deriving Show
  14. data App = App
  16. mkYesod "App" [parseRoutes|
  17. / HomeR GET
  18. |]
  20. instance Yesod App
  22. mimeType :: ContentType
  23. mimeType = "text/haskell-show"
  25. getHomeR :: Handler TypedContent
  26. getHomeR =
  27.     return $ TypedContent mimeType $ toContent $ show person
  28.   where
  29.     person = Person "Michael" 28
  31. main :: IO ()
  32. main = warp 3000 App

There are a few important things to note here.

  • We’ve used the toContent function. This is a typeclass function that can convert a number of data types to raw data ready to be sent over the wire. In this case, we’ve used the instance for String, which uses UTF8 encoding. Other common data types with instances are Text, ByteString, Html, and aeson’s Value.

  • We’re using the TypedContent constructor directly. It takes two arguments: a mime type, and the raw content. Note that ContentType is simply a type alias for a strict ByteString.

That’s all well and good, but it bothers me that the type signature for getHomeR is so uninformative. Also, the implementation of getHomeR looks pretty boilerplate. I’d rather just have a datatype representing “Haskell Show” data, and provide some simple means of creating such values. Let’s try this on for size:

  1. {-# LANGUAGE ExistentialQuantification #-}
  2. {-# LANGUAGE OverloadedStrings         #-}
  3. {-# LANGUAGE QuasiQuotes               #-}
  4. {-# LANGUAGE TemplateHaskell           #-}
  5. {-# LANGUAGE TypeFamilies              #-}
  6. import           Data.Text (Text)
  7. import           Yesod
  9. data Person = Person
  10.     { name :: Text
  11.     , age  :: Int
  12.     }
  13.     deriving Show
  15. data App = App
  17. mkYesod "App" [parseRoutes|
  18. / HomeR GET
  19. |]
  21. instance Yesod App
  23. mimeType :: ContentType
  24. mimeType = "text/haskell-show"
  26. data HaskellShow = forall a. Show a => HaskellShow a
  28. instance ToContent HaskellShow where
  29.     toContent (HaskellShow x) = toContent $ show x
  30. instance ToTypedContent HaskellShow where
  31.     toTypedContent = TypedContent mimeType . toContent
  33. getHomeR :: Handler HaskellShow
  34. getHomeR =
  35.     return $ HaskellShow person
  36.   where
  37.     person = Person "Michael" 28
  39. main :: IO ()
  40. main = warp 3000 App

The magic here lies in two typeclasses. As we mentioned before, ToContent tells how to convert a value into a raw response. In our case, we would like to show the original value to get a String, and then convert that String into the raw content. Often times, instances of ToContent will build on each other in this way.

ToTypedContent is used internally by Yesod, and is called on the result of all handler functions. As you can see, the implementation is fairly trivial, simply stating the mime type and then calling out to toContent.

Finally, let’s make this a bit more complicated, and get this to play well with selectRep.

  1. {-# LANGUAGE ExistentialQuantification #-}
  2. {-# LANGUAGE OverloadedStrings         #-}
  3. {-# LANGUAGE QuasiQuotes               #-}
  4. {-# LANGUAGE RecordWildCards           #-}
  5. {-# LANGUAGE TemplateHaskell           #-}
  6. {-# LANGUAGE TypeFamilies              #-}
  7. import           Data.Text (Text)
  8. import           Yesod
  10. data Person = Person
  11.     { name :: Text
  12.     , age  :: Int
  13.     }
  14.     deriving Show
  16. instance ToJSON Person where
  17.     toJSON Person {..} = object
  18.         [ "name" .= name
  19.         , "age"  .= age
  20.         ]
  22. data App = App
  24. mkYesod "App" [parseRoutes|
  25. / HomeR GET
  26. |]
  28. instance Yesod App
  30. mimeType :: ContentType
  31. mimeType = "text/haskell-show"
  33. data HaskellShow = forall a. Show a => HaskellShow a
  35. instance ToContent HaskellShow where
  36.     toContent (HaskellShow x) = toContent $ show x
  37. instance ToTypedContent HaskellShow where
  38.     toTypedContent = TypedContent mimeType . toContent
  39. instance HasContentType HaskellShow where
  40.     getContentType _ = mimeType
  42. getHomeR :: Handler TypedContent
  43. getHomeR = selectRep $ do
  44.     provideRep $ return $ HaskellShow person
  45.     provideJson person
  46.   where
  47.     person = Person "Michael" 28
  49. main :: IO ()
  50. main = warp 3000 App

The important addition here is the HasContentType instance. This may seem redundant, but it serves an important role. We need to be able to determine the mime type of a possible representation before creating that representation. ToTypedContent only works on a concrete value, and therefore can’t be used before creating the value. getContentType instead takes a proxy value, indicating the type without providing anything concrete.

If you want to provide a representation for a value that doesn’t have a HasContentType instance, you can use the provideRepType function, which requires you to explicitly state the mime type present.

Other request headers

There are a great deal of other request headers available. Some of them only affect the transfer of data between the server and client, and should not affect the application at all. For example, Accept-Encoding informs the server which compression schemes the client understands, and Host informs the server which virtual host to serve up.

Other headers do affect the application, but are automatically read by Yesod. For example, the Accept-Language header specifies which human language (English, Spanish, German, Swiss-German) the client prefers. See the i18n chapter for details on how this header is used.

Summary

Yesod adheres to the following tenets of REST:

  • Use the correct request method.

  • Each resource should have precisely one URL.

  • Allow multiple representations of data on the same URL.

  • Inspect request headers to determine extra information about what the client wants.

This makes it easy to use Yesod not just for building websites, but for building APIs. In fact, using techniques such as selectRep/provideRep, you can serve both a user-friendly, HTML page and a machine-friendly, JSON page from the same URL.