RESTful Content

One of the stories from the early days of the web is how search engines wiped out entire websites. When dynamic web sites were still a new concept, developers didn’t appreciate the difference between a GET and POST request. As a result, they created pages- accessed with the GET method- that would delete pages. When search engines started crawling these sites, they could wipe out all the content.

If these web developers had followed the HTTP spec properly, this would not have happened. A GET request is supposed to cause no side effects (you know, like wiping out a site). Recently, there has been a move in web development to properly embrace Representational State Transfer, also known as REST. This chapter describes the RESTful features in Yesod and how you can use them to create more robust web applications.

Request methods

In many web frameworks, you write one handler function per resource. In Yesod, the default is to have a separate handler function for each request method. The two most common request methods you will deal with in creating web sites are GET and POST. These are the most well-supported methods in HTML, since they are the only ones supported by web forms. However, when creating RESTful APIs, the other methods are very useful.

Technically speaking, you can create whichever request methods you like, but it is strongly recommended to stick to the ones spelled out in the HTTP spec. The most common of these are:

GET

Read-only requests. Assuming no other changes occur on the server, calling a GET request multiple times should result in the same response, barring such things as “current time” or randomly assigned results.

POST

A general mutating request. A POST request should never be submitted twice by the user. A common example of this would be to transfer funds from one bank account to another.

PUT

Create a new resource on the server, or replace an existing one. This method is safe to be called multiple times.

DELETE

Just like it sounds: wipe out a resource on the server. Calling multiple times should be OK.

To a certain extent, this fits in very well with Haskell philosophy: a GET request is similar to a pure function, which cannot have side effects. In practice, your GET functions will probably perform IO, such as reading information from a database, logging user actions, and so on.

See the routing and handlers chapter for more information on the syntax of defining handler functions for each request method.

Representations

Suppose we have a Haskell datatype and value:

data Person = Person { name :: String, age :: Int }
michael = Person "Michael" 25

We could represent that data as HTML:

<table>
    <tr>
        <th>Name</th>
        <td>Michael</td>
    </tr>
    <tr>
        <th>Age</th>
        <td>25</td>
    </tr>
</table>

or we could represent it as JSON:

{"name":"Michael","age":25}

or as XML:

<person>
    <name>Michael</name>
    <age>25</age>
</person>

Often times, web applications will use a different URL to get each of these representations; perhaps /person/michael.html, /person/michael.json, etc. Yesod follows the RESTful principle of a single URL for each resource. So in Yesod, all of these would be accessed from /person/michael.

Then the question becomes how do we determine which representation to serve. The answer is the HTTP Accept header: it gives a prioritized list of content types the client is expecting. Yesod provides a pair of functions to abstract away the details of parsing that header directly, and instead allows you to talk at a much higher level of representations. Let’s make that last sentence a bit more concrete with some code:

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes       #-}
{-# LANGUAGE TemplateHaskell   #-}
{-# LANGUAGE TypeFamilies      #-}
import           Data.Text (Text)
import           Yesod
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
getHomeR :: Handler TypedContent
getHomeR = selectRep $ do
    provideRep $ return
        [shamlet|
            <p>Hello, my name is #{name} and I am #{age} years old.
        |]
    provideRep $ return $ object
        [ "name" .= name
        , "age" .= age
        ]
  where
    name = "Michael" :: Text
    age = 28 :: Int
main :: IO ()
main = warp 3000 App

The selectRep function says “I’m about to give you some possible representations”. Each provideRep call provides an alternate representation. Yesod uses the Haskell types to determine the mime type for each representation. Since shamlet (a.k.a. simple Hamlet) produces an Html value, Yesod can determine that the relevant mime type is text/html. Similarly, object generates a JSON value, which implies the mime type application/json. TypedContent is a data type provided by Yesod for some raw content with an attached mime type. We’ll cover it in more detail in a little bit.

To test this out, start up the server and then try running the following different curl commands:

curl http://localhost:3000 --header "accept: application/json"
curl http://localhost:3000 --header "accept: text/html"
curl http://localhost:3000

Notice how the response changes based on the accept header value. Also, when you leave off the header, the HTML response is displayed by default. The rule here is that if there is no accept header, the first representation is displayed. If an accept header is present, but we have no matches, then a 406 “not acceptable” response is returned.

By default, Yesod provides a convenience middleware that lets you set the accept header via a query string parameter. This can make it easier to test from your browser. To try this out, you can visit http://localhost:3000/?_accept=application/json.

JSON conveniences

Since JSON is such a commonly used data format in web applications today, we have some built-in helper functions for providing JSON representations. These are built off of the wonderful aeson library, so let’s start off with a quick explanation of how that library works.

aeson has a core datatype, Value, which represents any valid JSON value. It also provides two typeclasses- ToJSON and FromJSON- to automate marshaling to and from JSON values, respectively. For our purposes, we’re currently interested in ToJSON. Let’s look at a quick example of creating a ToJSON instance for our ever-recurring Person data type examples.

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RecordWildCards   #-}
import           Data.Aeson
import qualified Data.ByteString.Lazy.Char8 as L
import           Data.Text                  (Text)
data Person = Person
    { name :: Text
    , age  :: Int
    }
instance ToJSON Person where
    toJSON Person {..} = object
        [ "name" .= name
        , "age"  .= age
        ]
main :: IO ()
main = L.putStrLn $ encode $ Person "Michael" 28

I won’t go into further detail on aeson, as the Haddock documentation already provides a great introduction to the library. What I’ve described so far is enough to understand our convenience functions.

Let’s suppose that you have such a Person datatype, with a corresponding value, and you’d like to use it as the representation for your current page. For that, you can use the returnJson function.

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes       #-}
{-# LANGUAGE RecordWildCards   #-}
{-# LANGUAGE TemplateHaskell   #-}
{-# LANGUAGE TypeFamilies      #-}
import           Data.Text (Text)
import           Yesod
data Person = Person
    { name :: Text
    , age  :: Int
    }
instance ToJSON Person where
    toJSON Person {..} = object
        [ "name" .= name
        , "age"  .= age
        ]
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
getHomeR :: Handler Value
getHomeR = returnJson $ Person "Michael" 28
main :: IO ()
main = warp 3000 App

returnJson is actually a trivial function; it is implemented as return . toJSON. However, it makes things just a bit more convenient. Similarly, if you would like to provide a JSON value as a representation inside a selectRep, you can use provideJson.

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes       #-}
{-# LANGUAGE RecordWildCards   #-}
{-# LANGUAGE TemplateHaskell   #-}
{-# LANGUAGE TypeFamilies      #-}
import           Data.Text (Text)
import           Yesod
data Person = Person
    { name :: Text
    , age  :: Int
    }
instance ToJSON Person where
    toJSON Person {..} = object
        [ "name" .= name
        , "age"  .= age
        ]
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
getHomeR :: Handler TypedContent
getHomeR = selectRep $ do
    provideRep $ return
        [shamlet|
            <p>Hello, my name is #{name} and I am #{age} years old.
        |]
    provideJson person
  where
    person@Person {..} = Person "Michael" 28
main :: IO ()
main = warp 3000 App

provideJson is similarly trivial, in this case provideRep . returnJson.

New datatypes

Let’s say I’ve come up with some new data format based on using Haskell’s Show instance; I’ll call it “Haskell Show”, and give it a mime type of text/haskell-show. And let’s say that I decide to include this representation from my web app. How do I do it? For a first attempt, let’s use the TypedContent datatype directly.

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE QuasiQuotes       #-}
{-# LANGUAGE TemplateHaskell   #-}
{-# LANGUAGE TypeFamilies      #-}
import           Data.Text (Text)
import           Yesod
data Person = Person
    { name :: Text
    , age  :: Int
    }
    deriving Show
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
mimeType :: ContentType
mimeType = "text/haskell-show"
getHomeR :: Handler TypedContent
getHomeR =
    return $ TypedContent mimeType $ toContent $ show person
  where
    person = Person "Michael" 28
main :: IO ()
main = warp 3000 App

There are a few important things to note here.

• We’ve used the toContent function. This is a typeclass function that can convert a number of data types to raw data ready to be sent over the wire. In this case, we’ve used the instance for String, which uses UTF8 encoding. Other common data types with instances are Text, ByteString, Html, and aeson’s Value.

• We’re using the TypedContent constructor directly. It takes two arguments: a mime type, and the raw content. Note that ContentType is simply a type alias for a strict ByteString.

That’s all well and good, but it bothers me that the type signature for getHomeR is so uninformative. Also, the implementation of getHomeR looks pretty boilerplate. I’d rather just have a datatype representing “Haskell Show” data, and provide some simple means of creating such values. Let’s try this on for size:

{-# LANGUAGE ExistentialQuantification #-}
{-# LANGUAGE OverloadedStrings         #-}
{-# LANGUAGE QuasiQuotes               #-}
{-# LANGUAGE TemplateHaskell           #-}
{-# LANGUAGE TypeFamilies              #-}
import           Data.Text (Text)
import           Yesod
data Person = Person
    { name :: Text
    , age  :: Int
    }
    deriving Show
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
mimeType :: ContentType
mimeType = "text/haskell-show"
data HaskellShow = forall a. Show a => HaskellShow a
instance ToContent HaskellShow where
    toContent (HaskellShow x) = toContent $ show x
instance ToTypedContent HaskellShow where
    toTypedContent = TypedContent mimeType . toContent
getHomeR :: Handler HaskellShow
getHomeR =
    return $ HaskellShow person
  where
    person = Person "Michael" 28
main :: IO ()
main = warp 3000 App

The magic here lies in two typeclasses. As we mentioned before, ToContent tells how to convert a value into a raw response. In our case, we would like to show the original value to get a String, and then convert that String into the raw content. Often times, instances of ToContent will build on each other in this way.

ToTypedContent is used internally by Yesod, and is called on the result of all handler functions. As you can see, the implementation is fairly trivial, simply stating the mime type and then calling out to toContent.

Finally, let’s make this a bit more complicated, and get this to play well with selectRep.

{-# LANGUAGE ExistentialQuantification #-}
{-# LANGUAGE OverloadedStrings         #-}
{-# LANGUAGE QuasiQuotes               #-}
{-# LANGUAGE RecordWildCards           #-}
{-# LANGUAGE TemplateHaskell           #-}
{-# LANGUAGE TypeFamilies              #-}
import           Data.Text (Text)
import           Yesod
data Person = Person
    { name :: Text
    , age  :: Int
    }
    deriving Show
instance ToJSON Person where
    toJSON Person {..} = object
        [ "name" .= name
        , "age"  .= age
        ]
data App = App
mkYesod "App" [parseRoutes|
/ HomeR GET
|]
instance Yesod App
mimeType :: ContentType
mimeType = "text/haskell-show"
data HaskellShow = forall a. Show a => HaskellShow a
instance ToContent HaskellShow where
    toContent (HaskellShow x) = toContent $ show x
instance ToTypedContent HaskellShow where
    toTypedContent = TypedContent mimeType . toContent
instance HasContentType HaskellShow where
    getContentType _ = mimeType
getHomeR :: Handler TypedContent
getHomeR = selectRep $ do
    provideRep $ return $ HaskellShow person
    provideJson person
  where
    person = Person "Michael" 28
main :: IO ()
main = warp 3000 App

The important addition here is the HasContentType instance. This may seem redundant, but it serves an important role. We need to be able to determine the mime type of a possible representation before creating that representation. ToTypedContent only works on a concrete value, and therefore can’t be used before creating the value. getContentType instead takes a proxy value, indicating the type without providing anything concrete.

If you want to provide a representation for a value that doesn’t have a HasContentType instance, you can use the provideRepType function, which requires you to explicitly state the mime type present.

Other request headers

There are a great deal of other request headers available. Some of them only affect the transfer of data between the server and client, and should not affect the application at all. For example, Accept-Encoding informs the server which compression schemes the client understands, and Host informs the server which virtual host to serve up.

Other headers do affect the application, but are automatically read by Yesod. For example, the Accept-Language header specifies which human language (English, Spanish, German, Swiss-German) the client prefers. See the i18n chapter for details on how this header is used.

Stateless

I’ve saved this section for the last, not because it is less important, but rather because there are no specific features in Yesod to enforce this.

HTTP is a stateless protocol: each request is to be seen as the beginning of a conversation. This means, for instance, it doesn’t matter to the server if you requested five pages previously, it will treat your sixth request as if it’s your first one.

On the other hand, some features on websites won’t work without some kind of state. For example, how can you implement a shopping cart without saving information about items in between requests?

The solution to this is cookies, and built on top of this, sessions. We have a whole section addressing the sessions features in Yesod. However, I cannot stress enough that this should be used sparingly.

Let me give you an example. There’s a popular bug tracking system that I deal with on a daily basis which horribly abuses sessions. There’s a little drop-down on every page to select the current project. Seems harmless, right? What that dropdown does is set the current project in your session.

The result of all this is that clicking on the “view issues” link is entirely dependent on the last project you selected. There’s no way to create a bookmark to your “Yesod” issues and a separate link for your “Hamlet” issues.

The proper RESTful approach to this is to have one resource for all of the Yesod issues and a separate one for all the Hamlet issues. In Yesod, this is easily done with a route definition like:

/                    ProjectsR      GET
/projects/#ProjectID ProjectIssuesR GET
/issues/#IssueID     IssueR         GET

Be nice to your users: proper stateless architecture means that basic features like bookmarks, permalinks and the back/forward button will always work.

Summary

Yesod adheres to the following tenets of REST:

• Use the correct request method.

• Each resource should have precisely one URL.

• Allow multiple representations of data on the same URL.

• Inspect request headers to determine extra information about what the client wants.

This makes it easy to use Yesod not just for building websites, but for building APIs. In fact, using techniques such as selectRep/provideRep, you can serve both a user-friendly, HTML page and a machine-friendly, JSON page from the same URL.