HTTP Server

You can find all the code for this chapter here

You have been asked to create a web server where users can track how many games players have won.

  • GET /players/{name} should return a number indicating the total number of wins
  • POST /players/{name} should record a win for that name, incrementing for every subsequent POST

We will follow the TDD approach, getting working software as quickly as we can and then making small iterative improvements until we have the solution. By taking this approach we

  • Keep the problem space small at any given time
  • Don’t go down rabbit holes
  • If we ever get stuck/lost, doing a revert wouldn’t lose loads of work.

Red, green, refactor

Throughout this book, we have emphasised the TDD process of write a test & watch it fail (red), write the minimal amount of code to make it work (green) and then refactor.

This discipline of writing the minimal amount of code is important in terms of the safety TDD gives you. You should be striving to get out of “red” as soon as you can.

Kent Beck describes it as:

Make the test work quickly, committing whatever sins necessary in process.

You can commit these sins because you will refactor afterwards backed by the safety of the tests.

What if you don’t do this?

The more changes you make while in red, the more likely you are to add more problems, not covered by tests.

The idea is to be iteratively writing useful code with small steps, driven by tests so that you don’t fall into a rabbit hole for hours.

Chicken and egg

How can we incrementally build this? We can’t GET a player without having stored something and it seems hard to know if POST has worked without the GET endpoint already existing.

This is where mocking shines.

  • GET will need a PlayerStore thing to get scores for a player. This should be an interface so when we test we can create a simple stub to test our code without needing to have implemented any actual storage code.
  • For POST we can spy on its calls to PlayerStore to make sure it stores players correctly. Our implementation of saving won’t be coupled to retrieval.
  • For having some working software quickly we can make a very simple in-memory implementation and then later we can create an implementation backed by whatever storage mechanism we prefer.

Write the test first

We can write a test and make it pass by returning a hard-coded value to get us started. Kent Beck refers this as “Faking it”. Once we have a working test we can then write more tests to help us remove that constant.

By doing this very small step, we can make the important start of getting an overall project structure working correctly without having to worry too much about our application logic.

To create a web server in Go you will typically call ListenAndServe.

  1. func ListenAndServe(addr string, handler Handler) error

This will start a web server listening on a port, creating a goroutine for every request and running it against a Handler.

  1. type Handler interface {
  2.     ServeHTTP(ResponseWriter, *Request)
  3. }

A type implements the Handler interface by implementing the ServeHTTP method which expects two arguments, the first is where we write our response and the second is the HTTP request that was sent to the server.

Let’s create a file named server_test.go and write a test for a function PlayerServer that takes in those two arguments. The request sent in will be to get a player’s score, which we expect to be "20".

  1. func TestGETPlayers(t *testing.T) {
  2.     t.Run("returns Pepper's score", func(t *testing.T) {
  3.         request, _ := http.NewRequest(http.MethodGet, "/players/Pepper", nil)
  4.         response := httptest.NewRecorder()
  6.         PlayerServer(response, request)
  8.         got := response.Body.String()
  9.         want := "20"
  11.         if got != want {
  12.             t.Errorf("got %q, want %q", got, want)
  13.         }
  14.     })
  15. }

In order to test our server, we will need a Request to send in and we’ll want to spy on what our handler writes to the ResponseWriter.

  • We use http.NewRequest to create a request. The first argument is the request’s method and the second is the request’s path. The nil argument refers to the request’s body, which we don’t need to set in this case.
  • net/http/httptest has a spy already made for us called ResponseRecorder so we can use that. It has many helpful methods to inspect what has been written as a response.

Try to run the test

./server_test.go:13:2: undefined: PlayerServer

Write the minimal amount of code for the test to run and check the failing test output

The compiler is here to help, just listen to it.

Create a file named server.go and define PlayerServer

  1. func PlayerServer() {}

Try again

  1. ./server_test.go:13:14: too many arguments in call to PlayerServer
  2.     have (*httptest.ResponseRecorder, *http.Request)
  3.     want ()

Add the arguments to our function

  1. import "net/http"
  3. func PlayerServer(w http.ResponseWriter, r *http.Request) {
  5. }

The code now compiles and the test fails

  1. === RUN   TestGETPlayers/returns_Pepper's_score
  2.     --- FAIL: TestGETPlayers/returns_Pepper's_score (0.00s)
  3.         server_test.go:20: got '', want '20'

Write enough code to make it pass

From the DI chapter, we touched on HTTP servers with a Greet function. We learned that net/http’s ResponseWriter also implements io Writer so we can use fmt.Fprint to send strings as HTTP responses.

  1. func PlayerServer(w http.ResponseWriter, r *http.Request) {
  2.     fmt.Fprint(w, "20")
  3. }

The test should now pass.

Complete the scaffolding

We want to wire this up into an application. This is important because

  • We’ll have actual working software, we don’t want to write tests for the sake of it, it’s good to see the code in action.
  • As we refactor our code, it’s likely we will change the structure of the program. We want to make sure this is reflected in our application too as part of the incremental approach.

Create a new main.go file for our application and put this code in

  1. package main
  3. import (
  4.     "log"
  5.     "net/http"
  6. )
  8. func main() {
  9.     handler := http.HandlerFunc(PlayerServer)
  10.     log.Fatal(http.ListenAndServe(":5000", handler))
  11. }

So far all of our application code has been in one file, however, this isn’t best practice for larger projects where you’ll want to separate things into different files.

To run this, do go build which will take all the .go files in the directory and build you a program. You can then execute it with ./myprogram.

http.HandlerFunc

Earlier we explored that the Handler interface is what we need to implement in order to make a server. Typically we do that by creating a struct and make it implement the interface by implementing its own ServeHTTP method. However the use-case for structs is for holding data but currently we have no state, so it doesn’t feel right to be creating one.

HandlerFunc lets us avoid this.

The HandlerFunc type is an adapter to allow the use of ordinary functions as HTTP handlers. If f is a function with the appropriate signature, HandlerFunc(f) is a Handler that calls f.

  1. type HandlerFunc func(ResponseWriter, *Request)

From the documentation, we see that type HandlerFunc has already implemented the ServeHTTP method. By type casting our PlayerServer function with it, we have now implemented the required Handler.

http.ListenAndServe(":5000"...)

ListenAndServe takes a port to listen on a Handler. If there is a problem the web server will return an error, an example of that might be the port already being listened to. For that reason we wrap the call in log.Fatal to log the error to the user.

What we’re going to do now is write another test to force us into making a positive change to try and move away from the hard-coded value.

Write the test first

We’ll add another subtest to our suite which tries to get the score of a different player, which will break our hard-coded approach.

  1. t.Run("returns Floyd's score", func(t *testing.T) {
  2.     request, _ := http.NewRequest(http.MethodGet, "/players/Floyd", nil)
  3.     response := httptest.NewRecorder()
  5.     PlayerServer(response, request)
  7.     got := response.Body.String()
  8.     want := "10"
  10.     if got != want {
  11.         t.Errorf("got %q, want %q", got, want)
  12.     }
  13. })

You may have been thinking

Surely we need some kind of concept of storage to control which player gets what score. It’s weird that the values seem so arbitrary in our tests.

Remember we are just trying to take as small as steps as reasonably possible, so we’re just trying to break the constant for now.

Try to run the test

  1. === RUN   TestGETPlayers/returns_Pepper's_score
  2.     --- PASS: TestGETPlayers/returns_Pepper's_score (0.00s)
  3. === RUN   TestGETPlayers/returns_Floyd's_score
  4.     --- FAIL: TestGETPlayers/returns_Floyd's_score (0.00s)
  5.         server_test.go:34: got '20', want '10'

Write enough code to make it pass

  1. //server.go
  2. func PlayerServer(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  5.     if player == "Pepper" {
  6.         fmt.Fprint(w, "20")
  7.         return
  8.     }
  10.     if player == "Floyd" {
  11.         fmt.Fprint(w, "10")
  12.         return
  13.     }
  14. }

This test has forced us to actually look at the request’s URL and make a decision. So whilst in our heads, we may have been worrying about player stores and interfaces the next logical step actually seems to be about routing.

If we had started with the store code the amount of changes we’d have to do would be very large compared to this. This is a smaller step towards our final goal and was driven by tests.

We’re resisting the temptation to use any routing libraries right now, just the smallest step to get our test passing.

r.URL.Path returns the path of the request which we can then use strings.TrimPrefix to trim away /players/ to get the requested player. It’s not very robust but will do the trick for now.

Refactor

We can simplify the PlayerServer by separating out the score retrieval into a function

  1. //server.go
  2. func PlayerServer(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  5.     fmt.Fprint(w, GetPlayerScore(player))
  6. }
  8. func GetPlayerScore(name string) string {
  9.     if name == "Pepper" {
  10.         return "20"
  11.     }
  13.     if name == "Floyd" {
  14.         return "10"
  15.     }
  17.     return ""
  18. }

And we can DRY up some of the code in the tests by making some helpers

  1. //server_test.go
  2. func TestGETPlayers(t *testing.T) {
  3.     t.Run("returns Pepper's score", func(t *testing.T) {
  4.         request := newGetScoreRequest("Pepper")
  5.         response := httptest.NewRecorder()
  7.         PlayerServer(response, request)
  9.         assertResponseBody(t, response.Body.String(), "20")
  10.     })
  12.     t.Run("returns Floyd's score", func(t *testing.T) {
  13.         request := newGetScoreRequest("Floyd")
  14.         response := httptest.NewRecorder()
  16.         PlayerServer(response, request)
  18.         assertResponseBody(t, response.Body.String(), "10")
  19.     })
  20. }
  22. func newGetScoreRequest(name string) *http.Request {
  23.     req, _ := http.NewRequest(http.MethodGet, fmt.Sprintf("/players/%s", name), nil)
  24.     return req
  25. }
  27. func assertResponseBody(t testing.TB, got, want string) {
  28.     t.Helper()
  29.     if got != want {
  30.         t.Errorf("response body is wrong, got %q want %q", got, want)
  31.     }
  32. }

However, we still shouldn’t be happy. It doesn’t feel right that our server knows the scores.

Our refactoring has made it pretty clear what to do.

We moved the score calculation out of the main body of our handler into a function GetPlayerScore. This feels like the right place to separate the concerns using interfaces.

Let’s move our function we re-factored to be an interface instead

  1. type PlayerStore interface {
  2.     GetPlayerScore(name string) int
  3. }

For our PlayerServer to be able to use a PlayerStore, it will need a reference to one. Now feels like the right time to change our architecture so that our PlayerServer is now a struct.

  1. type PlayerServer struct {
  2.     store PlayerStore
  3. }

Finally, we will now implement the Handler interface by adding a method to our new struct and putting in our existing handler code.

  1. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  2.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  3.     fmt.Fprint(w, p.store.GetPlayerScore(player))
  4. }

The only other change is we now call our store.GetPlayerScore to get the score, rather than the local function we defined (which we can now delete).

Here is the full code listing of our server

  1. //server.go
  2. type PlayerStore interface {
  3.     GetPlayerScore(name string) int
  4. }
  6. type PlayerServer struct {
  7.     store PlayerStore
  8. }
  10. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  11.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  12.     fmt.Fprint(w, p.store.GetPlayerScore(player))
  13. }

Fix the issues

This was quite a few changes and we know our tests and application will no longer compile, but just relax and let the compiler work through it.

./main.go:9:58: type PlayerServer is not an expression

We need to change our tests to instead create a new instance of our PlayerServer and then call its method ServeHTTP.

  1. //server_test.go
  2. func TestGETPlayers(t *testing.T) {
  3.     server := &PlayerServer{}
  5.     t.Run("returns Pepper's score", func(t *testing.T) {
  6.         request := newGetScoreRequest("Pepper")
  7.         response := httptest.NewRecorder()
  9.         server.ServeHTTP(response, request)
  11.         assertResponseBody(t, response.Body.String(), "20")
  12.     })
  14.     t.Run("returns Floyd's score", func(t *testing.T) {
  15.         request := newGetScoreRequest("Floyd")
  16.         response := httptest.NewRecorder()
  18.         server.ServeHTTP(response, request)
  20.         assertResponseBody(t, response.Body.String(), "10")
  21.     })
  22. }

Notice we’re still not worrying about making stores just yet, we just want the compiler passing as soon as we can.

You should be in the habit of prioritising having code that compiles and then code that passes the tests.

By adding more functionality (like stub stores) whilst the code isn’t compiling, we are opening ourselves up to potentially more compilation problems.

Now main.go won’t compile for the same reason.

  1. func main() {
  2.     server := &PlayerServer{}
  3.     log.Fatal(http.ListenAndServe(":5000", server))
  4. }

Finally, everything is compiling but the tests are failing

  1. === RUN   TestGETPlayers/returns_the_Pepper's_score
  2. panic: runtime error: invalid memory address or nil pointer dereference [recovered]
  3.     panic: runtime error: invalid memory address or nil pointer dereference

This is because we have not passed in a PlayerStore in our tests. We’ll need to make a stub one up.

  1. //server_test.go
  2. type StubPlayerStore struct {
  3.     scores map[string]int
  4. }
  6. func (s *StubPlayerStore) GetPlayerScore(name string) int {
  7.     score := s.scores[name]
  8.     return score
  9. }

A map is a quick and easy way of making a stub key/value store for our tests. Now let’s create one of these stores for our tests and send it into our PlayerServer.

  1. //server_test.go
  2. func TestGETPlayers(t *testing.T) {
  3.     store := StubPlayerStore{
  4.         map[string]int{
  5.             "Pepper": 20,
  6.             "Floyd":  10,
  7.         },
  8.     }
  9.     server := &PlayerServer{&store}
  11.     t.Run("returns Pepper's score", func(t *testing.T) {
  12.         request := newGetScoreRequest("Pepper")
  13.         response := httptest.NewRecorder()
  15.         server.ServeHTTP(response, request)
  17.         assertResponseBody(t, response.Body.String(), "20")
  18.     })
  20.     t.Run("returns Floyd's score", func(t *testing.T) {
  21.         request := newGetScoreRequest("Floyd")
  22.         response := httptest.NewRecorder()
  24.         server.ServeHTTP(response, request)
  26.         assertResponseBody(t, response.Body.String(), "10")
  27.     })
  28. }

Our tests now pass and are looking better. The intent behind our code is clearer now due to the introduction of the store. We’re telling the reader that because we have this data in a PlayerStore that when you use it with a PlayerServer you should get the following responses.

Run the application

Now our tests are passing the last thing we need to do to complete this refactor is to check if our application is working. The program should start up but you’ll get a horrible response if you try and hit the server at http://localhost:5000/players/Pepper.

The reason for this is that we have not passed in a PlayerStore.

We’ll need to make an implementation of one, but that’s difficult right now as we’re not storing any meaningful data so it’ll have to be hard-coded for the time being.

  1. //main.go
  2. type InMemoryPlayerStore struct{}
  4. func (i *InMemoryPlayerStore) GetPlayerScore(name string) int {
  5.     return 123
  6. }
  8. func main() {
  9.     server := &PlayerServer{&InMemoryPlayerStore{}}
  10.     log.Fatal(http.ListenAndServe(":5000", server))
  11. }

If you run go build again and hit the same URL you should get "123". Not great, but until we store data that’s the best we can do. It also didn’t feel great that our main application was starting up but not actually working. We had to manually test to see the problem.

We have a few options as to what to do next

  • Handle the scenario where the player doesn’t exist
  • Handle the POST /players/{name} scenario

Whilst the POST scenario gets us closer to the “happy path”, I feel it’ll be easier to tackle the missing player scenario first as we’re in that context already. We’ll get to the rest later.

Write the test first

Add a missing player scenario to our existing suite

  1. //server_test.go
  2. t.Run("returns 404 on missing players", func(t *testing.T) {
  3.     request := newGetScoreRequest("Apollo")
  4.     response := httptest.NewRecorder()
  6.     server.ServeHTTP(response, request)
  8.     got := response.Code
  9.     want := http.StatusNotFound
  11.     if got != want {
  12.         t.Errorf("got status %d want %d", got, want)
  13.     }
  14. })

Try to run the test

  1. === RUN   TestGETPlayers/returns_404_on_missing_players
  2.     --- FAIL: TestGETPlayers/returns_404_on_missing_players (0.00s)
  3.         server_test.go:56: got status 200 want 404

Write enough code to make it pass

  1. //server.go
  2. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  5.     w.WriteHeader(http.StatusNotFound)
  7.     fmt.Fprint(w, p.store.GetPlayerScore(player))
  8. }

Sometimes I heavily roll my eyes when TDD advocates say “make sure you just write the minimal amount of code to make it pass” as it can feel very pedantic.

But this scenario illustrates the example well. I have done the bare minimum (knowing it is not correct), which is write a StatusNotFound on all responses but all our tests are passing!

By doing the bare minimum to make the tests pass it can highlight gaps in your tests. In our case, we are not asserting that we should be getting a StatusOK when players do exist in the store.

Update the other two tests to assert on the status and fix the code.

Here are the new tests

  1. //server_test.go
  2. func TestGETPlayers(t *testing.T) {
  3.     store := StubPlayerStore{
  4.         map[string]int{
  5.             "Pepper": 20,
  6.             "Floyd":  10,
  7.         },
  8.     }
  9.     server := &PlayerServer{&store}
  11.     t.Run("returns Pepper's score", func(t *testing.T) {
  12.         request := newGetScoreRequest("Pepper")
  13.         response := httptest.NewRecorder()
  15.         server.ServeHTTP(response, request)
  17.         assertStatus(t, response.Code, http.StatusOK)
  18.         assertResponseBody(t, response.Body.String(), "20")
  19.     })
  21.     t.Run("returns Floyd's score", func(t *testing.T) {
  22.         request := newGetScoreRequest("Floyd")
  23.         response := httptest.NewRecorder()
  25.         server.ServeHTTP(response, request)
  27.         assertStatus(t, response.Code, http.StatusOK)
  28.         assertResponseBody(t, response.Body.String(), "10")
  29.     })
  31.     t.Run("returns 404 on missing players", func(t *testing.T) {
  32.         request := newGetScoreRequest("Apollo")
  33.         response := httptest.NewRecorder()
  35.         server.ServeHTTP(response, request)
  37.         assertStatus(t, response.Code, http.StatusNotFound)
  38.     })
  39. }
  41. func assertStatus(t testing.TB, got, want int) {
  42.     t.Helper()
  43.     if got != want {
  44.         t.Errorf("did not get correct status, got %d, want %d", got, want)
  45.     }
  46. }
  48. func newGetScoreRequest(name string) *http.Request {
  49.     req, _ := http.NewRequest(http.MethodGet, fmt.Sprintf("/players/%s", name), nil)
  50.     return req
  51. }
  53. func assertResponseBody(t testing.TB, got, want string) {
  54.     t.Helper()
  55.     if got != want {
  56.         t.Errorf("response body is wrong, got %q want %q", got, want)
  57.     }
  58. }

We’re checking the status in all our tests now so I made a helper assertStatus to facilitate that.

Now our first two tests fail because of the 404 instead of 200, so we can fix PlayerServer to only return not found if the score is 0.

  1. //server.go
  2. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  5.     score := p.store.GetPlayerScore(player)
  7.     if score == 0 {
  8.         w.WriteHeader(http.StatusNotFound)
  9.     }
  11.     fmt.Fprint(w, score)
  12. }

Storing scores

Now that we can retrieve scores from a store it now makes sense to be able to store new scores.

Write the test first

  1. //server_test.go
  2. func TestStoreWins(t *testing.T) {
  3.     store := StubPlayerStore{
  4.         map[string]int{},
  5.     }
  6.     server := &PlayerServer{&store}
  8.     t.Run("it returns accepted on POST", func(t *testing.T) {
  9.         request, _ := http.NewRequest(http.MethodPost, "/players/Pepper", nil)
  10.         response := httptest.NewRecorder()
  12.         server.ServeHTTP(response, request)
  14.         assertStatus(t, response.Code, http.StatusAccepted)
  15.     })
  16. }

For a start let’s just check we get the correct status code if we hit the particular route with POST. This lets us drive out the functionality of accepting a different kind of request and handling it differently to GET /players/{name}. Once this works we can then start asserting on our handler’s interaction with the store.

Try to run the test

  1. === RUN   TestStoreWins/it_returns_accepted_on_POST
  2.     --- FAIL: TestStoreWins/it_returns_accepted_on_POST (0.00s)
  3.         server_test.go:70: did not get correct status, got 404, want 202

Write enough code to make it pass

Remember we are deliberately committing sins, so an if statement based on the request’s method will do the trick.

  1. //server.go
  2. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  4.     if r.Method == http.MethodPost {
  5.         w.WriteHeader(http.StatusAccepted)
  6.         return
  7.     }
  9.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  11.     score := p.store.GetPlayerScore(player)
  13.     if score == 0 {
  14.         w.WriteHeader(http.StatusNotFound)
  15.     }
  17.     fmt.Fprint(w, score)
  18. }

Refactor

The handler is looking a bit muddled now. Let’s break the code up to make it easier to follow and isolate the different functionality into new functions.

  1. //server.go
  2. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  4.     switch r.Method {
  5.     case http.MethodPost:
  6.         p.processWin(w)
  7.     case http.MethodGet:
  8.         p.showScore(w, r)
  9.     }
  11. }
  13. func (p *PlayerServer) showScore(w http.ResponseWriter, r *http.Request) {
  14.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  16.     score := p.store.GetPlayerScore(player)
  18.     if score == 0 {
  19.         w.WriteHeader(http.StatusNotFound)
  20.     }
  22.     fmt.Fprint(w, score)
  23. }
  25. func (p *PlayerServer) processWin(w http.ResponseWriter) {
  26.     w.WriteHeader(http.StatusAccepted)
  27. }

This makes the routing aspect of ServeHTTP a bit clearer and means our next iterations on storing can just be inside processWin.

Next, we want to check that when we do our POST /players/{name} that our PlayerStore is told to record the win.

Write the test first

We can accomplish this by extending our StubPlayerStore with a new RecordWin method and then spy on its invocations.

  1. //server_test.go
  2. type StubPlayerStore struct {
  3.     scores   map[string]int
  4.     winCalls []string
  5. }
  7. func (s *StubPlayerStore) GetPlayerScore(name string) int {
  8.     score := s.scores[name]
  9.     return score
  10. }
  12. func (s *StubPlayerStore) RecordWin(name string) {
  13.     s.winCalls = append(s.winCalls, name)
  14. }

Now extend our test to check the number of invocations for a start

  1. //server_test.go
  2. func TestStoreWins(t *testing.T) {
  3.     store := StubPlayerStore{
  4.         map[string]int{},
  5.     }
  6.     server := &PlayerServer{&store}
  8.     t.Run("it records wins when POST", func(t *testing.T) {
  9.         request := newPostWinRequest("Pepper")
  10.         response := httptest.NewRecorder()
  12.         server.ServeHTTP(response, request)
  14.         assertStatus(t, response.Code, http.StatusAccepted)
  16.         if len(store.winCalls) != 1 {
  17.             t.Errorf("got %d calls to RecordWin want %d", len(store.winCalls), 1)
  18.         }
  19.     })
  20. }
  22. func newPostWinRequest(name string) *http.Request {
  23.     req, _ := http.NewRequest(http.MethodPost, fmt.Sprintf("/players/%s", name), nil)
  24.     return req
  25. }

Try to run the test

  1. ./server_test.go:26:20: too few values in struct initializer
  2. ./server_test.go:65:20: too few values in struct initializer

Write the minimal amount of code for the test to run and check the failing test output

We need to update our code where we create a StubPlayerStore as we’ve added a new field

  1. //server_test.go
  2. store := StubPlayerStore{
  3.     map[string]int{},
  4.     nil,
  5. }
  1. --- FAIL: TestStoreWins (0.00s)
  2.     --- FAIL: TestStoreWins/it_records_wins_when_POST (0.00s)
  3.         server_test.go:80: got 0 calls to RecordWin want 1

Write enough code to make it pass

As we’re only asserting the number of calls rather than the specific values it makes our initial iteration a little smaller.

We need to update PlayerServer‘s idea of what a PlayerStore is by changing the interface if we’re going to be able to call RecordWin.

  1. //server.go
  2. type PlayerStore interface {
  3.     GetPlayerScore(name string) int
  4.     RecordWin(name string)
  5. }

By doing this main no longer compiles

  1. ./main.go:17:46: cannot use InMemoryPlayerStore literal (type *InMemoryPlayerStore) as type PlayerStore in field value:
  2.     *InMemoryPlayerStore does not implement PlayerStore (missing RecordWin method)

The compiler tells us what’s wrong. Let’s update InMemoryPlayerStore to have that method.

  1. //main.go
  2. type InMemoryPlayerStore struct{}
  4. func (i *InMemoryPlayerStore) RecordWin(name string) {}

Try and run the tests and we should be back to compiling code - but the test is still failing.

Now that PlayerStore has RecordWin we can call it within our PlayerServer

  1. //server.go
  2. func (p *PlayerServer) processWin(w http.ResponseWriter) {
  3.     p.store.RecordWin("Bob")
  4.     w.WriteHeader(http.StatusAccepted)
  5. }

Run the tests and it should be passing! Obviously "Bob" isn’t exactly what we want to send to RecordWin, so let’s further refine the test.

Write the test first

  1. //server_test.go
  2. func TestStoreWins(t *testing.T) {
  3.     store := StubPlayerStore{
  4.         map[string]int{},
  5.         nil,
  6.     }
  7.     server := &PlayerServer{&store}
  9.     t.Run("it records wins on POST", func(t *testing.T) {
  10.         player := "Pepper"
  12.         request := newPostWinRequest(player)
  13.         response := httptest.NewRecorder()
  15.         server.ServeHTTP(response, request)
  17.         assertStatus(t, response.Code, http.StatusAccepted)
  19.         if len(store.winCalls) != 1 {
  20.             t.Fatalf("got %d calls to RecordWin want %d", len(store.winCalls), 1)
  21.         }
  23.         if store.winCalls[0] != player {
  24.             t.Errorf("did not store correct winner got %q want %q", store.winCalls[0], player)
  25.         }
  26.     })
  27. }

Now that we know there is one element in our winCalls slice we can safely reference the first one and check it is equal to player.

Try to run the test

  1. === RUN   TestStoreWins/it_records_wins_on_POST
  2.     --- FAIL: TestStoreWins/it_records_wins_on_POST (0.00s)
  3.         server_test.go:86: did not store correct winner got 'Bob' want 'Pepper'

Write enough code to make it pass

  1. //server.go
  2. func (p *PlayerServer) processWin(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  4.     p.store.RecordWin(player)
  5.     w.WriteHeader(http.StatusAccepted)
  6. }

We changed processWin to take http.Request so we can look at the URL to extract the player’s name. Once we have that we can call our store with the correct value to make the test pass.

Refactor

We can DRY up this code a bit as we’re extracting the player name the same way in two places

  1. //server.go
  2. func (p *PlayerServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  3.     player := strings.TrimPrefix(r.URL.Path, "/players/")
  5.     switch r.Method {
  6.     case http.MethodPost:
  7.         p.processWin(w, player)
  8.     case http.MethodGet:
  9.         p.showScore(w, player)
  10.     }
  11. }
  13. func (p *PlayerServer) showScore(w http.ResponseWriter, player string) {
  14.     score := p.store.GetPlayerScore(player)
  16.     if score == 0 {
  17.         w.WriteHeader(http.StatusNotFound)
  18.     }
  20.     fmt.Fprint(w, score)
  21. }
  23. func (p *PlayerServer) processWin(w http.ResponseWriter, player string) {
  24.     p.store.RecordWin(player)
  25.     w.WriteHeader(http.StatusAccepted)
  26. }

Even though our tests are passing we don’t really have working software. If you try and run main and use the software as intended it doesn’t work because we haven’t got round to implementing PlayerStore correctly. This is fine though; by focusing on our handler we have identified the interface that we need, rather than trying to design it up-front.

We could start writing some tests around our InMemoryPlayerStore but it’s only here temporarily until we implement a more robust way of persisting player scores (i.e. a database).

What we’ll do for now is write an integration test between our PlayerServer and InMemoryPlayerStore to finish off the functionality. This will let us get to our goal of being confident our application is working, without having to directly test InMemoryPlayerStore. Not only that, but when we get around to implementing PlayerStore with a database, we can test that implementation with the same integration test.

Integration tests

Integration tests can be useful for testing that larger areas of your system work but you must bear in mind:

  • They are harder to write
  • When they fail, it can be difficult to know why (usually it’s a bug within a component of the integration test) and so can be harder to fix
  • They are sometimes slower to run (as they often are used with “real” components, like a database)

For that reason, it is recommended that you research The Test Pyramid.

Write the test first

In the interest of brevity, I am going to show you the final refactored integration test.

  1. //server_integration_test.go
  2. package main
  4. import (
  5.     "net/http"
  6.     "net/http/httptest"
  7.     "testing"
  8. )
  10. func TestRecordingWinsAndRetrievingThem(t *testing.T) {
  11.     store := InMemoryPlayerStore{}
  12.     server := PlayerServer{&store}
  13.     player := "Pepper"
  15.     server.ServeHTTP(httptest.NewRecorder(), newPostWinRequest(player))
  16.     server.ServeHTTP(httptest.NewRecorder(), newPostWinRequest(player))
  17.     server.ServeHTTP(httptest.NewRecorder(), newPostWinRequest(player))
  19.     response := httptest.NewRecorder()
  20.     server.ServeHTTP(response, newGetScoreRequest(player))
  21.     assertStatus(t, response.Code, http.StatusOK)
  23.     assertResponseBody(t, response.Body.String(), "3")
  24. }
  • We are creating our two components we are trying to integrate with: InMemoryPlayerStore and PlayerServer.
  • We then fire off 3 requests to record 3 wins for player. We’re not too concerned about the status codes in this test as it’s not relevant to whether they are integrating well.
  • The next response we do care about (so we store a variable response) because we are going to try and get the player‘s score.

Try to run the test

  1. --- FAIL: TestRecordingWinsAndRetrievingThem (0.00s)
  2.     server_integration_test.go:24: response body is wrong, got '123' want '3'

Write enough code to make it pass

I am going to take some liberties here and write more code than you may be comfortable with without writing a test.

This is allowed! We still have a test checking things should be working correctly but it is not around the specific unit we’re working with (InMemoryPlayerStore).

If I were to get stuck in this scenario, I would revert my changes back to the failing test and then write more specific unit tests around InMemoryPlayerStore to help me drive out a solution.

  1. //in_memory_player_store.go
  2. func NewInMemoryPlayerStore() *InMemoryPlayerStore {
  3.     return &InMemoryPlayerStore{map[string]int{}}
  4. }
  6. type InMemoryPlayerStore struct {
  7.     store map[string]int
  8. }
  10. func (i *InMemoryPlayerStore) RecordWin(name string) {
  11.     i.store[name]++
  12. }
  14. func (i *InMemoryPlayerStore) GetPlayerScore(name string) int {
  15.     return i.store[name]
  16. }
  • We need to store the data so I’ve added a map[string]int to the InMemoryPlayerStore struct
  • For convenience I’ve made NewInMemoryPlayerStore to initialise the store, and updated the integration test to use it:
    1. //server_integration_test.go
    2. store := NewInMemoryPlayerStore()
    3. server := PlayerServer{store}
  • The rest of the code is just wrapping around the map

The integration test passes, now we just need to change main to use NewInMemoryPlayerStore()

  1. //main.go
  2. package main
  4. import (
  5.     "log"
  6.     "net/http"
  7. )
  9. func main() {
  10.     server := &PlayerServer{NewInMemoryPlayerStore()}
  11.     log.Fatal(http.ListenAndServe(":5000", server))
  12. }

Build it, run it and then use curl to test it out.

  • Run this a few times, change the player names if you like curl -X POST http://localhost:5000/players/Pepper
  • Check scores with curl http://localhost:5000/players/Pepper

Great! You’ve made a REST-ish service. To take this forward you’d want to pick a data store to persist the scores longer than the length of time the program runs.

  • Pick a store (Bolt? Mongo? Postgres? File system?)
  • Make PostgresPlayerStore implement PlayerStore
  • TDD the functionality so you’re sure it works
  • Plug it into the integration test, check it’s still ok
  • Finally plug it into main

Refactor

We are almost there! Lets take some effort to prevent concurrency errors like these

  1. fatal error: concurrent map read and map write

By adding mutexes, we enforce concurrency safety especially for the counter in our RecordWin function. Read more about mutexes in the sync chapter.

Wrapping up

http.Handler

  • Implement this interface to create web servers
  • Use http.HandlerFunc to turn ordinary functions into http.Handlers
  • Use httptest.NewRecorder to pass in as a ResponseWriter to let you spy on the responses your handler sends
  • Use http.NewRequest to construct the requests you expect to come in to your system

Interfaces, Mocking and DI

  • Lets you iteratively build the system up in smaller chunks
  • Allows you to develop a handler that needs a storage without needing actual storage
  • TDD to drive out the interfaces you need

Commit sins, then refactor (and then commit to source control)

  • You need to treat having failing compilation or failing tests as a red situation that you need to get out of as soon as you can.
  • Write just the necessary code to get there. Then refactor and make the code nice.
  • By trying to do too many changes whilst the code isn’t compiling or the tests are failing puts you at risk of compounding the problems.
  • Sticking to this approach forces you to write small tests, which means small changes, which helps keep working on complex systems manageable.