Go microservices, part 4 - testing and mocking with GoConvey

03 March 2017 // Erik Lupander

How should one approach testing of microservices? Are there any unique challenges one needs to take into account when establishing a testing strategy for this particular domain? In part 4 of this blog series, we will take a look at this topic.

  • The subject of testing microservices in the unit context
  • Write unit tests in the BDD-style of GoConvey
  • Introduce a mocking technique
    Since this part won’t change the core service in any way, no benchmarks this time.

Introduction to testing microservices

First of all, one should keep the principles of the testing pyramid in mind.

pyramid

Unit tests should form the bulk of your tests as integration-, e2e-, system- and acceptance tests are increasingly expensive to develop and maintain.

Secondly - microservices definitely offers some unique testing challenges and part of those is just as much about using sound principles when establishing a software architecture for your service implementations as the actual tests. That said - I think many of the microservice-specifics are beyond the realm of traditional unit tests which is what we’re be going to deal with in this part of the blog series.

Anyway, a few bullets I’d like to stress:

  • Unit test as usual - there’s nothing magic with your business logic, converters, validators etc. just because they’re running in the context of a microservice.
  • Integration components such as clients for communicating with other services, sending messages, accessing databases etc. should be designed with dependency injection and mockability taken into account.
  • A lot of the microservice specifics - accessing configuration, talking to other services, resilience testing etc. can be quite difficult to unit-test without spending ridiculous amounts of time writing mocks for a rather small value. Save those kind of tests to integration-like tests where you actually boot dependent services as Docker containers in your test code. It’ll provide greater value and will probably be easier to get up and running as well.

Source code

As before, you may checkout the appropriate branch from the cloned repository to get the completed source of this part up front:

  1. git checkout P4

Introduction

Unit testing in Go follows some idiomatic patterns established by the Go authors. Test source files are identified by naming conventions. If we, for example, want to test things in our handlers.go file, we create the file handlers_test.go in the same directory. So let’s do that.

We’ll start with a sad path test that asserts that we get a HTTP 404 if we request an unknown path:

  1. package service
  3. import (
  4.         . "github.com/smartystreets/goconvey/convey"
  5.         "testing"
  6.         "net/http/httptest"
  7. )
  9. func TestGetAccountWrongPath(t *testing.T) {
  11.         Convey("Given a HTTP request for /invalid/123", t, func() {
  12.                 req := httptest.NewRequest("GET", "/invalid/123", nil)
  13.                 resp := httptest.NewRecorder()
  15.                 Convey("When the request is handled by the Router", func() {
  16.                         NewRouter().ServeHTTP(resp, req)
  18.                         Convey("Then the response should be a 404", func() {
  19.                                 So(resp.Code, ShouldEqual, 404)
  20.                         })
  21.                 })
  22.         })
  23. }

This test shows the “Given-When-Then” Behaviour-driven structure of GoConvey and also the “So A ShouldEqual B” assertion style. It also introduces usage of the httptest package where we use it to declare a request object as well as a response object we can perform asserts on in a convenient manner.

Run it by moving to the root “accountservice” folder and type:

  1. > go test ./...
  2. ?       github.com/callistaenterprise/goblog/accountservice    [no test files]
  3. ?       github.com/callistaenterprise/goblog/accountservice/dbclient    [no test files]
  4. ?       github.com/callistaenterprise/goblog/accountservice/model    [no test files]
  5. ok      github.com/callistaenterprise/goblog/accountservice/service    0.012s

Wonder about ./…? It’s us telling go test to run all tests in the current folder and all subfolders. We could also go into the /service folder and type go test which then would only execute tests within that folder.

Since the “service” package is the only one with test files in it the other packages report that there are no tests there. That’s fine, at least for now!

Mocking

The test we created above doesn’t need to mock anything since the actual call won’t reach our GetAccount func that relies on the DBClient we created in part 3. For a happy-path test where we actually want to return something, we somehow need to mock the client we’re using to access the BoltDB. There are a number of strategies on how to do mocking in Go. I’ll show my favourite using the stretchr/testify/mock package.

In the /dbclient folder, create a new file called mockclient.go that will be an implementation of our IBoltClient interface.

  1. package dbclient
  3. import (
  4.         "github.com/stretchr/testify/mock"
  5.         "github.com/callistaenterprise/goblog/accountservice/model"
  6. )
  8. // MockBoltClient is a mock implementation of a datastore client for testing purposes.
  9. // Instead of the bolt.DB pointer, we're just putting a generic mock object from
  10. // strechr/testify
  11. type MockBoltClient struct {
  12.         mock.Mock
  13. }
  15. // From here, we'll declare three functions that makes our MockBoltClient fulfill the interface IBoltClient that we declared in part 3.
  16. func (m *MockBoltClient) QueryAccount(accountId string) (model.Account, error) {
  17.         args := m.Mock.Called(accountId)
  18.         return args.Get(0).(model.Account), args.Error(1)
  19. }
  21. func (m *MockBoltClient) OpenBoltDb() {
  22.         // Does nothing
  23. }
  25. func (m *MockBoltClient) Seed() {
  26.         // Does nothing
  27. }

MockBoltClient can now function as our explicitly tailored programmable mock. As stated above, this code implicitly implements the IBoltClient interface since the MockBoltClient struct has functions attached that matches the signature of all functions declared in the IBoltClient interface.

If you dislike writing boilerplate code for your mocks, I recommend taking a look at Mockery which can generate mocks for any Go interface.

The body of the QueryAccount function may seem a bit weird, but it is simply how strechr/testify provides us with a programmable mock where we have full control of its internal mechanics.

Programming the mock

Let’s create another test function in handlers_test.go:

  1. func TestGetAccount(t *testing.T) {
  2.         // Create a mock instance that implements the IBoltClient interface
  3.         mockRepo := &dbclient.MockBoltClient{}
  5.         // Declare two mock behaviours. For "123" as input, return a proper Account struct and nil as error.
  6.         // For "456" as input, return an empty Account object and a real error.
  7.         mockRepo.On("QueryAccount", "123").Return(model.Account{Id:"123", Name:"Person_123"}, nil)
  8.         mockRepo.On("QueryAccount", "456").Return(model.Account{}, fmt.Errorf("Some error"))
  10.         // Finally, assign mockRepo to the DBClient field (it's in _handlers.go_, e.g. in the same package)
  11.         DBClient = mockRepo
  12.         ...
  13. }

Next, replace the … above with another GoConvey test:

  1. Convey("Given a HTTP request for /accounts/123", t, func() {
  2.         req := httptest.NewRequest("GET", "/accounts/123", nil)
  3.         resp := httptest.NewRecorder()
  5.         Convey("When the request is handled by the Router", func() {
  6.                 NewRouter().ServeHTTP(resp, req)
  8.                 Convey("Then the response should be a 200", func() {
  9.                         So(resp.Code, ShouldEqual, 200)
  11.                         account := model.Account{}
  12.                         json.Unmarshal(resp.Body.Bytes(), &account)
  13.                         So(account.Id, ShouldEqual, "123")
  14.                         So(account.Name, ShouldEqual, "Person_123")
  15.                 })
  16.         })
  17. })

This test performs a request for the known path /accounts/123 which our mock knows about. In the “When” block, we assert HTTP status, unmarshal the returned Account struct and asserts that the fields match what we asked the mock to return.

What I like about GoConvey and the Given-When-Then way of writing tests is that they are really easy to read and have great structure.

We might as well add another sad path where we request /accounts/456 and assert that we get a HTTP 404 back:

  1. Convey("Given a HTTP request for /accounts/456", t, func() {
  2.         req := httptest.NewRequest("GET", "/accounts/456", nil)
  3.         resp := httptest.NewRecorder()
  5.         Convey("When the request is handled by the Router", func() {
  6.                 NewRouter().ServeHTTP(resp, req)
  8.                 Convey("Then the response should be a 404", func() {
  9.                         So(resp.Code, ShouldEqual, 404)
  10.                 })
  11.         })
  12. })

Finish by running our tests again:

  1. > go test ./...
  2. ?       github.com/callistaenterprise/goblog/accountservice    [no test files]
  3. ?       github.com/callistaenterprise/goblog/accountservice/dbclient    [no test files]
  4. ?       github.com/callistaenterprise/goblog/accountservice/model    [no test files]
  5. ok      github.com/callistaenterprise/goblog/accountservice/service    0.026s

All green! GoConvey actually has an interactive GUI that can execute all tests everytime we save a file. I won’t go into detail about it but looks like this and also provides stuff like automatic code coverage reports:

goconvey-goblog.png

Other types of tests

These GoConvey tests are unit tests though the BDD-style of writing them isn’t everyone’s cup of tea. There are many other testing frameworks for Golang, a quick search using your favourite search engine will probably yield many interesting options.

If we move up the testing pyramid we’ll want to write integration tests and finally acceptance-style tests perhaps using something such as cucumber. That’s out of scope for now but we can hopefully return to the topic of writing integration tests later on where we’ll actually bootstrap a real BoltDB in our test code, perhaps by using the Go Docker Remote API and a pre-baked BoltDB image.

Another approach to integration testing is automating deployment of the dockerized microservice landscape. See for example the blog post I wrote last year where I use a little Go program to boot all microservices given a .yaml specification, including the support services and then performing a few HTTP calls to the services to make sure the deployment is sound.

Summary

In this part we wrote our first unit tests, using the 3rd party GoConvey and stretchr/testify/mock libraries to help us. We’ll do more tests in later parts of the blog series.

In the next part, it’s time to finally get Docker Swarm up and running and deploy the microservice we’ve been working on into the swarm.