Go With Go(Lang): Features and Testing Frameworks

Test Automation using GoLang

In the Part 1 of this article Go With Go(Lang): Why Go? we discussed how to select programming language for test automation and why Go is a strong contestant. In this part we will take a look at couple more exciting features that make automation even easier and take a brief overview of most popular Go testing frameworks. 

My favorite GoLang feature is goroutines. Often automated tests need to wait for something to happen. The wait time may differ, it can be milliseconds or seconds, even minutes. Such situations are common when product under test posts something to messaging services, either built in house or external (Kafka, PubSub). It occurs when system software under test is built using microservices architecture. It also happens when test queries huge data sets(and big data is everywhere now) or downloads a big file. It happens everywhere where we see asynchronous or event-driven behavior.

The most common mistake to handle such scenarios is using sleeps. This is big NO in test automation. It’s anti-pattern. Sleeps make automated tests either slow or flaky, or both. If you have them in your test just get rid of them as soon as possible.

Instead of sleeps automated tests should wait for event to happen or for timeout, in other worlds they should be able to handle asynchronicity. Goroutines make implementing asynchronous logic easy. So what is it — goroutine? An official definition says that goroutine is a lightweight thread of execution. That’s all we need to know for the scope of this article.

There are couple features that make goroutine great for automation:

• You can spin many goroutines even on regular laptop and don’t even notice it (each goroutine uses only 2KB of memory). Moreover they use only the memory they need.
• It is easy to implement and communicate between goroutines using channels.
• goroutines are easy to use, no advance coding skills needed

Last one is very important, you don’t need to be a Go guru to use goroutines and implement multithreading in your tests. All you need to know is covered in basic go tutorial (don't get me wrong if you write a production code you need to read and learn more about concurrency before even starting). 

I used Java and Python before to automate complex test cases and implementing asynchronous, event-driven logic(read multithreading) in these languages requires advanced coding skills. It is much easier if you use Go. 

In the example below test verifies that a message created by hitting an API endpoint is posted to Kafka queue. Test should pass if message we sent is found in the queue, and fail if it is not or timeout has been reached. 

func TestPostToKafkaQueue(t *testing.T) {
    var waitGroup sync.WaitGroup
    //create and post message to Kafka queue
    resp, _ := buildAndSendMessage(url, id, true)
    //fail test if creating message resulted in error
    assert.Equal(t, resp.Status, “200 OK”)
    //setting up a timeout. Test will stop when timeout is reached
    ctx, cancel := context.WithTimeout(context.Background(),   timeout)
    defer cancel()
    waitGroup.Add(1)
//this is our goroutine. In the code below we start separate      thread that will be constantly consuming messages from queue and checking that it is message that we expected to see
    go func(){
       result, err := consume(ctx, topic, true, topic, id)
       if err != nil {
          fmt.Println(“ERROR: Consuming error”, err)
          assert.FailNow(t, "Failed to consume message from queue")
       }
       fmt.Println(“Found event in kafka topic:”, result)
       waitGroup.Done()
       ctx.Done()
       assert.Equal(t, result, expected_result)
       }()
    waitGroup.Wait()
}

There are two other features that make Go extremely well suited for test automation:

• No Exceptions. Ability to handle errors by yourself is a great plus for writing automated tests and frameworks. It gives us tool to pinpoint what exactly went wrong and makes debugging of failed test cases a piece of cake. For example, we automated end-to-end test that fully simulates data flow in production system. The test was running for several hours and data was processed by different system components. When it failed we had very specific error messages and were able to find a failure root cause in seconds.
• Multiple return values. You can return error and partial response, test result and message that explains it. Everything. The most used case of this feature is returning some object and error.

Here is an example of using multiple responses in the test that hits /execute API endpoint and verifies that it returns 200 OK. In case if request fails it hits google.com and returns response from it and error from request to /execute (please don’t ask me why someone will have such weird test, maybe we want to check that we still able to connect to Google, which became synonym of having Internet connection)

func httpRequestMultipleResponseValues(url string) (*http.Response, error) {
  client := &http.Client{}
  r, err := http.NewRequest(http.MethodGet, url)
  r.Close = true
  if err != nil {
    return nil, err
  }
response, err := client.Do(r)
  if err != nil {
    //if error is returned when we hit 'url' we will hit google  and return response from google and 'url' error
     r, _ := http.NewRequest(http.MethodGet, "https://google.com/")
     responseAlt, _ := client.Do(r)
     return responseAlt, err
  }
//in case of positive scenario we return /execute response and nil as an error
  return response, nil
}
func TestMultipleReturns(t *testing.T) {
  url := fmt.Sprint("https://localhost:8080/execute")
  resp, err := httpRequestMultipleResponseValues(url)
  if err != nil {
     if resp.Status != http.StatusOK {
     fmt.Println("Additional requests failed")
     }
     assert.FailNow(t, "Test Failed!")
  }
  assert.Equal(t, resp.Status, "200 OK")
}

Now let’s talk about tooling. First of all there is no need for web framework for Go. HTTP, JSON, HTML templating support is built in language natively which is a huge plus. 

There are multiple testing frameworks that can help with organizing tests, providing additional assets, reporting functionality and a lot of other useful features. One of them is GoConvey. It is BDD framework for gophers. It allows to keep test description and test code in one place. GoConvey works with go test, can be used in the terminal or browser providing great reports. It takes less than minute including download time to start using it. Unfortunately the development of GoConvey has stopped in August 2017. 

func TestExecute_POST(t *testing.T) {
Convey("Given call to /execute EP of test API", t, func() {
   resp, err := httpRequest("https://localhost:8080/execute", url)
   Convey("When successful response returned", func() {
      if err != nil {
        fmt.Println(“Requests failed”, err)
        So(true, ShouldEqual, false)
      }
      fmt.Println(“Response:”, resp.Status)
      Convey("The response HTTP status code should be 200", func() {
        So(resp.Status, ShouldContainSubstring, "200")
    })
   })
})
}

Another, more popular BDD-style Go testing framework is Ginkgo. It is best paired with the Gomega matcher library but is designed to be matcher-agnostic. Ginkgo has more complicated setup than GoConvey, but it has better test organization, more features and it is more mature. Ginkgo is built on top of the testing package so it is compatible with other tests even if they do not use Ginkgo. 

Last release — August 2018. If I was selecting framework for automation today I would choose Ginkgo.

Ginkgo claims that it supports testing asynchronicity, but I would not say that this is Ginkgo feature, it is just regular GoLang routines nicely wrapped.

It("should post to the channel, eventually", func(done Done) {
 c := make(chan string, 0)
go DoSomething(c)
 Expect(<-c).To(ContainSubstring(“Done!”))
 close(done)
}, 0.2)

Here is an example of Ginkgo test (I took it from Ginkgo documentation)

func TestApi(t *testing.T) {
   RegisterFailHandler(Fail)
   RunSpecs(t, "Api Suite")
}
var _ = Describe("Api", func() {
    Describe("Test Convey APIs — execute ep", func() {
    resp, _ := httpRequest(“http://localhost:8080/execute")
     Context("Hit execute EP", func() {
       It("Should get response 200 OK", func() {
          Expect(resp.Status).To(Equal(“200 OK”))
       })
     }) 
    })
})

I think that Go is mostly fit for backend testing, but there is universal WebDriver client for Go — Agouti. I have never used it personally but I believe it has a future. Agouti provides Gomega matchers, works with Ginkgo and with Spec (another BBD test organizer). Last release was in March 2018 so it’s development is pretty active.

There are multiple GoLang testing tools for load testing. The most mature of them is Vegeta, HTTP load testing tool. It can be used both as a command line utility and as a library and is open source as all other tools mentioned in this article. Last release was in October 2018.

Note: All release dates I mentioned are actual as of December 2018. 

Before you go I just want to emphasize couple points.

• testing lib is enough to create unit, integration and e2e tests
• go test always runs your test no matter what framework you pick up
• there are many testing frameworks and they all are open source
• asynchronous, event driven tests are really easy to build
• one framework — all tests (api, load, backend end-to-end)

And last one — the point of this article is not to convince everybody to use GoLang for automation, but to make testing community more open for something new. There are many tools and languages that might fit your automation needs much better than most popular. Choose your language and tools wisely and have a happy automation!

Resources I used to prepare this article:

• Awesome Go — list of awesome Go frameworks, libraries and software.
• Basic testing patterns in Go by Sebastian Dahlgren
• The beauty of Go by Kanishk Dudeja
• Why Go is skyrocketing in popularity by Jeff Rouse
• Why should you learn Go? by Keval Patel
• 5 Reasons Why We switched from Python To Go by Tigran Bayburtsyan
• GopherCon 2018 — From Prototype to Production: Lessons from Building and Scaling Reddit’s Ad Serving Platform with Go by Deval Shah
• 5 simple tips and tricks for writing unit tests in #golang by Mat Ryer