In the beginning of my developer career, I learned to use in-memory databases for running integration tests. The in-memory database made it easy to spin up a local database which is often needed for integration testing. However, there are some drawbacks with this approach.

A Clever Illusion

Using an in-memory database does not reflect the real database running in production. The local database may look and feel like the real deal, but it’s essentially an illusion, and a clever one.

The real database running in your production environment has vastly more underlying code paths, dependencies, frameworks, integrations and most likely handles concurrency differently.

As an example, many years ago I remember feeling overly happy and proud when I had completed a complex feature. It worked locally, and the integration tests I wrote to make sure the feature behaved as a expected, passed with flying colors. However, once the feature was deployed and received traffic, an optimistic lock occurred. I had written a test for this scenario, which passed locally. But in the production environment, something went wrong, something that the local illusion could not account for.

Breaking The Illusion

Instead of using in-memory databases or other such tricks, use e.g. Docker to spin up an ephemeral container running your database of choice. This provides you with the same foundation as your production environment. Now you can be more confident about the accuracy of your tests.

I have used two libraries in the past that facilitates the use of Docker containers in integration testing:

• https://github.com/testcontainers/testcontainers-java
• https://github.com/ory/dockertest

Before running your tests, you programmatically spin up a docker container and provide the connection details to your setup method that will provide the database object to the rest of your test code. When the tests are finished, you can remove the container.

Below you’ll find a short example how you can use dockertest by ory in your Go environment.

Create a TestMain:

1func TestMain(m *testing.M) {
2	// Docker setup here
3}

Create a new pool object that connects to the default docker socket:

1pool, err := dockertest.NewPool("")
2if err != nil {
3	log.Fatalf("Could not construct pool: %s", err)
4}

Define the image you want:

1resource, err := pool.Run("mysql", "5.7", []string{"MYSQL_ROOT_PASSWORD=secret"})
2if err != nil {
3	log.Fatalf("Could not start resource: %s", err)
4}

Wait for the database to become available:

 1if err := pool.Retry(func() error {
 2	var err error
 3	db, err = sql.Open("mysql", fmt.Sprintf("root:secret@(localhost:%s)/mysql", resource.GetPort("3306/tcp")))
 4	if err != nil {
 5		return err
 6	}
 7	return db.Ping()
 8}); err != nil {
 9	log.Fatalf("Could not connect to database: %s", err)
10}

Run your tests!

1code := m.Run()
2
3
4if err := pool.Purge(resource); err != nil {
5	log.Fatalf("Could not purge resource: %s", err)
6}
7
8os.Exit(code)

When all tests have finished, the container will be removed.

Doesn’t Get Easier Than That!

That’s it! Now go and update your development environment and supercharge your integration tests!