While the industry’s focus has been all about how containerization can help us build microservices, there’s a growing realization that their isolation, state control, and low overhead are useful in a lot of different applications. Leveraging containerization at various levels of the software testing and system validation process is a use case that is becoming popular throughout the industry. Containers provide a way to quickly stand up test scenarios, easily control system state, and to scale out and simulate lots of different types of loads on a system. Over the past few years, I’ve incorporated a number of containerized testing patterns into my development process; here is an overview of a few that I have found to be the most useful.
Basic Cases
Two very important fundamentals of building an effective testing strategy are controlling system state, and layered testing. These basic cases work well to facilitate both of these concepts.
Unit, Feature, and Service Tests
These types of tests generally do not require any external dependencies, or at least emphasize keeping external dependencies to a minimum. Even with the emphasis on little to no setup and fast results, it’s still valuable to run these tests inside an application or service container. The most common form of this is running tests in a continuous integration build system. Having these tests run in containers gives you the ability to run them in parallel, and makes it easy to port them over to different environments. If you’ve ever had your tests run in any tool like CircleCI, Travis, or even Jenkins, you’re probably already familiar with this approach.
Environment Setup and Teardown
In microservice architectures and in some monolithic systems, there will be one or more containerized services that need to be running in order to run integration and end-to-end tests. Using automated tests to stand up these containers to create a known base state and tear them down upon test completion is a common pattern. There are even libraries like Test Containers that make this process easy. It’s also easy to build your own libraries to control container setup if you require more customization. Additionally, you can use docker compose and compose file layering to easily setup and teardown test environments. Deciding on which approach to use is highly dependent on the testing problems you’re trying to solve, but I find it generally works well to use the tools that get you up and running quickly and then expand upon your test environments from there.
Containerized Tests
It’s also possible to create application containers that only include your tests and any runtime dependencies. This is an especially useful model for integration and end-to-end tests where you want to test as close to a production environment as possible. One of the primary benefits of this approach is that it’s easy to ensure test code and configurations are not accidentally included in the application under test. It also makes it easy to set up and run tests at higher levels of integration in different environments like a developer’s desktop, on-prem test cluster, or application deployment pipeline.
Controlling Test State
Understanding and controlling the state of an application or system under test is one of the most challenging aspects of testing at high levels of integration. Containerization makes this a bit easier because you can stop, start, and save service states to a new image. Adding this level of control allows you to add some interesting capabilities to your testing:
- For cases where you might want to test various database states, you can skip the seeding and stand up database containers that are already in the state you want to test. There is some initial setup cost to building these different states, but once they’re working, you can save them as images and always be sure your database is in the state that you want it to be in.
- You can also keep these old containers around if a test fails so you can analyze system state, or the state of data in the database. This can be extremely helpful in determining the cause of a failure. When using this technique, it is important to manage the lifecycle of your containers and clearly index your container to a specific test execution instance. It is also important to make sure old containers are cleaned up as soon as failure investigations are complete.
- If a new failure scenario is discovered, you can save the failing states as a new image that can be used in future tests to make sure that new scenario is consistently tested from that point on.
Testing with Different Runtimes
Testing all supported application runtimes, frameworks, and operating systems is tedious and time consuming. Teams work very hard to limit the number of supported technologies to a manageable level, but eventually the requirements of the market your product is trying to serve will win out. Leveraging the power of containers, you can easily automate a lot of interoperability testing:
- You can create containers that use different versions of your language interpreter, or versions of your binary that use different compiler versions or runtimes.
- You can create multiple container images for each version of a database engine your application supports.
- For Linux applications, you can use system containers like Linux Containers or LXD to test on different distros or releases. This does get trickier when testing non-Linux operating systems. In that case virtual machines and actual hardware are still the most reliable solutions out there.
Simulating Appliances and Single Board Computers
The types of testing applications that I’ve talked about so far have been well-suited for application containers. Application containers, mainly Docker and Podman are great for running a single application or service. System containers are containerization tools that still use Linux kernel namespacing for isolation, but run an entire system including service management tools like systemd. System containers are an excellent tool for simulating custom systems like SBCs or Linux appliances, because they have many of the capabilities of a virtual machine, with the flexibility and low overhead of containerization. Emulating these systems for testing purposes can allow you to test any services designed to run on them before the actual hardware is available. It can also help you scale and load test system components to hundreds or even thousands of machines without having to acquire and setup that amount of physical hardware.
There is no substitute for the actual hardware, so care must be taken to balance tests on simulated hardware and tests on the real thing.
Simulating Service Failures and Recovery
Whether you’re running a monolith with dependencies on external services or running a full microservice architecture, it’s important to test that service failures are handled gracefully, and that you receive the appropriate failure notifications. Using a containerized test environment you can strategically bring down different services to test how the system handles these failures as well as how the system recovers when services come back up. These types of tests are also useful for testing and proving compliance with any failover and recovery requirements for the application. You can also test different types of service startup and shutdown orders to make sure your application can handle the most common scenarios, as well as understanding which scenarios should be avoided.
Conclusion
While we usually think about containerization as a tool to build modern microsystem architectures, they can be an extremely useful tool for testing software at all levels of integration. The basic cases are now being widely used, but scaling, testing different runtimes, and even hardening your stack against service failures can all be improved by leveraging containerization. Additionally, in a testing context, you can bend, and even throw away a lot of the rules of containers. Make the images large, make them do more than one thing, and form them into tools that can help you make great quality software. Lightweight and isolated Linux environments are very flexible and with some imagination and computing resources, they can be used to bring your testing efforts to the next level.
Loved the article? Hated it? Didn’t even read it?
We’d love to hear from you.
