An Approach to Product Quality for Microservices: Introducing Testing-as-a-Service

Development, Testing, and Operations. Everyone knows the story. Developers spend good chunks of time writing code then pass it on to QA teams that test this code and after a slew of back-n-forth for bug fixes etc, the Operations teams pick the code to deploy it. This means releasing new software can take an exorbitant amount of time. A proposed solution to help quicken the software development lifecycle and enable the adoption of a DevOps culture is from a practice where software engineers are responsible for ensuring the quality of their code by using testing services provided from other team members or, Testing-as-a-Service.

Change is Already Happening

Before delving into this topic though, let’s take a brief look back at how things were done before and compare them to the direction the industry is headed now. Originally, applications were built as monolithic applications where all of the components were packaged in one large tightly-coupled entity but this leads to a few problems.

The first is that any change to this type of system frustratingly requires the whole monolith to be reconstructed and deployed. This is especially the case today as more of our applications operate in the cloud. Another issue comes up concerning scaling. In order to scale a monolith application, engineers must extend the capacity of the entire app instead of being able to simply scale the parts where expansion is needed the most.

In response to this, many IT operations are adopting a microservices-based approach which differs completely from monoliths in that the components are loosely coupled and can be scaled and deployed independently from one another. With the introduction of containers as well, microservice architectures have gained a ton of momentum lately.

Inefficiencies in CD


A typical SDLC practice for microservices based platforms is to embrace CI/CD for efficient deployments. In a typical Continuous Deployment (CD) practice the part that is the most cumbersome and takes up an enormous amount of time comes during the various testing stages. So we wanted to investigate this a bit deeper and discovered one aspect stood out amongst the rest.

Usually there is a dedicated QA or test team that is collaborating with a developer after they finish writing the code for some specific feature or service. Communication is key here because whoever is testing the code must be briefed by the developer and quickly become an expert on everything about that particular code (i.e. how it works, what it is supposed to do, how it is supposed to run).

This is where we propose a three-pronged approach to combat the above and these ultimately comprises this whole idea of testing-as-a-service.

A Three-Pronged Approach to Testing-as-a-Service

Testing as a Microservice

So what does this mean? It really begins with a simple fundamental change: treating testing as a software engineering function. This is not meant to imply that organizations should remove their QA/test teams, but should instead change the way in which these teams operate. Rather than having one team write code and another team specifically set up to test that code, organizations can transform these groups who have incredibly similar skill sets and create two distinct but interchangeable roles: a feature developer and a test harness developer.

The feature developer’s mission is to design a feature and deliver it to the end user. The test developer’s task is to write test harnesses and provide these to the feature developer. The feature developer is responsible for writing test plans and cases and then executing the automated tests themselves with the test harnesses constructed by the test developer. In a way, the test developer’s customer is the feature developer, while the feature developer’s customer remains the end user. The amount of time saved in the testing phase of the SDLC can be visualized in the pictures below when implementing this kind of culture change in your organization.

Traditional Testing

Testing as a Microservice

Micro Test Harness

Despite the adoption of microservices by a lot of companies, often times a single test framework is still used. It is basically counterintuitive/counterproductive for organizations to use these monolithic frameworks that are ridden with bugs and overloaded with huge amounts of code to test microservices. If engineers can break up a monolithic architecture into microservices, then why not break up a large single test framework into micro test harnesses? Now they have a library of test code that is completely adapted for one or multiple specific microservices.

Legacy Test Framework

Micro Test Harness

Monitoring and Alerting as a Service

The way alerting has been done for a while now is a test tool like Jenkins is used to automatically run tests and provide engineers the proper information to know whether or not a change has integrated smoothly with some feature they wrote. The challenge here is playing the waiting game. It can take anywhere from 20 minutes to 20 hours to get feedback depending on the amount of test case scenarios put in place.

So, what if teams could obtain this same feedback in near real-time? Using a real-time monitoring system like SignalFx, developers can constantly monitor an application or a platform so that when any changes are made to a feature, anomalies will be detected immediately which will trigger an alert so they can instantly be aware of any problems.

Traditional Alerting

Monitoring and Alerting as a Service

Benefits of Testing-as-a-Service

Blazing Fast Release Cycles. Since the transfer of information about a feature between developer and tester has essentially been removed, the feature developer is now able to ship and deploy code much faster.

Enable a DevOps Culture. Now software developers can be plugged into either a feature or test developer role which helps add a great unintended benefit of boosting overall company morale.


With the adoption of microservices in today’s cloud architecture, classic software development practices are slowly changing and adapting to this new environment. Testing a whole concoction of code all at once in today’s world means time and as always — time means everything!

Follow us on Twitter »

About the authors

Ram Jothikumar

Ram has been in the software industry for over 15 years, with a background in DevOps/Tools/Automation. At SignalFx, Ram leads the Infrastructure and Engineering Services team, which is responsible for building tools and frameworks that empower rapid deployment of quality code while maintaining a high level of site reliability. Before SignalFx, he led the NSX Scalability/Performance team at Nicira and VMware.

Enjoyed this blog post? Sign up for our blog updates