Testing Strategy for Microservices Applications in a Nutshell

What are Microservices?

Microservices also known as the microservice architecture is an architectural style that structures an application as a collection of services that are

• Highly maintainable and testable
• Loosely coupled
• Independently deployable
• Organised around business capabilities
• Owned by a small team

The microservice architecture enables the rapid, frequent and reliable delivery of large, complex applications. 

The trend has grown famous in recent years as Company's look to become more Agile and move towards a DevOps culture and continuous testing processes. 

It also enables an organisation to evolve its own required technology stack for it's microservices design.Here in the diagram below each microservice can have its's own tech stack meaning it's own programming language for frontend/backend and it's own database.

Microservices Test Automation

Usually the testing activities and testing levels we normally do for a monolithic application is quite typical and we all are used to it. When it comes to Microservices testing the test anatomy changes a bit. I have listed down the necessary tests below explaining on what testing types we usually contemplate for testing microservices.

Let’s start with the testing pyramid of Microservices

We have different test levels here in Microservices. Suppose, you have 4 separate microservices all together contribute to a whole single product of the UI application.

How will you do the testing of whole product and the individual microservices? 

Well, It was a million dollar question to me at-least when I was new to the Microservices world of testing. Coming from a Monolithic way of testing background got pretty much used to it and was trying to fit all the testing types like Regression Tests, User Acceptance Tests, Smoke Tests and Sanity Tests to Microservices architecture but it did not work in that way for me.

Then, I deep dived into the Microservices testing strategy from Martin fowler Microservices Testing and realised the actual ways to perform the testing needed. So to start with different levels of testing and keep it simple let's take the Testing Pyramid diagram above.

The Monolithic typical way of 3-Tier Architecture

• Presentation Layer: The presentation layer is the actual UI 

• Business Logic Layer (OR) Service Layer: This is the actual service layer where your applications business logic like for example calculations and logical decisions happen here. It send the request from the UI to the database layer and gets the response back from the database layer process it and sends back it to presentation layer. This layer is like a medium of processing and moving data between the other two layers.

• Database Layer: The information is stored in the database and retrieved when requested by the Business logic layer for processing.The data from database is sent back to the business logic layer for processing and finally sent back to the UI, the presentation layer.           

In Microservice we have the similar kind of architecture but with a few additional components like data mappers, gateways, external interface connection like the diagram below.

So the concept mostly works in the same way like 3-Tier architecture at higher level for the individual Microservice.Coming back to the point which set of test types goes where for a whole product and an individual microservice?

Like discussed above we have 4 Microservices here and Let's talk on the set of tests go for individual microservice.

Individual Microservice Testing:

• Unit tests: These are our white box testing test cases done by developers usually, testers also can perform this based on the demand for resources in the team and timelines.    
• Mutation tests
• Property Based tests
• Fuzz tests
• Testing with test doubles (Test Stubs and mocks)           

• Component tests: These tests are testing the entire microservice as a component and these tests are usually API tests but in my case I have to do UI testing for the whole component which is a single microservice. The tests are performed mostly by developers and Testers can also take part in it based on the requirements in their project.

• Contract and Integration tests: The Contract tests are called consumer driven contract tests basically API tests checking the contract between one single microservice to the other microservice. Basically the producer is one microservices and having contract with other microservice.
  
  For example, from the diagram below Producer provides three fields id, name and age and Consumer microservice A only wants to consume as per the contract only two fields id and name from Producer Microservice. The contract tests are often referred to as Sanity tests to check whether the latest versions of the services are deployed correctly to a test environment.

 

The contract tests are always API tests checking the communication between two individual microservices one as a provider and the other as a consumer through API's verifying the necessary and required data called as a contract. The contract tests are performed by the developers and the testers.

Integration Tests are checking the communication and interaction between the components.For example, Frontend can call services properly. Services, data mappers can access and communicate with Database correctly. Basically to verify the communication paths and interactions between components to detect interface defects.Basically ensuring whether the wiring is correct. These tests can be both API and UI. 

Integration tests perform testing on the components inside its own microservice, communicating with other microservices and external services like facebook, linkedin and google.

The Component and Integration tests are often called as Service Layer tests.

• Functional tests: These tests are UI based tests when adding a new feature to the UI for that particular microservice. In my case I moved all the new feature UI testing to this area.

• Canary releases and Canary testing: Deploying a new version of the application to limited number of people and getting feedback from testing that. These can be done via synthetic users or synthetic data but they need to be well organised and the traffic needs to be well monitored. This needs a big support from our DevOps Team.
  
  The canary release is to make the new application version ready for a small group of users to see the behaviour and feedback from the users and in this phase, If the team receives a positive feedback they make the application available from small set of users to all the users. Canary Testing is checking the new feature or new version of the code in the customers production environment.

• Regression Tests: The Regression tests are composed and consolidated in the project by combining the necessary and existing UI & API based Unit, Component, Contract and Integration tests to run them for every new feature deployments.

Now let's combine all the 4 microservices to a single page application (SPA). Now what test types goes in here?

Testing of all the 4 Microservices together as a single product (SPA)

• Smoke Tests: Basic tests that checks the basic and main functionality of the application, such as the url is opening up correctly and all the necessary tabs and page components of the application are working as expected.

• E2E Journey tests(API subcutaneous tests): These tests are called as subcutaneous tests by Martin Fowler as these tests are API based tests and they just run one step below the UI presentation layer meeting the actual business logic just as the UI tests does. 
  
  In case if we have dependency problems or Flakiness on the UI automation tests while checking the end to end workflow we can rely on these tests so that even the E2E UI tests fail we will come to know still the business logic works but it's just because of database dependencies or data issues etc.

• E2E Journey tests (UI): UI Journey tests verify main functionality of the application on the UI from start to end passing through all the 4 microservices from start to end as a journey. So here we don't write any edge test cases or negative test cases, we develop just only the main critical path of the workflow which is called the happy path.
  Test execution should not take more than 20 minutes maximum in our case. The total number of test cases count should be as less as possible because at this stage we will for sure have a lot of configuration and dependency issues. These tests are referred as expensive tests so we minimise them here to save the cost and time.

• Performance tests: The Performance of the whole application is measured with resources consumption like CPU, memory, disk and network with load and stress made by ramping up users on the application. We actually do the load, stress, spike and endurance tests here.

• Test in Production (TIP): These tests can be the same smoke tests, E2E Journey tests and Performance Load tests depending on the project and teams requirements which are executed on the customers environment in production to verify that everything is working as expected.

       Testing in Production is usually 

       Monitoring of the UI application  

       performance metrics with Datadog, 

       Prometheus or Azure Application Insights 

       as a part of post production validation.

       Note:- Stress, Spike and Endurance 

       tests are not run on customers production

       environment as it can block the 

       environment so these tests are not     

       recommended here for TIP except the 

       load tests.

• Exploratory Tests: We also do some manual exploratory testing on the SPA whole product to see if we can find some discrepancies while deploying new features on a single microservice impacting the whole product or on any version upgrades.

The E2E Journey tests, Performance tests and TIP are performed on the UI Presentation layer and are called Presentation Layer tests.

Finally, I managed to fit the Monolithic test types Smoke, Regression and Sanity tests to Microservice test types like below as Pre-Deployment and Post-Deployment tests. 

The following Pre-Deployment (UI & API) tests to run by Individual Microservices teams with support from QA for every new feature deployments are

• Smoke tests
• Regression tests
  • Unit tests
  • Integration tests
  • API Contract tests
  • Component tests
• Sanity tests (New Feature tests will go as a part of canary release)
• Business or QA team perform manual Exploratory testing on internal QA Env

    The following Post-Deployment tests to be maintained by QA team

• Smoke tests
• UI & API Journey tests
• Performance tests
• TIP - Application Monitoring
  • Real user Monitoring
  • Synthetic user monitoring (Synthetic testing)

Pre-Deployment and Post-Deployment Tests

Pre-Deployment Tests

The following folder structure of Pre-Deployment tests to be developed and executed by the Individual Microservices teams with the support from QA team on every new feature deployments.

The test structure mentioned below to be included in the individual teams CI/CD pipeline.        

Post-Deployment Tests

The following Post-Deployments tests are executed on Internal staging, Customers test & prod Environments

Test Integration into CI/CD Pipeline 

The diagram illustrates the overall CI/CD process for the tests to integrate into the CI/CD pipeline on new feature deployments.

Test Coverage

• By combining Unit, Integration and Component tests we achieve a high test coverage for each microservice and when grouped them all together as one system product to ensure the required business logic is met. Test coverage is achieved here let's say 70%.

• With Contract tests, test coverage is achieved let's say 15%

• With Performance tests, UI & API Journey tests and Exploratory tests (if applicable) test coverage is achieved let's say 15% but here performance tests being most crucial to be taken care.

Monitoring(TIP)

Once the code is deployed in to customers production environment, as a part of post production testing as a QA person you may need to check the application statistics in terms of load, response times and performance metrics of your application by setting up a monitoring tool like Datadog, Grafana or Azure Application Insights. The monitoring tool is set up to monitor your application's production environment behaviour. In case of any discrepancies, one can view the metrics on the tool's dashboard and also you can setup an alert option basically sending the alerts to your team when integrated the tool with communication channel like slack or Microsoft teams.

Happy Automation testing Guys :) !