Quality assurance consists of many activities, but often our primary goal is to verify and validate software our team is working on. Through various means we try to establish whether it fulfills customer requirements and other criteria like code quality, documentation, security, and so on. But who ensures that our job was done correctly? Even when we have a colleague who can review our test artifacts, they probably don't have enough time or context to do this thoroughly due to their own commitments.

Photo by Erik Mclean

This is why I always liked mutation testing as it aims to evaluate the quality of tests themselves. The idea is that we introduce small changes in the software under test and see if our tests managed to catch unexpected behavior by failing. We can, for instance, modify an SQL query from this:

select * from shopping_carts where user_id = ?

To this:

select * from shopping_carts where user_id != ?

So even though an insert method somewhere else would correctly add items to a shopping cart, it'll appear to be empty to the user due to the inverted where clause. If we don't have a test to validate the shopping cart after adding some items, all our tests will pass, revealing a gap in coverage. Like any other form of testing, mutation testing isn't exhaustive and it cannot prove the absence of bugs. Still, it can be a useful tool.

On my current project, I'm the sole test engineer in a team with several developers. We merge pull requests only when regression tests are implemented, so I sometimes find myself under pressure to handle multiple tasks at once - especially when there are failed tests. Without my input, it can be difficult to understand what a test is doing and what exactly is causing a failure: is it a bug or does a test need to be updated?

After reading many posts about AI in testing on softwaretestingweekly.com, I decided to give it a try and integrate Claude into our pipelines. The idea was to feed it test reports, grant it read-only access to the repository and the changes in a PR, and ask it to analyze everything and leave a comment with its findings.

Photo by cottonbro studio

One month later, here are my thoughts.

I wrote about WireMock last year. It's a very powerful tool that lets us replace external services with stubs while testing. Since it can be configured on the fly by tests, it allows us to validate virtually any behavior. Here's how I use it:

In this sequence diagram, a test tells WireMock to return a JSON body for every request to /fetch-details. During execution, the test calls /some-endpoint to validate it. This endpoint needs data from an external service to produce a response, so it requests /fetch-details and gets the JSON body that the test fed WireMock at the start. By changing what /fetch-details returns, we can easily simulate various scenarios.

In this post, I want to cover a case I dealt with recently: form data with decimal numbers.

Awaitility is an excellent Java library. It's especially useful for working with eventually consistent APIs. For instance, when a client sends a POST, PUT, or DELETE request, you might need to make several GET requests before observing the changes, which is terrible for automated tests. A common solution to this problem is to use Awatility. You can ask it to execute code for n seconds until the request succeeds or the timer expires.

await()
    .atMost(Duration.ofSeconds(5))
    .until(() -> apiClient().get(id).getStatus().equals("expected"));

Unfortunately, the default error messages could use some work.

Photo by Mike

Postman has long been, and continues to be, an industry standard for API testing. Numerous beginner courses for QA engineers dedicate sections to explaining how to use it. Software developers, product owners, database administrators, and many others often have Postman installed as well. Another prominent tool is Insomnia.

Both, unfortunately, push their users to create accounts and share often commercial information with their cloud services. I don't mind online collaboration, it can be really helpful, especially in large teams. However, I dislike when perfectly capable products start introducing artificial barriers that persuade customers to use features they don't need and never asked for. Do I need a cloud workspace to work on a course project? No. There's a term for that: enshittification.

A few months ago, I discovered another API client called Bruno, which offers a healthy alternative.

As a test engineer, I often need to see what requests applications are making and what responses they are receiving. While browsers offer developer tools that include network monitoring, I find them inconvenient. Additionally, you can't use them with mobile and desktop software.

Recently, I discovered Burp Suite, a tool for security assessment and penetration testing that comes with many useful features. Although the professional edition is quite expensive, the free community edition is sufficient for regular testing activities.

Key functionality

• A proxy server to intercept traffic.
• A convenient user interface to analyze network activity.
• The ability to modify and repeat requests.
• The option to store individual requests for future use; however, the history is erased in the community edition when you close the program.
• A text decoder that automatically translates strings like "%7B%22key%22:%22value%22%7D" to {"key":"value"}.

Playwright is a relatively new automation library for browser testing. Despite that, it has already gained some user base, with articles and tutorials regularly appearing on Software Testing Weekly. It comes with many features necessary for UI tests, but crucially, it can also work with API requests made by the web app under test. This allows us to either completely mock responses from a backend server, or alter them - depending on what we want to achieve. In this post, I will show how to do both things in Java.

A couple of years ago, I created a simple website designed to assist QA engineers in practicing some of the most commonly used test design techniques: Equivalence Partitioning and Boundary Value Analysis. The site provides a specification of a feature along with multiple implementations; one is correct, and the others are not. The objective was to show that the correct test input can effectively detect all errors. However, my initial intent was not clear to the end users because I did not provide correct answers or any explanations! Let me do that in this post.

It might be hard to find a fully independent service today. Even in the case of a monolithic architecture, where all components of the software are tightly coupled within a single codebase, there's still a need to integrate with the outer world. For instance, internal banking logic can be self-sufficient. But functionalities like police security checks and interbank money transfers can't be done in isolation.

During the development phase of a product or feature, programmers may write stubs and mocks to substitute missing dependencies. As QA engineers, however, we want to test the code that will go into production, touching the same statements and execution paths as the real data. Furthermore, we need to validate the system's behavior with various inputs, hence it should be possible to change how the external service responds to the program under test.

WireMock is a tool that enables us to do exactly this. It launches an HTTP server that can be configured to respond to specific requests in a desired manner. Let's say there is a system that should verify the recipient's IBAN against a database. If the response is positive, the transfer proceeds; if negative, the transfer must be blocked.

Testcontainers is a great tool for managing external services required for the proper functioning of an entire application or its components during testing. It automatically deploys these services before a test begins and then stops them once testing is over. This simplifies the process of running tests across multiple environments.

While working on integration tests for the service behind this blog, I stumbled upon an issue. It lies on the border between Testcontainers and Spring. When executing a single test or an entire class, everything worked as expected. But when executing multiple test classes (e.g. with ./gradlew test), the database became inaccessible.

In this post I'm exploring one of the possible solutions to this issue.

A few weeks ago, a potential employer asked me to complete a test assignment. The task was to implement a bunch of API and UI tests. This requirement is quite standard in the field, but I had not previously compiled any work ready to be shared alongside my CV.

Therefore, I made the decision not only to complete the assginment but also to document and explain it here in this blog post.