Canary testing refers to the incremental deployment of new software code to a few users only. The new code could be a new feature or an additional functionality to an already existing application. This type of software testing helps ensure that any problem can easily be patched since only a limited number of users are affected.
Canary testing aims to prevent any software bug from negatively impacting the whole production or a huge user base. By limiting the number of affected users, developers can immediately detect and address any issue.
Other interesting terms…
Read More about “Canary Testing”
Canary testing is one of the most popular ways developers use to test out new code. Netflix, for one, uses the process before launching new features and functionalities. The company describes canary testing as a “technique to reduce the risk of deploying a new version of software into production.”
Why Is It Called “Canary Testing”?
Canary testing is also called “canary deployment” or “canary release.” The term can be traced back to the mining industry in the 1980s, when miners would carry a canary into coal mines. When the bird dies, it’s a sign that the site has too much toxic gas, prompting the miners to evacuate.
In the software industry, no birds suffer during canary testing. Still, the term stuck, and the testing process has become a norm, especially when developing new features and functionalities.
How Does Canary Testing Work?
There are several types of software testing procedures available. In most cases, when developers finish coding a specific feature, they would perform automated tests to detect basic bugs.
When the code passes the automated tests, it would then be deployed to a testing environment where software testers or other colleagues can try it out to discover hidden issues. Once found, bugs are fixed. The new code is then deployed to production and implemented in the end users’ environment.
This new Instagram feature that allows you to restore deleted photos, for example, went through a series of tests before it became available to the public.
While the procedures detailed above can help reduce the number of bugs, there could still be significant issues that make it through production. One reason for this is that the testing and production environments are very different. For instance, they differ in terms of storage and bandwidth capacity. The operating systems (OSs) they use could also make a difference.
For this reason, canary testing comes in handy. It allows developers to see how their code works in the actual user environment without risking the experience of the whole user base.
When performing canary testing, developers would select a small percentage of users. In most cases, they would have no idea that they are using a new code, the same way that canaries in coal mines don’t know why they were carried along.
When the canaries or test users encounter problems, these are immediately reported so that developers can make changes to the code until it becomes ready for large-scale deployment.
Canary Testing versus Smoke Testing
By now, you know what is canary testing and that its focus is to ensure that the new code is working as it should. On the other hand, smoke testing in software development concentrates on an application’s significant functionalities or features.
When releasing a new version of an antivirus program, for instance, you would want to make sure that the software can still detect intrusions and alert users to such anomalies. Simply put, you can think of smoke testing as a car tune-up where every important part of the vehicle is checked. Canary testing is more comparable to checking newly installed tires.
—
Learning what is canary testing may sound complicated, but there are already platforms that allow developers to automate the process. Some solutions also help them analyze canary testing results to provide fixes better and make the proper adjustments. Testing-as-a-service (TaaS) has also made canary testing and all other types of testing more convenient.
