Companies that build software for external or internal usage have to implement a testing mechanism before the software can be deployed to the production environment. There is one important aspect to consider when building and executing the test strategy—how the tester is accessing the developed piece of software. Some organizations have a dedicated shared environment that is updated with new code during development and testing, while others rely on individual environments that are managed by the owners of those. This article focuses on the benefits and drawbacks of using the individual test environment versus the shared environment.
In my career as a software test engineer, I have used both types and have some thoughts and ideas to share that might be useful to the companies that are on their way to establishing the QA process or considering optimizing their existing process.
First, let’s talk about the shared test environment. Usually, the deployment to this environment happens by developers who write the code or the DevOps engineers. The implication of this type of deployment for the QA process is that QA engineers or testers do not control the environment. They are given the “built” product and only need to worry about the actual testing. I would consider this to be a huge benefit, especially if the QA engineer is less experienced in configuration management. Building a functioning environment requires specific knowledge and takes time. Maintenance of this environment is also time-consuming. Therefore, by getting the ready-made test environment, QA engineers don’t need to spend their time on setup and troubleshooting and are able to focus on testing the product, detecting issues, and tracking fixes.
Another important benefit of using the shared environment is fewer divergences. Everyone on the team is sharing a database, services, and the version of the deployed source code, which makes it easier to understand the failures and bugs, debug, and troubleshoot. Contrary to this, when every team member is using his personal environment, there is a pretty high chance of dissonance. It could be anything from using different branches or environment variables to the size of the virtual machine itself. And if you ask me how often this becomes an issue, I would say that after the initial setup, the environments tend to diverge in most of the cases. Oftentimes, it becomes pretty difficult to isolate the bug or even determine if the failure is a bug, a test issue, or a setup problem. When the environment is shared, everyone on a team is relying on the same configuration, and the issues are easier to detect.
However, it would be unfair to only talk about the benefits of the shared test environment as there are also a number of disadvantages. So, what are the drawbacks?
I am joking here, but the main disadvantage is that having your own test environment is indeed awesome—and when someone is using a shared environment, he would never experience how convenient it could be. I would stop here, but let me still name some major drawbacks of a shared environment.
First and foremost, there is no flexibility in a shared environment. There is no good way to switch among branches. Are there any? Well, not unless the team needs this and it is centrally managed. Testing two branches of the same repository requires the additional effort of provisioning a separate test environment per branch, and it is only possible when the test environments are provisioned dynamically. In my experience, a shared environment is related to a pretty static release image and the waterfall SDLC. Contrary to this, using the personal test environment allows switching gears on an as-needed basis and is perfectly fitting to the needs of the Agile cycle. From my experience, I tend to work on numerous projects at the same time and switch branches several times a day. This flexibility allows delivering several projects in parallel without a dependency on release timelines.
Moreover, a shared environment implies using shared resources—the databases and services. Is this a problem? It is not an issue per se, but it tends to turn into a problem when the size of the testing team is greater than one. There is always a risk of stepping on each other’s toes. The entries in the database could be overwritten or deleted causing the tests to error. I’ve been there, and I know how uneasy it is to manage “sharing”. It is very unpleasant when your day of work is eliminated by the running of a simple query by your peer. It is equally unpleasant when you unknowingly do the same to your colleague. A shared environment requires excellent communication, some protection techniques, trust, and more. So when we are saying that it is easier to use, I recommend that you stop and think twice. Is it really?
Here comes a huge shout-out to the individual test environment, which is more flexible, has fewer dependencies, and is so pleasant to use. I am using a virtual ec2 instance at my current job, and it took me a while to become familiar with the process of setting it up and maintaining it. I definitely started with frustration and dislike—but after eight months of usage, I am a huge fan. I don’t have to worry about the data I might accidentally corrupt. I don’t need to wait on other people to finish their part so I can safely start working on my part. I feel more productive because I can easily switch context and work on a different project if needed. I don’t have to wait on developers to deploy the code. Instead, I visit GitHub and check out the branch I’d like to start testing. This is so much more transparent and productive.
Of course, an individual test environment is more expensive and takes more time and knowledge to manage. If anything isn’t working, there is no shared responsibility to fix it. This falls under the owner’s umbrella.
I conducted a survey targeting people who work in software development to find out the preferred method. The results of this survey confirm that most people prefer using an individual environment because it is more flexible.
The results of the survey are the following:
- More than 90 percent of respondents have experience using both a shared and individual environment. 72 percent prefer using an individual environment.
- People mainly like using a shared environment because it has fewer divergences (52 percent). The next selected benefit of using a shared environment is that it is easier to use for inexperienced QA.
- In ranking reasons for giving a preference for an individual environment, most respondents (66 percent) selected flexibility as a top reason. The next category was less dependencies (28 percent).
The survey results confirmed my personal attitude to the preferred test bed. The majority of the QA engineers prefer using an individual environment as it gives more flexibility and allows performing testing more efficiently. This type of setup is more expensive and requires specific knowledge and support; however, in my view, the benefits far outweigh the expenses.
Originally published on StickyMinds.