Many project managers are looking for a universal answer to the question: "What should be the ratio of testers and developers?" In some companies, it comes to the approval of standards: for example, the number of testers should be 40% of the development team, or for each developer there should be one tester. To substantiate this ratio, some universal statistics on the industry is often selected. Is there an optimal recipe?
There is no right ratio
Universal answers are always fraught with inaccuracy. Imagine: you come to the doctor and begin to complain to him about your problem. He, without listening, prescribes a medicine:
- - What weight do you have? 80? So, according to the standard, you need to drink 2 tablets per day.
- "But wait, Doctor, I don't have a cold, I have a broken leg!"
- - Don't be distracted, we have a standard. Next!
By reducing the cost of diagnosing, assessing the situation and finding the right solution for you, you are likely to subsequently lose much more time on the inevitable problems in this case.
When savings are expensive
If your "standard" does not suit your project, and you recruit too many testers to the team , this is not the worst thing. As a rule, it is much worse – a shortage of people. Due to a late-found error, the release dates of the entire project can be postponed, and a defect in industrial operation sometimes leads not only to unnecessary costs (for corrections, technical support, releases of hotfixes), but also to an inevitable loss of reputation.
And the most important thing, although not always noticeable, is the impact on the overall productivity of the team. According to the Theory of Constraints (as well as in accordance with common sense), the productivity of the entire production chain is determined by the productivity of the weakest link, which means that the total speed of development will rest on the least productive stage.
Let's say that the team of testers does not have time to do their work. What, then, does the development team do with the remaining surplus of resources? As a rule, it partially compensates for the lack of testing! As a result, developers have to spend a lot of effort on additional activities. We may encounter several options for such labor costs:
- if testers start bugs too late (a week or more after the build is ready), the fix takes much longer.
- if testers start bugs are not clear enough, developers themselves spend time figuring out and localizing problems;
- if customers find bugs already in the received software, a significant part of the development team is engaged in bug fix, release of fixes and technical support.
- It turns out that the tasks of testing slow down the whole process. By saving on testing, which is cheaper than the development process, we lose development productivity!
More is not better
Many project managers still believe in the project triangle:
"Want to do something better, more? Either hire more people or extend the timeframe." Fortunately, this model is incomplete. It does not take into account either the qualifications of the team or the specifics of the work process. When you recklessly add resources to a project, control is lost and redundant "white noise" is added.
I personally participated in the project to save a large software company with a worldwide reputation. The problem at the entrance sounded like a disaster: "We broke the deadline for the release of the product for a year and a half. There are already more than 100 people in the team, but every day we find more and more problems, and the release dates are postponed further. " In the first month, we reduced the project team to 40 people, two months later a successful release was released.
How do you find your own "right" ratio?
If there are too few testers, their problems fall on the shoulders of the development. If there is too much, control is lost, distracting "white noise" is generated, and this also negatively affects the overall design performance. Building your unique, effective team will require analysis of the process, product, and team. Below we will consider one of the possible algorithms of your actions.
Step 1: First, build the process, then expand the team.
Before you add to the team, make sure that your employees are working productively enough. The 5M technique from Kaizen and the "Tree of Current Reality" from the Theory of Constraints will help you with this. First, you should assess the optimality of the current processes and only then make a decision to expand the team, if necessary. Sometimes you can get a completely opposite result – the number of employees needs to be reduced!
Step 2: Consider the criticality of the software product.
The ratio of testers to developers largely depends on the importance of the products you are developing. The depth of testing browser toys and software in avionics is significantly different: the price of an error increases - therefore, the price of its detection also increases. It becomes insufficient to find only obvious problems. It is necessary to check all imaginable and unthinkable situations, creating models and making sure that their coverage is complete.
In non-critical software, minor errors are often not corrected at all, and therefore their search and registration do not make sense. The higher the degree of responsibility of the project, the more attention is paid to trifles: there are no "small" mistakes in such tasks.
I once worked on a team testing a low-level real-time operating system used in the space industry. There were 7-8 testers per developer, and we carefully covered every level with tests: from interaction with hardware and mathematical functions to integration and system functioning.
Step 3: Consider the supported environments.
Requirements for supported environments significantly affect the cost of testing, and to a lesser extent - on development. A good example of such a dependency we see in the development of Android applications. A huge number of phone manufacturers, different versions of Android, alternative firmware, shells and settings of the system itself together give an infinite number of combinations. Theoretically, the development for them is carried out in accordance with the general guideline. In practice, unexpected defects are possible under specific conditions. You can only learn about them empirically through testing. To detect an error associated with a particular firmware of a particular hardware, it may be necessary to check a hundred other valid combinations. Development will be involved only if a problem is detected, and testing - in any case.
Step 4: Consider the complexity of the product.
Testing costs grow exponentially depending on its complexity. It is important for us to check not only the operation of individual operations or the processing of specific data, but also the connections with other scenarios or objects of our system. Combinatorics in testing comes into play. Suppose there are 3 fields in the customer card, each of which takes 3 possible values - these are 9 possible combinations. Let's add a fourth field with similar business logic. For the developer, this will entail an increase in labor costs by 33%. But in testing, we already get 27 combinations instead of 9 - and, consequently, the cost increase is not by 33%, but by 200%.
Do not miss from the analysis and such a phenomenon as regression testing. Let's say we have a mature complex product, the code base of which contains one and a half million lines of code. The developer makes a small change, and... What to check? How do I know if these edits didn't break anything?
As an example, I will cite another case from my practice. Once I worked as a tester in a web company that develops fairly simple sites that do not contain serious functionality (90% of developments are ordinary "business cards"). My part-time job was enough to check the results of the work of 30 developers and layout designers, without missing a single serious mistake. Simple products and testing is very simple.
Step 5: Consider the connections with external products.
The more integrations there are in the system, the more testers are required. Errors can be contained both in third-party libraries and in the incorrect format of the data received from the provider. Developers do not need to hammer their heads with this until the testers confirm the presence of the problem. Finally, there are situations when there are no errors, but for full confidence in this, it is still necessary to conduct full testing of the integration.
Once I was given a revision performed by a programmer in less than a day for testing. It had 100 lines of code that provided data transfer to an external task management system through its API. For 3 (three!) days of testing, I started about 40 serious functional defects. All of them were caused by incorrect operation of the external API or its outdated documentation. The developer had to bypass these errors with "crutches" on our part.
Findings
Coming up with some kind of universal number and following it is very simple. Choosing the best ratio for your project is much more difficult. If you want the project to be not just "normal", but truly successful - you will have to work hard! The best results will be achieved if you first analyze the current tasks, implement the optimal process, and only then make a decision on the number of testers and developers. All sorts of "only correct" attitudes and regulations can only lead you away from an effective solution.
No comments:
Post a Comment