Should developers write end to end tests? Should QA write end to end tests? Or both?
Case for developers writing end to end tests
In an ideal world, I believe that developers should write their own end to end tests.
This makes sense from a responsibility point of view.
Consider: If a small company hires a contractor developer, then the developer must produce a working product as part of their contract. The developer probably won’t subcontract a QA specialist to test their software for them. It’s purely up to the developer to test their code to ensure it works correctly.
Similarly, if a large company hires a developer, they would expect the developer’s code to work at least 95% of the time. Otherwise, if they can’t produce things correctly, what’s the point of hiring them?
QA should be optional. They should be there as an additional layer of defence for the odd and rare mistake. QA should not be required for the main 95% of the product to work. (The 5% or so difference is acceptable because anyone, no matter how good they are, makes mistakes sometimes.)
Anyway, by that argument, developers must test their work properly to ensure it works correctly. Since end to end tests are very important when testing, developers should probably write their own end to end tests.
Another reason for developers to write end to end tests is that some end to end tests can be fairly complicated. They may have to seed the database in different ways, make HTTP requests and do other complicated things. They may be easier for developers to write.
Case for QA writing end to end tests
There are a few reasons that makes QA potentially better at writing end to end tests.
In-short, QA are just great at testing, probably much better than most developers:
- They specialise at testing.
- They are better at prioritizing tests because they understand which use cases are important and which are not.
- They are great at breaking down software to find bugs.
- They are better at thinking like a user and finding all of the weird and unintended ways in which a user might use the software.
This means that the quality of their tests will probably be better. They’ll also be able to do the work faster.
Additionally, since end to end test are expensive (they take a long time to write and a long time to run), you can’t have a large number of them. Since QA know which use cases are most important, they’ll be able to handle this point better too. They can correctly prioritize which end to end tests to have and which to neglect.
Finally, since QA can do this work faster than developers, it may be more cost effective for the company to hire a dedicated QA team / person for it. All things being the same, if an employee does a task in 10 hours, rather than in 12 hours, the company saves money.
Combined effort
Like with many things, the best answer may be "somewhere in the middle". There are good reasons for both developers and QA to write end to end tests. Therefore, you could consider some collaborative options:
- End to end tests could have collective ownership. Both QA and developers could own them and have complete visibility and editing rights over them.
- QA can own the end to end tests. Developers can have full visibility of them, can ask questions about them, can suggest additions or alterations to them, etc.
- QA could decide what to test, but then ask developers to implement the tests. Alternatively, they can ask the developers for help whenever they want, if they’re not 100% confident in their coding skills.
- And so on…
Additionally, I believe that the Ministry of testing – Modern testing principles also recommend collaboration. Many of their principles seem to suggest: ‘Let developers do the heavy code testing. Let’s coach developers on how to test well and have ownership of quality.’
Sidenote: Development efficiency
Developers should have access to end to end tests, even if they don’t write them
Tests make development faster. If developers code something incorrectly, a test tells them now, not 2 weeks later when things have gone wrong in production. This means that they can fix it immediately. It also means that they probably know where the new bug is: It’s somewhere in the code they just changed 2 minutes ago.
In general, earlier feedback is usually better.
The most important tests are end to end tests. For this reason, it’s highly recommended that developers to have access to them. They should be able to run them during development.
Early end to end tests can improve efficiency
During development, developers manually test their work end to end. For example, in front end development, developers will make a change, open the browser and test that things are working correctly. If they’re working on a web API, they might make a change, manually send an HTTP request and check the response to ensure that the API works correctly.
This manual work will be repeated very often and it takes time.
In the front end case, a developer may need to clear cookies, open the website, log in, add a product to cart and then check the cart page, every time.
However, with automated end-to-end tests, that entire process can be automated and performed very quickly. This can save a significant amount of development time.
For this reason, it may be beneficial to have an end to end test, for the current feature, before development starts. This could be achieved if developers write their own end to end tests, or if QA write end to end tests upfront.
It’s not necessary, but it may be helpful.
Conclusion
There are good reasons for both developers and QA to write end to end tests.
Alternatively, developers and QA can collaborate. For example, QA could own the end to end tests, but developers can periodically review them and suggest alterations, etc.
Finally, regardless of who writes them, developers should have access to end to end tests. It may also be beneficial for developers to have an end to end test for the current story before they begin development.
Alright, I hope you enjoyed this article and hopefully got some value out of it.
If you have any comments, suggestions, differences in opinion or anything you want to share, please feel free to leave a comment.
See you next time.
Good Article. Thanks for sharing.