My notes from the DevOps Handbook

by Gene Kim, Jez Humble, Patrick Debois, John Willis

Test Driven Development

One of the most effective ways is to write tests as part of the daily work by first writing them. The steps for this are the following:

  1. Ensure test fails. Write test for functionality planned to be added.
  2. Ensure the test passes. Write the functional code.
  3. Refactor the code. Ensure the test passes.

Teams using TDD: 60%-90% better quality in 15%-35% more time.

Tests are also a form of a documentation providing examples how to use the API.

Automate as many of our manual tests as possible

Reduce the reliance on manual testing.

Although testing can be automated, creating quality cannot. To have humans executing tests that should be automated is a waste of human potential.

We can enable our testers to work on high value activities that cannot be automated, such as exploratory testing or improving the test process itself.

A small number of reliable, automated tests are always preferable over large number of manual or unreliable automated tests. Focus on testing only the business goals that we are trying to achieve. It's better to run a few tests that we trust than to run tests that aren't reliable.

Start with a small number of reliable automated tests and add to them over time, creating an ever increasing level of assurance that we will quickly detect any changes to the system that take us out of a deployable state.

Integrate performance testing

Write and run performance tests that validate our performance across the entire application stack as part of the deployment pipeline.

Large amount of compute and I/O requirements, makes performance tests more complex than creating the production environment itself.

To find performance problems early, we should log performance results and evaluate each run against previous results. If for example we deviate more than 2%, we can fail the pipeline.