My notes from the DevOps Handbook

by Gene Kim, Jez Humble, Patrick Debois, John Willis

Build fast and reliable automated validation test suite

Integrating the code at the end of the project with large batch sizes, big bang integrations, giant merges result in all the undesired outcomes. That is why continuous integration practices were developed.

Automated tests from the fastest to slowest:

When facing deadline, developers may stop creating unit tests as part of their daily work, regardless of how we have defined "done". To prevent this we need to make test coverage visible or even fail validation if it drops below a certain level.

Catch errors as early in our automated testing as possible

Goal of our automated testing is to find errors as early as possible. Any errors should be found with the fastest category of testing possible. For each error found in acceptance or integration testing, we should create a unit test that can find this error faster, earlier and cheaper.

If writing a unit test feels too difficult or expensive, that suggests that our architecture is too tightly coupled. We should make it more loosely coupled so that modules can be tested independently.

Ensure tests run quickly

Our tests should be able to run in parallel across many different servers. We may also want to run multiple test categories in parallel, for example security and performance.

Any build that passes automated tests will be enabled for exploratory and manual testing. Any tester should use the latest build that has passed all the automated tests as opposed to the one marked as ready for test. That way testing is done as early as possible.