DevOps: Phases of Continuous Integration

1. Why we want to change?
2. Get measurable business improvements/impacts by CI/CD - take incremental approach.
3. Focus on organization and architecture rather than tools, code, infrastructure.
4. Create culture of continuous improvement.

From 2008 to 2011:

1. Overall development codes reduced by 40%
2. Programs under development increased by 140%
3. Development costs per program down 78%
4. Resources now driving innovation increased by 5X

This section is borrowed from Jenkins: The Definitive Guide.

Continuous Integration is not an all-or-nothing affair. In fact, introducing CI into an organization takes us on a path that progresses through several distinct phases. Each of these phases involves incremental improvements to the technical infrastructure as well as, perhaps more importantly, improvements in the practices and culture of the development team itself. The following sections paint an approximate picture of each phase.

Phase 1 - No Build Server

Initially, the team has no central build server of any kind. Software is built manually on a developer's machine, though it may use an Ant script or similar to do so. Source code may be stored in a central source code repository, but developers do not necessarily commit their changes on a regular basis. Some time before a release is scheduled, a developer manually integrates the changes, a process which is generally associated with pain and suffering.

Phase 2 - Nightly Builds

In this phase, the team has a build server, and automated builds are scheduled on a regular (typically nightly) basis. This build simply compiles the code, as there are no reliable or repeatable unit tests. Indeed, automated tests, if they are written, are not a mandatory part of the build process, and may well not run correctly at all. However developers now commit their changes regularly, at least at the end of every day. If a developer commits code changes that conflict with another developer's work, the build server alerts the team via email the following morning. Nevertheless, the team still tends to use the build server for information purposes only-they feel little obligation to fix a broken build immediately, and builds may stay broken on the build server for some time.

Phase 3 - Nightly Builds and Basic Automated Tests

The team is now starting to take Continuous Integration and automated testing more seriously. The build server is configured to kick off a build whenever new code is committed to the version control system, and team members are able to easily see what changes in the source code triggered a particular build, and what issues these changes address. In addition, the build script compiles the application and runs a set of automated unit and/or integration tests. In addition to email, the build server also alerts team members of integration issues using more proactive channels such as Instant Messaging. Broken builds are now generally fixed quickly.

Phase 4 - Enter the Metrics

Automated code quality and code coverage metrics are now run to help evaluate the quality of the code base and (to some extent, at least) the relevance and effectiveness of the tests. The code quality build also automatically generates API documentation for the application. All this helps teams keep the quality of the code base high, alerting team members if good testing practices are slipping. The team has also set up a "build radiator," a dashboard view of the project status that is displayed on a prominent screen visible to all team members.

Phase 5 - Getting More Serious About Testing

The benefits of Continuous Integration are closely related to solid testing practices. Now, practices like Test-Driven Development are more widely practiced, resulting in a growing confidence in the results of the automated builds. The application is no longer simply compiled and tested, but if the tests pass, it is automatically deployed to an application server for more comprehensive end-to-end tests and performance tests.

Phase 6 - Automated Acceptance Tests and More Automated Deployment

Acceptance-Test Driven Development is practiced, guiding development efforts and providing high-level reporting on the state of project. These automated tests use Behavior-Driven Development and Acceptance-Test Driven Development tools to act as communication and documentation tools and documentation as mush as testing tools, publishing reports on test results in business terms that non-developers can understand. Since these high-level tests are automated at an early stage in the development process, they also provide a clear idea of what features have been implemented, and which remain to be done. The application is automatically deployed into test environments for testing by the QA team either as changes are committed, or on a nightly basis; a version can be deployed(or "prompted") to UAT and possibly production environments using a manually-triggered build when testers consider it ready. The team is also capable of using the build server to back out a release, rolling back to a previous release, if something goes horribly wrong.

Phase 7 - Continuous Deployment

Confidence in the automated unit, integration and acceptance tests is now such that teams can apply the automated deployment techniques developed in the previous phase to push out new changes directly into production. The progression between levels here is of course somewhat approximate, and may not always match real-world situations. For example, we may well introduce automated web tests before integrating code quality and code coverage reporting. However, it should give a general idea of how implementing a Continuous Integration strategy in a real world organization generally works.