CI/CD and Deployment Fundamentals

Before Continuous Integration (CI), software teams worked on separate branches for days or weeks and then attempted to merge them at a point called "integration day." Integration day was reliably painful. Changes that were individually correct turned out to be incompatible when combined. The larger the batch, the harder it was to isolate which change had introduced a conflict.

Continuous Integration is the practice of merging developer changes into a shared branch at least daily, and automatically running tests on every merge. The goal is to find integration problems within hours of their introduction, when the engineer who wrote the conflicting code still has it in their head. A CI system that detects a build failure 15 minutes after a commit is a useful tool. A CI system that runs weekly is an integration report, not a feedback loop.

The DORA (DevOps Research and Assessment) programme has tracked CI adoption and its effects since 2014. It now publishes five delivery metrics rather than the original four keys: change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate. Two of those names matter. Failed deployment recovery time is the metric that used to be called MTTR, renamed because what is being measured is recovery from a failed deployment, not mean repair time in general. Deployment rework rate is the addition: the proportion of deployments that were not planned but happened because of a production incident, which is a direct read on how much of a team's throughput is corrective work. Elite performers in the 2023 State of DevOps report deploy on demand with lead times under one hour. They achieve this through trunk-based development: all changes go directly to the main branch, and features are protected by feature flags rather than long-lived branches. Long-lived feature branches are a CI anti-pattern: they accumulate integration debt for every day they are not merged.

CI/CD is used as a single abbreviation, but the three terms it covers have distinct meanings that matter for architectural and compliance conversations.

Continuous Integration (CI) is the practice of integrating code changes frequently and running automated tests on every integration. It produces fast feedback but does not address how changes reach production.

Continuous Delivery (CD) extends CI: every build that passes automated tests is placed in a state where it could be deployed to production. The word "could" is precise. In continuous delivery, the decision to deploy is a deliberate, human-controlled action. The pipeline builds, tests, and packages the artefact. A person (or a governance gate) decides when to release it. This is appropriate for regulated environments where a change approval record is required before production deployment.

Continuous Deployment removes the manual gate entirely. Every build that passes automated tests is deployed to production automatically. This requires high test coverage, strong feature flag infrastructure for incomplete features, and an automated rollback mechanism on failed health checks. Most organisations practice continuous delivery rather than continuous deployment because their compliance, change management, or risk frameworks require a human approval point.

A CI/CD pipeline is an ordered sequence of automated stages. Each stage validates a different aspect of the change and can block subsequent stages if it fails. The ordering principle is fast-to-slow: cheap, fast checks run first to give developers quick feedback on obvious failures, and expensive, slow checks run later.

Commit stage (unit tests, lint, type check, build). Runs on every commit to every branch. Target: under 5 minutes. Unit tests verify individual functions in isolation. Linting and type checking catch syntax and type errors. A SAST (Static Application Security Testing) scan checks for known vulnerability patterns. The build step confirms the artefact compiles and packages correctly. A commit stage that takes 45 minutes breaks the feedback loop: developers context-switch before the result arrives.

Integration stage (integration tests, contract tests, image build).Runs after the commit stage passes. Integration tests verify components work together with real dependencies: a test that calls the API and checks database state is an integration test. Contract tests verify that a service conforms to the interface its consumers expect. The Docker image is built and tagged with a deterministic version.

Security scan stage (dependency scan, DAST, container scan).Runs OWASP Dependency-Check or Trivy against the built image to identify vulnerable dependencies. DAST (Dynamic Application Security Testing) tools such as OWASP ZAP make test requests against a running instance. This stage implements the dependency scanning gate described in the Security by Design module.

Staging deployment (smoke tests, performance baseline).The versioned artefact is deployed to a staging environment that mirrors production configuration. Smoke tests verify the deployed system starts and responds correctly. A performance baseline confirms the change has not degraded response times beyond a defined threshold.

Production deployment (rolling or blue-green, health checks, rollback).The validated artefact is deployed to production using a strategy that limits blast radius. Automated health checks monitor error rates and latency after deployment. Rollback must be a button press, not a new deployment pipeline run.

Feature flags (also called feature toggles) decouple deployment from release. Code ships to production in a disabled state, controlled by a flag. The flag controls whether the new code path runs, allowing: dark launching (testing with internal users before customer exposure), ring deployments (enabling for 1% then 10% then 100% of users), and instant kill switches (disabling a problematic feature without a rollback deployment).

Trunk-based development is the branching strategy that makes CI work at scale. All engineers commit directly to the main branch (the trunk) multiple times per day. Features that are not ready for users are deployed behind a disabled feature flag. There are no long-lived feature branches: a branch that lives longer than one day is a liability, accumulating integration debt with every commit to main.

Feature flag debt is a real operational risk. Each active flag is a branch in production code. Teams that accumulate 50 or more active flags report difficulty reasoning about system behaviour and testing all combinations. The governance rule: temporary release flags must be removed within two sprints of the feature going to 100%. Permanent operational toggles require explicit ownership and review cycles.

A standard deployment environment chain is: development, testing, staging, production. Environment parity means staging mirrors production as closely as possible: same infrastructure configuration, same managed service tiers, same data volume characteristics, same network topology. A staging environment that differs from production in meaningful ways means every production deployment is the first time the change runs in a realistic environment.

Canary deployments route a small percentage of real production traffic to the new version before full rollout. Automated monitoring gates check error rate and p99 latency at each step. If metrics degrade, the deployment rolls back automatically. This is the lowest-risk path to production for high-impact changes.