Terraform #08 — Testing & CI/CD

Goal: Automate quality checks for your Terraform code. A proper testing and approval pipeline is table stakes for production infrastructure.

8.1. The Terraform testing pyramid

                  ▲   few, slow, expensive
        ┌─────────────────────┐
        │  E2E / Integration  │  terraform test (apply mode), Terratest
        ├─────────────────────┤
        │   Policy / Security │  checkov, tfsec, OPA, Sentinel
        ├─────────────────────┤
        │   Plan-time tests   │  terraform test (plan mode), preconditions
        ├─────────────────────┤
        │  Static / Lint      │  fmt, validate, tflint
        └─────────────────────┘
                  ▼   many, fast, cheap  (run on every commit)

Run lots of cheap tests at the bottom, fewer expensive ones at the top.

8.2. Layer 1 — Static checks (run constantly)

terraform fmt -check -recursive    # is the code formatted? (CI fails if not)
terraform validate                 # valid syntax and references? (no API calls)

tflint — more powerful than validate

validate misses a lot of logic errors and anti-patterns. tflint catches:

• Provider-specific syntax errors (e.g. an instance_type that doesn't exist).
• Naming convention violations.
• Unused variables or outputs.

.tflint.hcl:

plugin "aws" {
  enabled = true
  version = "0.27.0"
  source  = "github.com/terraform-linters/tflint-ruleset-aws"
}
 
rule "terraform_naming_convention" { enabled = true }
rule "terraform_unused_declarations" { enabled = true }

tflint --init
tflint --recursive

8.3. Layer 2 — Native test framework (terraform test)

Available since Terraform 1.6+. Tests live in *.tftest.hcl files. This is the modern approach and is replacing Terratest for most use cases.

A test file structure:

# tests/basic.tftest.hcl
 
variables {
  environment = "dev"
}
 
# plan-mode run — NO real resources created, fast
run "validate_naming" {
  command = plan
 
  assert {
    condition     = local_file.config.filename == "./out/myapp-dev.json"
    error_message = "Config filename doesn't match the naming convention."
  }
}
 
# apply-mode run — creates real resources, checks them, then destroys
run "create_and_check" {
  command = apply
 
  assert {
    condition     = fileexists(local_file.config.filename)
    error_message = "Config file should exist after apply."
  }
}

Run:

terraform test                 # run all *.tftest.hcl in tests/
terraform test -verbose
terraform test -filter=tests/basic.tftest.hcl

Advanced features of terraform test

• command = plan (fast, no resources) vs command = apply (real E2E).
• Multiple sequential run blocks sharing state within one test file.
• expect_failures — verify that a validation or precondition fails as expected.
• module {} block — test a module from a different location.
• Mock providers (1.7+) — test without hitting real cloud APIs.

run "invalid_env_is_rejected" {
  command = plan
  variables { environment = "invalid" }
 
  expect_failures = [var.environment]   # expect the validation to block this
}

8.4. Layer 2 — Terratest (Go)

Terratest (Gruntwork) is a Go library for E2E testing. Very flexible and powerful, but you need to know Go, and tests are slower.

func TestTerraformVpc(t *testing.T) {
  opts := &terraform.Options{ TerraformDir: "../examples/basic" }
  defer terraform.Destroy(t, opts)
  terraform.InitAndApply(t, opts)
 
  vpcID := terraform.Output(t, opts, "vpc_id")
  assert.NotEmpty(t, vpcID)
}

Which one to use? terraform test handles most cases (simpler, no Go required). Reach for Terratest when you need complex validations — like making HTTP requests to a newly-created endpoint or checking actual cloud API behavior.

8.5. Layer 3 — Security & policy

checkov / tfsec / Trivy — catch security misconfigurations

checkov -d .                       # scan the whole directory
checkov -d . --compact --quiet
trivy config .                     # Trivy also scans IaC

Catches things like: public S3 buckets, security groups open to 0.0.0.0/0 on port 22, unencrypted RDS instances, missing logging...

Policy as Code — OPA/Conftest & Sentinel

• OPA (Open Policy Agent) + Conftest: write policies in Rego, apply them against a plan JSON.
• Sentinel: HashiCorp's own policy language, used in HCP Terraform/Enterprise.

terraform plan -out=tfplan
terraform show -json tfplan > plan.json
conftest test plan.json            # apply Rego policies against the plan

Example policy use cases: block instance sizes above a threshold, enforce required tags, deny certain regions...

8.6. CI/CD pipeline

Standard GitOps workflow

   PR opened ──► CI runs: fmt, validate, tflint, checkov, plan ──► posts plan as PR comment
                                                                           │
   Review & approve ◄─────────────────────────────────────────────────────┘
        │
   Merge to main ──► CD: terraform apply (with manual approval gate for prod)

CI/CD principles for Terraform

1. Plan on PR, apply after merge. Reviewers see the plan before it lands.
2. Save the plan, apply exactly that plan (plan -out → apply tfplan).
3. Manual approval for prod (environment protection rules).
4. OIDC instead of long-lived keys (GitHub Actions → AWS via OIDC, no stored secrets).
5. State locking prevents parallel applies.
6. Don't log secrets (mask them, use -no-color for clean plan output).

GitHub Actions (see sample file in hands-on)

Two jobs:

• plan (runs on PR): fmt → init → validate → tflint → checkov → plan → post comment.
• apply (runs on merge to main): init → apply (with manual approval for prod).

Atlantis — GitOps via PR comments

Atlantis is widely used: comment atlantis plan or atlantis apply in a PR, and it runs the operation and posts the result back. Many teams use it to standardize their Terraform workflow.

8.7. pre-commit hooks — catch issues before they leave your machine

.pre-commit-config.yaml:

repos:
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.92.0
    hooks:
      - id: terraform_fmt
      - id: terraform_validate
      - id: terraform_tflint
      - id: terraform_docs
      - id: terraform_checkov

pre-commit install        # install the hooks into git
pre-commit run --all-files

Every git commit automatically runs fmt/validate/lint — failures are caught before they ever hit CI. Saves the whole team time.

8.8. Hands-on Labs

In hands-on/:

• Sample code (main.tf) + real test files (tests/*.tftest.hcl) — ready to run with terraform test.
• Sample CI file (ci-github-actions.yml.example), .tflint.hcl, .pre-commit-config.yaml.

Follow LAB.md:

1. Lab 1 — Write and run terraform test with both command=plan and command=apply assertions.
2. Lab 2 — Test expect_failures for a validation rule.
3. Lab 3 — Read through the GitHub Actions pipeline and understand each step.

✅ Module 08 completion criteria

• Can run terraform test with both command=plan and command=apply.
• Can write an expect_failures test for a validation rule.
• Understand the testing pyramid and where each type of test belongs.
• Can explain the "plan on PR, apply after merge" pipeline.
• Know why OIDC is better than access keys in CI.
• Installed and understand pre-commit hooks.

➡️ Next: Module 09 — Security & Refactoring