Terraform #02 — State Management: Remote Backend, Locking & Import

Goal: Understand the state file — the heart of Terraform. This is the topic that causes the most production pain. Know it well before you need it.

This is the most important module. State questions come up constantly in technical interviews about Terraform.

2.1. What is state and why does it exist?

State (terraform.tfstate) is a JSON file that maps between:

   Config (.tf)          State (.tfstate)          Reality (Cloud)
   ───────────           ─────────────────         ───────────────
   aws_instance.web  ◄──►  id: i-0abc123    ◄──►  Actual EC2 instance

State answers three critical questions:

1. Which real-world object does this config resource correspond to? (config → ID mapping)
2. Metadata — dependency ordering, auxiliary data.
3. Performance — cached attributes so Terraform doesn't have to query the entire cloud every plan.

❗ Without state, Terraform doesn't know what it created. It would try to create everything again (duplicates), and wouldn't know what to update or destroy.

2.2. The problem with local state

By default, state lives in a local terraform.tfstate file. This is fine for solo learning, but breaks down fast for teams and production:

⚠️ Never commit terraform.tfstate to Git. It contains secrets and causes merge conflicts.

2.3. Remote backend — the solution

The backend controls where state is stored and how operations run.

            ┌─────────────────────────────────────────┐
            │          Remote Backend (S3)             │
  Dev A ──►│  s3://my-tfstate/prod/terraform.tfstate  │◄── Dev B
            │                                          │
            │  + Lock (DynamoDB)  ← only one apply     │
            │  + Versioning       ← rollback anytime   │
            │  + Encryption       ← secrets are safe   │
            └─────────────────────────────────────────┘

S3 backend (AWS) — the most common choice

terraform {
  backend "s3" {
    bucket         = "my-company-tfstate"
    key            = "prod/network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-locks"   # for locking (pre-1.10)
    # Terraform 1.10+ can use use_lockfile = true (S3-native locking, no DynamoDB needed)
  }
}

Terraform 1.10+: The S3 backend supports use_lockfile = true — it locks using S3 conditional writes, no DynamoDB required. DynamoDB locking is still widely used in older codebases.

Other common backends

⚠️ The chicken-and-egg problem

To use an S3 backend, you need an S3 bucket + DynamoDB table. But if you create them with Terraform, where does that state go?

Standard bootstrap approach:

1. Create the bucket + table using Terraform with local state (a separate "bootstrap" project).
2. Or create them manually with the AWS CLI.
3. All other projects then point their backend at these resources.
4. (Optional) Migrate the bootstrap project's own state into the bucket with terraform init -migrate-state.

2.4. Partial backend configuration

You can't use variables inside a backend block — Terraform evaluates backends before variables are loaded. The workaround: declare the backend type only, pass the rest at init time.

# backend.tf — only declare the type
terraform {
  backend "s3" {}
}

# Pass the details at init
terraform init -backend-config="bucket=my-tfstate" \
               -backend-config="key=prod/terraform.tfstate" \
               -backend-config="region=us-east-1"
 
# Or use a file:
terraform init -backend-config=backend-prod.hcl

backend-prod.hcl:

bucket = "my-tfstate"
key    = "prod/terraform.tfstate"
region = "us-east-1"

This is the key technique for using one codebase across multiple environments (Module 07).

2.5. State manipulation commands

⚠️ State surgery warnings

• state rm followed by apply can make Terraform create a duplicate — it now thinks the resource doesn't exist.
• state push with the wrong state = you've broken everything. Always state pull > backup.json first.
• Prefer moved blocks (Module 09) over state mv when possible — they're version-controlled and reviewable.

2.6. Import — bringing existing resources under management

This comes up constantly: infrastructure was created manually (ClickOps), and now you want Terraform to manage it.

Old way: terraform import command

# terraform import <resource_address> <real_id>
terraform import aws_s3_bucket.legacy my-existing-bucket

After this, you have to manually write the resource "aws_s3_bucket" "legacy" {} block to match reality, then run plan until you see "No changes."

New way (Terraform 1.5+): import block — recommended

import {
  to = aws_s3_bucket.legacy
  id = "my-existing-bucket"
}
 
resource "aws_s3_bucket" "legacy" {
  bucket = "my-existing-bucket"
  # ... other attributes
}

The real time-saver: Terraform 1.5+ can generate the resource block for you:

terraform plan -generate-config-out=generated.tf

Terraform writes the resource block based on what's actually deployed. You clean it up. Much faster when importing dozens of resources at once.

2.7. Drift — when reality diverges from state

Drift happens when someone modifies a resource outside Terraform (e.g. changes a security group rule in the AWS console).

# Detect drift without changing anything
terraform plan -refresh-only
 
# Accept the new reality into state (without touching infrastructure)
terraform apply -refresh-only

When you run a normal plan, Terraform will want to bring reality back to match config — it'll plan to undo the manual change. That's a feature, not a bug. Config is the source of truth.

2.8. State security

• Encryption at rest: encrypt = true (S3) + enable SSE/KMS on the bucket.
• Encryption in transit: backends use HTTPS.
• Restrict bucket access with tight IAM policies.
• Enable versioning on the bucket — rollback is possible when you need it.
• ⚠️ State always contains secrets in plaintext — even sensitive values. Securing state is non-negotiable.
• Terraform 1.x State Encryption (available in OpenTofu, in development for Terraform) — encrypts values inside the state file itself.

2.9. Hands-on Labs

Go to hands-on/ and follow LAB.md:

1. Lab 1 — Inspect local state: state list/show, read the JSON structure.
2. Lab 2 — state mv and state rm (safe, using the local provider).
3. Lab 3 — Import an existing resource using the import block.
4. Bootstrap — Sample files to create an S3 backend + DynamoDB lock table.

✅ Module 02 completion criteria

• Can explain why you shouldn't commit state to Git or use local state for a team.
• Can configure an S3 backend with locking.
• Understand partial backend config and why it matters for multi-environment setups.
• Know the difference between state rm (unmanage) and destroy (delete).
• Can import a resource using the import block and -generate-config-out.
• Can detect and handle drift using -refresh-only.

➡️ Next: Module 03 — Variables & Outputs