Cloud Infrastructure Checklist

DOCUMENT CONTROL BLOCK

• Document ID: SOP-OPS-TR-8842
• Effective Date: October 24, 2023
• Version: 4.2.0
• Review Cadence: Semi-Annual
• Target Audience: Cloud Operations Engineers, Site Reliability Engineers (SRE), DevSecOps Specialists, Infrastructure Architects

---

1. Executive Summary & Scope

This Standard Operating Procedure (SOP) defines the institutional-grade governance, validation, and execution framework for provisioning, auditing, and hardening cloud infrastructure across all Template Registry production and staging environments.

Operational Boundaries: This procedure applies to all Infrastructure-as-Code (IaC) deployments, cloud resource modifications, and container orchestration clusters deployed across AWS, Azure, and GCP.

Regulatory Standards: Execution of this SOP ensures mandatory compliance with SOC 2 Type II (Trust Services Criteria), ISO/IEC 27001:2022 (Information Security Management), and GDPR Article 32 (Security of Processing). Deviation from these operational parameters without an approved Engineering Change Request (ECR) constitutes a policy violation.

---

2. Prerequisites & Tool Requirements

Prior to initiating any cloud infrastructure deployment or verification cycle, personnel must secure and verify the following access vectors, tooling, and credentials:

• Software & Tooling:
  • Terraform v1.5.x or OpenTofu equivalent (pinned to state locking backend).
  • AWS CLI v2 / Azure CLI / gcloud SDK (configured with short-lived session tokens via IAM Identity Center).
  • TFLint and Checkov (Static Analysis Security Testing for IaC).
  • Kubectl (matched to target cluster control plane version $\pm 1$).
• Access & Permissions:
  • Multi-Factor Authentication (MFA) enforced on all cloud provider management consoles.
  • Temporary elevated execution roles via Privileged Access Management (PAM) infrastructure.
• Certifications:
  • Active Certified Solutions Architect (AWS/Azure/GCP) or Certified Kubernetes Administrator (CKA) designation for the lead operator.

---

3. Roles & Responsibilities (RACI Matrix Table)

---

4. Step-by-Step Operational Procedure

Phase 1: Pre-Flight & Static Analysis Validation

• 1.1 Authenticate to the designated cloud management plane using transient, role-assumed credentials with least-privilege boundaries.
• 1.2 Pull the latest master branch of the infrastructure repository and initialize the remote state locking backend.
• 1.3 Execute static code analysis on all Terraform/OpenTofu modules using checkov -d . --framework terraform to catch misconfigurations.
• 1.4 Validate syntax, formatting, and provider dependencies via terraform validate and terraform fmt -check.
• 1.5 Generate and review the execution plan (terraform plan -out=tfplan.binary) ensuring zero unauthorized state destructions or data-store replacements.

Phase 2: Network Topography & Core Security Baseline Deployment

• 2.1 Verify Virtual Private Cloud (VPC) / Virtual Network (VNet) CIDR block allocations against the Enterprise IP Address Management (IPAM) registry to prevent overlap.
• 2.2 Confirm segregation of Public, Private, and Database subnets across a minimum of three availability zones (AZs).
• 2.3 Validate that Internet Gateways (IGW) and NAT Gateways are provisioned strictly in accordance with ingress/egress security boundaries.
• 2.4 Apply hardened Security Groups and Network Access Control Lists (NACLs), ensuring default-deny posture on all inbound ports.
• 2.5 Provision and configure centralized Transit Gateways (TGW) or Virtual WAN hubs for inter-VPC traffic routing.

Phase 3: Compute, Storage, & Identity Provisioning

• 3.1 Deploy compute resources (EC2, AKS/EKS nodes, or Cloud Run instances) using immutable AMIs/base images sourced from the Template Registry hardened golden image pipeline.
• 3.2 Ensure all block storage volumes (EBS/Managed Disks) are encrypted at rest using Customer Managed Keys (CMK) via AWS KMS or Azure Key Vault.
• 3.3 Provision Object Storage buckets (S3/Blob) with public access blocks strictly enabled, object versioning active, and lifecycle transition rules enforced.
• 3.4 Bind fine-grained IAM roles to compute workloads using instance profiles or Workload Identity, eliminating long-lived static access keys.

Phase 4: Observability, Logging, & Verification

• 4.1 Enable comprehensive audit logging (AWS CloudTrail, Azure Activity Logs, GCP Cloud Audit Logs) routing to an immutable, write-once-read-many (WORM) centralized logging bucket.
• 4.2 Deploy observability agents (Datadog, Prometheus, or CloudWatch agent) configured to scrape host metrics, container performance, and application APM traces.
• 4.3 Configure CloudWatch/PagerDuty alert hooks for high-severity anomalies: CPU utilization $> 85%$, unauthenticated IAM requests, and security group modifications.
• 4.4 Execute post-deployment smoke tests verifying network routing, DNS resolution, and internal health-check endpoints.

---

5. Quality Assurance & Verification Criteria

To achieve certification and sign-off from the Office of the Chief Operations Architect, the deployed infrastructure must meet the following empirical benchmarks:

• Audit Checklist:
  • Zero critical or high-severity vulnerabilities flagged during automated IaC scanning.
  • All storage volumes and databases verified as encrypted using approved enterprise KMS keys.
  • Publicly accessible endpoints restricted strictly behind Layer 7 Web Application Firewalls (WAF) and Load Balancers.
• Tolerance Thresholds:
  • Network latency between internal microservices $\leq 15\text{ms}$ round-trip time (RTT).
  • Infrastructure provisioning execution time $\leq 20\text{ minutes}$ from clean state initialization.
• Acceptance Benchmarks:
  • Successful execution of integration test suite against the target environment.
  • Sign-off recorded in the Template Registry Change Management portal by both the Lead Engineer and the Security Officer.

---

6. Practical Troubleshooting Scenarios

Scenario A: Terraform State Lock Contention / Orphaned Lock

• Symptom: terraform apply fails with an error indicating that the state file is locked by a stale process or terminated pipeline.
• Immediate Corrective Action Protocol:
  1. Verify via CI/CD orchestration dashboard (GitHub Actions / GitLab CI) that no concurrent pipelines are actively modifying the workspace.
  2. Query the DynamoDB state locking table (or storage lock equivalent) to identify the LockID and Info payload.
  3. Execute terraform force-unlock <LOCK_ID> only after confirming with the engineering team that no upstream modifications are in flight.
  4. Re-run terraform refresh to synchronize local memory with the remote state backend before attempting re-deployment.

Scenario B: IAM Role Assumption Failure (Access Denied / 403)

• Symptom: Deployment pipeline halts during Phase 1 or 2 with AccessDenied or NotAuthorized exceptions when interacting with cloud APIs.
• Immediate Corrective Action Protocol:
  1. Inspect the execution environment variables to verify correct temporal token generation (AWS_SESSION_TOKEN, AZURE_CLIENT_ID, etc.).
  2. Validate that the IAM execution policy includes the exact resource ARNs targeted by the current Terraform module manifest.
  3. Verify that organizational Service Control Policies (SCPs) or Azure Policy assignments do not explicitly deny the attempted operation (e.g., prohibiting unencrypted S3 bucket creation).
  4. If token expiration is identified, trigger an immediate credential refresh via the enterprise PAM gateway and re-authenticate.