Published6 min read

The 30-Minute "Instant" Cloud Deployment: What AWS Is Actually Doing While Terraform Waits

Cloud marketing promises infrastructure in seconds, yet a greenfield multi-region Terraform apply still leaves you staring at a spinner for half an hour. Here is what is actually happening inside the AWS control plane while you wait.

If you listen to cloud keynotes, modern Infrastructure as Code (IaC) is practically instantaneous. You write a few declarative blocks of HCL, type terraform apply, and a globally distributed edge architecture materializes out of thin air.

Then you actually run a greenfield terraform apply on a brand-new multi-region AWS stack, and you spend the next twenty-five minutes watching your terminal print:

aws_acm_certificate_validation.cert: Still creating... [3m10s elapsed]
aws_s3control_multi_region_access_point.mrap: Still creating... [14m40s elapsed]
aws_cloudfront_distribution.mrap_distribution: Still creating... [6m20s elapsed]

After a decade of working on large-scale network and messaging infrastructure, I actually find that 30-minute wait reassuring. Whenever a global system claims to provision “instantly,” it usually means it hasn’t actually propagated anything to the edge yet—it just wrote a row to a regional database and handed you a 200 OK while the real convergence happens asynchronously.

When you provision a stack like the one behind this site (documented in Part 2 of our setup series)—regional S3 buckets, an S3 Multi-Region Access Point (MRAP), ACM TLS, Route 53, and CloudFront—Terraform isn’t just flipping flags in a single data center. It is waiting on three very real, globally distributed state machines to converge across the planet.

Here is what is actually happening under the hood during that 30-minute coffee break.


Timeline of a Greenfield Multi-Region terraform apply

Because Terraform builds a Directed Acyclic Graph (DAG) of resource dependencies, it cannot attach an S3 Multi-Region Access Point policy until the MRAP exists, and it cannot finish provisioning the CloudFront distribution until both the MRAP origin and the us-east-1 TLS certificate are live:

00:00Regional S3 Buckets& IAM Roles (~15s)

00:15 – 04:00ACM DNS Validation(Route 53 INSYNC + ACM Poll)

00:15 – 18:00S3 Multi-Region Access Point(Global Anycast Routing Setup)

18:00 – 26:00CloudFront Edge Propagation(600+ Global PoPs + SigV4a OAC)

26:00Route 53 Alias Records& MRAP Policy Lock (~15s)

Each of those three slow stages is constrained by a different piece of distributed systems physics.


Stage 1: The 4-Minute DNS Handshake (aws_acm_certificate_validation)

Creating a public TLS certificate in AWS Certificate Manager (aws_acm_certificate) takes less than a second. Validating that you actually control the domain (aws_acm_certificate_validation) takes anywhere from two to five minutes.

Why the pause?

  1. Route 53 Anycast Convergence: When Terraform writes the _acme-challenge style CNAME validation record into your Route 53 Hosted Zone, Route 53 doesn’t consider the change committed until every authoritative Route 53 nameserver across its global anycast network acknowledges the new zone serial (INSYNC).
  2. Negative Caching & Backoff Polling: ACM’s validation workers query public authoritative DNS from multiple vantage points to prevent BGP hijacking of domain validation. Rather than hammering your nameservers every 100 milliseconds, the validation state machine polls on an exponential backoff schedule.
Warning

The External Registrar Trap: If you registered your domain outside AWS and forgot to update your registrar’s NS records to match the four nameservers in your Route 53 Hosted Zone before running terraform apply, aws_acm_certificate_validation will sit at Still creating... for 45 minutes until the Terraform resource timeout finally kills the run.


Stage 2: The 15-Minute Heavy Lifter (aws_s3control_multi_region_access_point)

The single longest wait in the entire stack—often 12 to 20 minutes on creation or deletion—is aws_s3control_multi_region_access_point.

At first glance, an MRAP looks like a simple alias hostname (<alias>.accesspoint.s3-global.amazonaws.com) pointing at a list of S3 buckets. If it were just a CNAME or a round-robin DNS record, it would provision in ten seconds.

Instead, an S3 Multi-Region Access Point provisions a dedicated global routing construct on the AWS Global Accelerator / S3 edge network:

  1. Global Anycast Endpoint Allocation: AWS allocates a unique global alias and registers routing policies across its edge locations worldwide so that any request signed with SigV4a and addressed to your MRAP hostname is ingested at the nearest AWS edge PoP over the AWS private backbone.
  2. Cross-Region Health & Latency Probing: The MRAP control plane binds your underlying regional S3 buckets (e.g., Tokyo ap-northeast-1, Ireland eu-west-1, N. Virginia us-east-1, Oregon us-west-2, Stockholm eu-north-1) into an active-active routing table, initializing continuous health checks and replication state tracking across each region’s s3control endpoint.
  3. Asynchronous Request-Token Polling: Under the hood, s3control:CreateMultiRegionAccessPoint doesn’t return an access point—it returns a RequestTokenARN. The Terraform AWS provider has to loop on DescribeMultiRegionAccessPointOperation until every participating region reports that its local data plane has converged.

That is also why MRAP bucket membership is immutable after creation: you aren’t editing a JSON array in a database; you are provisioning a multi-region data-plane routing fabric.


Stage 3: Pushing Config to 600+ Edge PoPs (aws_cloudfront_distribution)

Once the ACM certificate is issued and the S3 MRAP is online, Terraform finally creates aws_cloudfront_distribution—which adds another 5 to 10 minutes to the clock.

CloudFront’s architecture splits the world into a centralized Control Plane (in us-east-1) and a massively distributed Data Plane spanning over 600 Points of Presence (PoPs) and Regional Edge Caches worldwide.

When you create a distribution with a viewer-request CloudFront Function and native SigV4a Origin Access Control (OAC):

  • The us-east-1 control plane compiles your routing table, TLS SNI bindings (example.com, www.example.com), Managed-SecurityHeadersPolicy, and the bytecode for your viewer-request JavaScript function.
  • It then pushes that bundle out to every edge location around the globe and waits for the fleet to report Deployed before Terraform marks the resource complete.

Cold Provisioning vs. Warm Operations

The contrast between Day 0 provisioning and Day 2 operations is the real takeaway here:

OperationWhat Moves Across the WireTypical Duration
Initial Greenfield terraform applyACM DNS validation + MRAP global anycast setup + CloudFront 600-PoP propagation~20–30 minutes
Subsequent terraform plan / applyControl-plane state diff against S3 .tfstate (.tflock)~8–15 seconds
Warm Astro Production Build (npm run build)Incremental HTML compilation + .cache/ WebP/AVIF & SHA-256 OG card reuse~1.5–2.0 seconds
Incremental S3 Deploy (scripts/s3-deploy.sh)Single list-objects-v2 ETag diff + parallel upload of changed files + targeted CF invalidation~5–10 seconds

Paying a one-time 25-minute “physics tax” on Day 0 is what buys you sub-10-second deployments and automatic multi-region failover for the rest of the site’s life. Next time terraform apply hangs at 14m00s elapsed on an S3 Multi-Region Access Point, don’t kill the terminal—AWS is just wiring up your global backbone.

Ed Stuurman

About the author

Ed Stuurman

Infrastructure and reliability engineer based in Japan. Passionate about resilient systems, bare-metal hardware, cloud automation with Terraform, and all things homelab.

Contact

Thoughts or questions on this article?

Found a bug, tested a similar setup, or want to discuss the details? Drop a message below.

Emails are delivered directly to ed@stuurman.com.