It’s been five months since I rebuilt my website using Astro and decided to chuck the static files into Google Cloud Storage (GCS), fronted by a load balancer and with Cloud CDN enabled. However, the static files were held in a single multi-regional storage bucket, causing an increase in latency for all users outside of the EU region. This design choice meant users outside of Europe would more than likely find this website loaded slower for them compared to users in Europe.

I wasn’t happy with this as I wanted global users to have a similar experience, a speedy experience. I finally decided to do two things:

• Tidy up the many deployed websites/web services, consolidating them into a single Google Cloud project.
• Migrate my Astro website from Google Cloud Storage into Cloud Run.

Changing the Design

You might be wondering why I’d change the infrastructure design so soon after the launch of the new site. I was wondering the same. I could have created more storage buckets across the globe and set up the necessary bucket backends, but the configuration would not have been as “clean” as Cloud Run would have been. In addition to the cleaner infrastructure design, you understand that users outside of Europe were potentially seeing a 5-7x increase in load times, and I knew I had to look at improving their speeds.

Here’s why I wanted to migrate the infrastructure to Cloud Run.

• I love Cloud Run.
• Non-European users had a 5-7x increase in load times.
• In Google Cloud, to reduce latency and page load times for non-European users, operational maintenance would’ve been higher than I wanted.
• I’ve been looking at learning Go.

Measuring Current Performance

How do I measure the load times of my website? For the last three years, I’ve used a service called UpDown.io (here’s a sexy referral link). This service is similar to other services, like Pingdom, which sends a request to said webpage and reports back on how successful that request is. UpDown.io focuses on the impact on the user and uses APDEX to determine how satisfied users will be. Here’s what the APDEX looked like across different source regions.

The APDEX is consistently below 1.0, giving users a less-than-perfect experience. What does this mean? Some users experience a page load time of >1s. Realistically, this website loads faster than most, but I want users to have as good of an experience as possible.

Going Compute Serverless

I may have created more work by switching to Cloud Run, but it is my preferred strategic option. Yes, I have extra work to complete to get my statically generated site to work in Cloud Run, but it’s worth it in the long run.

Here’s the plan:

1. Migrate this existing website from one Google Cloud project to another.
2. Create an HTTP server in Go.
3. Create a CI pipeline in Cloud Build.
   • The pipeline will generate the static files using Astro, build and package the HTTP server, create a Docker image, deploy the image to three Cloud Run instances, invalidate the CDN cache and push the updated artefacts to a Cloud Storage bucket¹.
4. Front the Cloud Run services with a load balancer.
5. Point the existing j.gog.gs A record to the new load balancer.
6. Fingers crossed, Google can sort out their systems quickly to minimise downtime.

Here’s the proposed architecture that I’m wanting to move to. It is very similar to the architecture when using Google Cloud Storage.

The Code

Here’s all the code for the Go server, Dockerfile and Cloud Build pipeline. The code for the Go server is based on the work from AlexEdwards.net and TheDeveloperCafe.com.

// main.go
package main

import (
	"fmt"
	"log"
	"net/http"
)

var PORT = 8080

func main() {
	dir := http.Dir("./dist")
	fs := http.FileServer(dir)
	mux := http.NewServeMux()
	mux.Handle("/", fs)

	err := http.ListenAndServe(fmt.Sprintf(":%d", PORT), mux)
	if err != nil {
		log.Fatal(err)
	}
}

# Start by building the application.
FROM golang:1.22 as build

WORKDIR /go/src/app
COPY ./build/go/go.mod ./build/go/*.go ./

RUN CGO_ENABLED=0 go build -o /go/bin/main

# Now copy it into our base image.
FROM gcr.io/distroless/static-debian12
WORKDIR /app
COPY --from=build /go/bin/main /app
COPY ./dist ./dist
CMD ["/app/main"]

steps:
  # Retrieves cached images and extracts them
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    id: "gather-artifacts"
    script: |
      #!/usr/bin/env bash
      echo "Starting to gather cached artifacts from ${_ARTIFACT_BUCKET_LOCATION}."
      echo ""
      gcloud storage cp ${_ARTIFACT_BUCKET_LOCATION}/${_ARTIFACT_CACHE} .
      echo "Obtained cached artifacts ${_ARTIFACT_CACHE} from ${_ARTIFACT_BUCKET_LOCATION}."
      echo ""
      echo "Unzipping cache artifacts."
      tar -xf ${_ARTIFACT_CACHE}
      echo "Finished unzipping cache artifacts."
    waitFor:
      - "-"

  # Installs Node packages
  - name: "node:${_NODE_VERSION}-bookworm"
    id: "install"
    entrypoint: "npm"
    args: ["install"]
    waitFor:
      - "gather-artifacts"

  # Runs the build process to generate static website files
  - name: "node:${_NODE_VERSION}-bookworm"
    id: "build"
    entrypoint: "npm"
    args: ["run", "build"]
    waitFor:
      - "install"

  # Runs Cypress tests against the generates files
  - name: "cypress/included:13.7.0"
    id: "test"
    script: |
      npm install && npm run test
    waitFor:
      - "build"

  # Builds docker image
  - name: "gcr.io/cloud-builders/docker"
    id: "build-image"
    args:
      [
        "build",
        "-f",
        "go.Dockerfile",
        "-t",
        "${_ARTIFACT_IMAGE_NAME}:${TAG_NAME}",
        ".",
      ]
    waitFor:
      - "test"

  # Pushes docker image to Artifact Registry
  - name: "gcr.io/cloud-builders/docker"
    id: "push-image-to-ar"
    args: ["push", "${_ARTIFACT_IMAGE_NAME}:${TAG_NAME}"]
    waitFor:
      - "build-image"

  # Zips generated images to be cached and stored in GCS
  # Runs in parallel with below step
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    id: "update-artifacts"
    script: |
      #!/usr/bin/env bash
      echo "Starting to zip updated local cached artifacts."
      echo ""
      tar -cf ${_ARTIFACT_CACHE} ./cache
      echo ""
      echo "Finished zipping local cache artifacts. Ready to upload."
    waitFor:
      - "build-image"

  # Deploy docker image to EU region
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    id: "deploy-to-eu"
    entrypoint: "gcloud"
    args:
      - "run"
      - "deploy"
      - "{CLOUD_RUN_EU_SERVICE_NAME}"
      - "--project=${_GCP_PROJECT_DEPLOY}"
      - "--region={EU_REGION}"
      - "--service-account={SERVICE_ACCOUNT}"
      - "--image=${_ARTIFACT_IMAGE_NAME}:${TAG_NAME}"
      - "--platform=managed"
      - "--ingress=internal-and-cloud-load-balancing"
      - "--allow-unauthenticated"
      - "--timeout=30"
      - "--min-instances=0"
      - "--max-instances=10"
      - "--execution-environment=gen2"
      - "--cpu=1"
      - "--memory=512Mi"
      - "--no-use-http2"
      - "--no-cpu-boost"
    waitFor:
      - "push-image-to-ar"

  # Deploy to US region
  # Waits for EU region to deploy
  # Runs in parallel with below step
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    id: "deploy-to-us"
    entrypoint: "gcloud"
    args:
      - "run"
      - "deploy"
      - "{CLOUD_RUN_US_SERVICE_NAME}"
      - "--project=${_GCP_PROJECT_DEPLOY}"
      - "--region={US_REGION}"
      - "--service-account={SERVICE_ACCOUNT}"
      - "--image=${_ARTIFACT_IMAGE_NAME}:${TAG_NAME}"
      - "--platform=managed"
      - "--ingress=internal-and-cloud-load-balancing"
      - "--allow-unauthenticated"
      - "--timeout=30"
      - "--min-instances=0"
      - "--max-instances=10"
      - "--execution-environment=gen2"
      - "--cpu=1"
      - "--memory=512Mi"
      - "--no-use-http2"
      - "--no-cpu-boost"
    waitFor:
      - "deploy-to-eu"

  # Deploy to ASIA region
  # Waits for EU region to deploy
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    id: "deploy-to-asia"
    entrypoint: "gcloud"
    args:
      - "run"
      - "deploy"
      - "{CLOUD_RUN_ASIA_SERVICE_NAME}"
      - "--project=${_GCP_PROJECT_DEPLOY}"
      - "--region={ASIA_REGION}"
      - "--service-account={SERVICE_ACCOUNT}"
      - "--image=${_ARTIFACT_IMAGE_NAME}:${TAG_NAME}"
      - "--platform=managed"
      - "--ingress=internal-and-cloud-load-balancing"
      - "--allow-unauthenticated"
      - "--timeout=30"
      - "--min-instances=0"
      - "--max-instances=10"
      - "--execution-environment=gen2"
      - "--cpu=1"
      - "--memory=512Mi"
      - "--no-use-http2"
      - "--no-cpu-boost"
    waitFor:
      - "deploy-to-eu"

  # Once all deployments are completed, this step invalidates the CDN cache
  # to allow new/updated files to becomes available to users
  - name: "gcr.io/google.com/cloudsdktool/cloud-sdk:slim"
    entrypoint: "gcloud"
    id: "invalidate-cdn"
    args:
      [
        "compute",
        "url-maps",
        "invalidate-cdn-cache",
        "${_URL_MAP}",
        "--path=/*",
        "--async",
        "--project=${_GCP_PROJECT_DEPLOY}",
      ]
    waitFor:
      - "deploy-to-us"
      - "deploy-to-asia"

timeout: 900s

options:
  automapSubstitutions: true
  substitutionOption: ALLOW_LOOSE

substitutions:
  _GCP_PROJECT_DEPLOY: "{DESTINATION_PROJECT_ID}"
  _NODE_VERSION: 20.11.1
  _ARTIFACT_BUCKET_LOCATION: "{BUCKET_INCLUDING_PATH}"
  _ARTIFACT_CACHE: "cache.tar.gz"
  _ARTIFACT_IMAGE_NAME: "{IMAGE_NAME}"
  _URL_MAP: "{URL_MAP_NAME}"

artifacts:
  objects:
    location: ${_ARTIFACT_BUCKET_LOCATION}
    paths: [$_ARTIFACT_CACHE]

Here’s a breakdown of the Cloud Build CI file.

1. Retrieve cache.tar.gz from Cloud Storage and extract files.
   • This cache file holds the cached assets, mainly images, to help speed up the build step.
2. Install Node packages.
3. Run Cypress tests against built files.
4. Build Docker image.
   • First step in the Dockerfile is to build the Go proxy server.
   • Second step is to copy the Go binary file from the first step into the image, and copy the dist directory from the pipeline into the image.
5. Docker image is pushed to Artifact Registry.
6. Image cache directory archive is created.
7. Deployment of the Docker image to EU region starts in parallel with step 6.
8. Once the deployment to the EU region is successful, the Docker image is deployed to US and ASIA regions.
9. The Cloud CDN cache is invalidated to ensure new pages and assets flush through to users.
10. As part of the artifacts section, once all build steps are completed the cache.tar.gz file is copied to Cloud Storage.

Eternal Glory

The results speak for themselves. Here is a breakdown of the performance gain across the regions.

The regions that benefit the most are North America and Australasia, which makes sense since there’s a Cloud Run service deployed into each of these continents, reducing the latency and page load times for users.

The takeaway is “how well your infrastructure performs is only as good as the design you create, on the platform you’ve chosen”. Ensure you’re designing your infrastructure to work with your use cases, focusing on user impact. Your users are integral in growing your business. Look after them.

Over One Month On

It’s been a month and a half since I migrated my website to the new infrastructure. You might be wondering what the response times are, given the infrastructure has had time to stabilise.

The migration has been a success! The latency numbers from UpDown.io are better on the new infrastructure. The reported APDEX number is highly consistent and stable, even with a reduced latency check of 0.5s.

Overall, the website runs faster for North American, Australasia and Asia users and is highly consistent. My takeaway is…

design your solution to meet your goals, taking into account limitations with the platform and tooling you’ve selected

My goals changed after I realised latency for non-European users was terrible.