Cloud Run GPU Image Update - Quota Bypass

Problem

When deploying updated container images to an existing Cloud Run service with GPU (e.g., NVIDIA L4), gcloud run deploy fails with quota errors even though the service is already running with a GPU. The quota check blocks both zonal and non-zonal redundancy configurations, making it impossible to deploy updated code.

Context / Trigger Conditions

• gcloud run deploy returns:

  metadata.annotations[run.googleapis.com/maxScale]: You do not have quota for
  using GPUs with zonal redundancy.
  

  Followed by:

  metadata.annotations[run.googleapis.com/maxScale]: You do not have quota for
  using GPUs without zonal redundancy.
  

• The GPU service already exists and has a running revision
• You're trying to deploy an updated container image, not change GPU configuration
• The deploy script uses gcloud run deploy with --gpu flags

Solution

Instead of gcloud run deploy (which re-validates all resource quotas), use gcloud run services update which only updates the specified fields on the existing service:

# Instead of this (fails with quota error):
gcloud run deploy divine-transcoder \
  --image gcr.io/PROJECT/divine-transcoder \
  --region us-central1 \
  --gpu 1 --gpu-type nvidia-l4 \
  --cpu 4 --memory 16Gi \
  ...

# Use this (updates image on existing service):
gcloud run services update divine-transcoder \
  --region us-central1 \
  --image gcr.io/PROJECT/divine-transcoder:latest

Key differences:

• gcloud run deploy creates a new service or replaces the full configuration, triggering quota checks
• gcloud run services update --image only updates the container image on the existing service, preserving all existing GPU/CPU/memory configuration without re-validating quotas

Verification

# Verify new revision is active
gcloud run revisions list --service SERVICE_NAME --region REGION --limit=3

# Check the service is serving traffic
gcloud run services describe SERVICE_NAME --region REGION --format='value(status.url)'

Example

# Build and push updated image
docker build --platform linux/amd64 -t gcr.io/my-project/divine-transcoder .
docker push gcr.io/my-project/divine-transcoder

# Update only the image (bypasses GPU quota re-validation)
gcloud run services update divine-transcoder \
  --region us-central1 \
  --image gcr.io/my-project/divine-transcoder:latest

# Output:
# Deploying...
# Creating Revision...done
# Routing traffic...done
# Service [divine-transcoder] revision [divine-transcoder-00010-vf4] has been deployed

Notes

• This only works for existing services that already have GPU configured
• If you need to change GPU type, CPU, memory, or other settings, you'll need to request additional quota via https://g.co/cloudrun/gpu-quota
• First-time GPU deployments in a region get automatic quota of 3 GPUs (non-zonal)
• Quota increases for non-zonal redundancy are granted more quickly than zonal
• If deploy scripts use gcloud run deploy, consider adding a fallback to gcloud run services update --image when quota errors are detected

References

• Cloud Run GPU support for services
• Cloud Run Quotas and Limits
• Cloud Run GPUs GA announcement