Upgrading an air-gapped API gateway: zero-downtime updates without internet access

Deploying an API gateway inside an air gap is a solved problem. Keeping it current is where most teams get stuck. An air-gapped API gateway upgrade has to move a new release across a controlled boundary, apply it to a platform that is serving live traffic, and prove afterwards that nothing regressed. There is no vendor control plane to orchestrate the rollout, no registry to pull from, and no support engineer who can look at your environment. The upgrade process is yours, end to end, and your accreditation documentation probably requires you to describe it in writing.

The teams that struggle are not the ones with the strictest boundaries. They are the ones running a gateway that was never designed to be updated offline. When the product assumes connectivity for licensing checks, plugin downloads, or control plane synchronization, every patch cycle becomes a negotiation with the vendor and the security office at the same time. The result we see in the field is predictable: air-gapped gateways run versions that are twelve or eighteen months old, because upgrading hurts more than the known vulnerabilities do.

This post walks through how a Zerq upgrade works in a fully disconnected environment: what crosses the boundary, how the rollout keeps serving traffic, how you roll back without a registry, and what the audit trail shows the accreditor afterwards.

Why connected-gateway assumptions break offline

Most gateway architectures assume the control plane is reachable. Apigee and Azure API Management manage hybrid data planes from a cloud tenant, so a data plane that can never reach that tenant is running in a degraded mode the vendor did not design for. AWS API Gateway does not have an offline mode at all. Kong's hybrid deployment keeps the control plane connected even when data planes are self-hosted, and enterprise plugin distribution assumes you can reach a package repository. Each of these products can be bent toward disconnection, but bending is the operative word: you inherit workarounds instead of a supported path, and every upgrade re-tests those workarounds. We covered the architectural pattern behind this in the control plane comparison.

The second failure mode is dependency surface. A gateway built on a JVM runtime with a plugin ecosystem has a large tree of artifacts that all need to cross the boundary at compatible versions. The more pieces a release has, the more ways an offline transfer can produce a partial upgrade, and partial upgrades are the specific thing a change advisory board in a classified environment exists to prevent.

The third is verification. In a connected environment, a bad rollout is an inconvenience: you check the vendor status page, pull a fixed image, move on. Inside an air gap, everything you need to diagnose and recover must already be inside the boundary before you start. An upgrade process that does not include a self-contained rollback path is not a process; it is a plan to have an incident.

What crosses the boundary in a Zerq release

Zerq ships as a small, fixed set of artifacts. The gateway and management API are a single Go binary packaged as one container image. The management UI and the developer portal are two more images. There is no plugin marketplace to mirror, no license server to reach, and no control plane outside your boundary; configuration and audit data live in your own MongoDB. That fixed artifact set is what makes the offline release process short enough to run every patch cycle instead of twice a year.

On the connected side, a release batch is built with pinned tags:

cd deploy
# Build and tag all release images with one pinned version.
# Mutable tags like "latest" are never used: the tag is the
# unit of change control, so it must name exactly one build.
IMAGE_TAG=v1.10 ./build-push.sh

# Export each image to an archive for transfer.
docker save -o zerq-backend-v1.10.tar backend-v1.10
docker save -o zerq-frontend-v1.10.tar frontend-v1.10
docker save -o zerq-developer-portal-v1.10.tar developer-portal-v1.10

Alongside the archives, keep a signed manifest of image names and tags for each release batch. This is the document your transfer authority checks at the boundary and your auditor checks afterwards:

{
  "release": "v1.10",
  "created": "2026-09-17",
  "images": [
    { "name": "backend-v1.10",          "sha256": "9f2c…" },
    { "name": "frontend-v1.10",         "sha256": "41ab…" },
    { "name": "developer-portal-v1.10", "sha256": "d708…" }
  ]
}

Runtime environment files and certificate material travel separately from the image archives. Mixing them in one bundle is the most common cause of a failed offline deployment we see: the images import cleanly, but the identity or TLS files that match the new release never made it across, and the stack comes up half-configured.

On the offline host, import and verify before touching the running stack:

docker load -i zerq-backend-v1.10.tar
docker image ls | rg "backend-|frontend-|developer-portal-"
docker compose -f deploy/docker-compose.platform.images.yml config | rg "image:"

The last command matters more than it looks. It confirms that the compose file resolves backend, frontend, and developer portal to the same release generation. A tag mismatch here means you are about to run a mixed-version platform, and the time to find that out is before the rollout, not during it.

The zero-downtime rollout, step by step

On Kubernetes, the upgrade is a standard rolling update, and Zerq's probe design is what makes it safe. The backend exposes two health endpoints. /livez answers process-level liveness only. /readyz is dependency-aware: it pings MongoDB and, when Redis is the configured cache, pings Redis too, each with a two-second timeout. A new pod does not report ready until it can actually reach the state it needs to serve traffic, so the orchestrator never routes requests to a replica that is still warming up.

Back up before you change anything. Run ./mongodb.sh backup from the deploy folder. It takes a BSON dump of the platform database with mongodump, preserving indexes and collection options, into a timestamped folder under deploy/mongo-backups/. This is your configuration state, not just data: collections, proxies, clients, profiles, policies, and audit history all live here.
Update the image references in your workload manifests to the new pinned tags and apply them. Then watch the rollout: kubectl -n zerq rollout status deploy/backend. Kubernetes replaces pods one at a time, and each new pod must pass /readyz before the old one is retired.
Confirm image tag consistency across pods in the same deployment with kubectl -n zerq get pods -o wide and check that restart counts are not climbing. Flapping readiness probes right after a rollout are the signature of a config object that changed without triggering the rollout, or a secret key the new version expects and cannot find.
Repeat for the frontend and developer portal deployments. They are stateless Next.js services and roll the same way.
Validate an end-to-end path through the gateway, not just the health endpoint:

curl -i https://api.example.com/health
curl -i https://api.example.com/orders/123 \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Client-ID: k8s-ops-client" \
  -H "X-Profile-ID: k8s-ops-profile"

A 200 here proves the full chain: ingress, gateway auth, profile enforcement, upstream connectivity. Checking auth outcomes matters as much as the happy path; a 401 or 403 that behaves differently after the upgrade is a regression even if health checks are green.

On Docker Compose, the same release lands with docker compose -f docker-compose.platform.images.yml up -d against the new pinned tags, followed by the same validation. Compose cannot give you overlapping replicas on one host, so for hard zero-downtime requirements, Kubernetes is the right deployment mode; that trade-off is part of why we recommend it for production in regulated environments.

Why a restarted replica is safe

Zero downtime during an upgrade depends on an architectural property, not on operator care: a gateway replica must be disposable. In Zerq, replicas hold no authoritative state. Configuration lives in MongoDB, rate limit counters and coordination locks live in Redis, and every backend pod that starts runs a full bootstrap sweep of MongoDB before it begins work, loading all eligible proxies and starting their scheduled workers. Config changes made during the rollout window reach every pod, old and new, through a Redis Pub/Sub broadcast on the workflow.trigger.sync channel, so there is no polling interval during which a new pod serves stale routing. Scheduled work stays exactly-once through the mixed-version window because cron execution is guarded by a per-slot Redis lock, so an old pod and a new pod never both fire the same trigger. An engineer drawing this on a whiteboard needs three boxes: stateless gateway replicas at the top, MongoDB as the source of truth for config and audit below, Redis beside it carrying shared counters, locks, and the change broadcast. The upgrade replaces the top boxes one at a time while the bottom two never restart.

Rollback without a registry

Inside an air gap, rollback capability is something you stage, not something you fetch. The previous release's images are still loaded on your hosts or nodes, so rolling back is a redeploy of the last known-good pinned tags: kubectl -n zerq rollout undo deploy/backend on Kubernetes, or pointing compose back at the prior tags. Because the data schema lives in MongoDB and the pre-upgrade mongodb.sh backup is sitting on disk, a worst-case recovery restores that dump with ./mongodb.sh restore and redeploys the old images, in that order: data dependencies first, then identity, then backend and UI services, then validation of one real consumer path. Keep at least one prior release batch inside the boundary at all times. The day you need it is the day you cannot go get it.

The audit trail your accreditor will ask for

Every administrative change involved in an upgrade cycle is recorded in Zerq's audit log, stored as structured JSON in your MongoDB and visible to the dedicated Auditor role. When your team promotes configuration between enclaves using collection export and import, each import lands as an audited management action, and imported proxies arrive in draft status so a human publishes them deliberately rather than by side effect.

A single entry answers the questions a change review actually asks:

{
  "timestamp": "2026-09-17T09:41:22Z",
  "actor_id": "j.reyes",
  "actor_type": "user",
  "action": "UPDATE",
  "resource_type": "proxy",
  "resource_id": "68c1f0a2…",
  "http_method": "PUT",
  "url": "/api/v1/proxies/68c1f0a2…",
  "ip_address": "10.40.2.17",
  "request_id": "req_7f3d…",
  "response_status": 200
}

actor_id comes from the OIDC token, so it names a real identity in your own IdP, not a shared admin login. actor_type distinguishes a human from service automation, which matters when your CI pipeline applies configuration. The request body is captured on create and update, so the reviewer sees what changed, not just that something changed. The separation of duties is structural: the Auditor role can read all of this and modify none of it.

A pre-upgrade checklist for your air-gapped API gateway

Does the signed release manifest list every image, tag, and digest, and does the transferred set match it exactly?
Do backend, frontend, and portal resolve to the same release generation in your manifests before rollout?
Is a fresh mongodb.sh backup on disk inside the boundary, and has a restore of a previous backup been tested in the last quarter?
Are the previous release's images still present locally for rollback?
Did runtime env files and certificates transfer with this release, and do they match what the new version expects?
Is there a validation script that exercises one authenticated consumer path, not just /health?
Who reviews the audit log after the change window, and do they hold the Auditor role rather than admin?

If any answer is no, the upgrade is not ready, whatever the calendar says.

What this looks like in practice

A defence integrator runs Zerq inside a classified enclave: gateway, management UI, portal, MongoDB, Redis, and Keycloak, all within the boundary, serving mission APIs to internal systems. Before adopting a defined offline release process, upgrades happened roughly twice a year because each one required a bespoke transfer request, and the platform routinely ran months behind on patches, which their security office flagged at every assessment.

Now a release batch is built and signed on the connected side each patch cycle, walked through the cross-domain transfer with its manifest, and imported onto the cluster. The rollout runs during business hours because readiness gating keeps traffic on healthy pods throughout; consumers see no interruption and no change in auth behavior. The previous batch stays staged for rollback, the pre-upgrade Mongo dump satisfies the recovery-point question in their accreditation package, and the audit log export gives the assessor a complete, per-identity record of every configuration change in the window. Patch latency dropped from months to weeks, and the upgrade procedure went from a bespoke event to a documented routine, which is precisely what government and public sector accreditation frameworks want to see.

Closing

An air-gapped API gateway upgrade should be a routine, not a project. Zerq keeps the artifact set small enough to transfer and verify: three images and a signed manifest. Probes gate traffic so rollouts replace replicas without dropping requests. State lives in your MongoDB and Redis, so replicas are disposable and rollback is a staged, local operation. And the audit trail records every change with a real identity attached, inside your boundary, where your compliance team can read it and nobody outside can.

Zerq is an enterprise API gateway built for regulated industries — one platform for API management, AI agent access, compliance audit, and developer portal, running entirely in your own infrastructure. See how it works or request a demo to walk through your specific requirements.