Running a cloud native platform where there is no cloud
Some life sciences data has to stay inside a closed network. Hospital systems, defense and government research, and organizations with strict data residency rules run environments with no route to the public internet. If a cloud based platform cannot run in that environment, it is not useful to the team that needs it.
Why this is hard
Managed services are absent
Cloud native platforms are built on managed services — object storage, managed databases, message queues, identity providers, container orchestration — all provided by the cloud vendor and all absent in an air gapped environment. Every one of those dependencies has to be replaced with something that runs locally, and the replacement has to deliver the same behavior, or the platform behaves differently on premises than it does in the cloud and the team ends up supporting two products.
Updates
Software that expects to pull updates, container images, and dependencies from the internet cannot. Everything has to be bundled and delivered offline, verified, and installed by a process that works without a connection.
Scale
The elasticity of the cloud — spinning up compute for a large analysis and releasing it after — does not exist on fixed hardware. Workloads have to be scheduled against what is physically there.
How we approach the deployment
We use containers so each part of the platform carries its own dependencies. That makes the deployment less dependent on the underlying hardware. Kubernetes, or a lighter orchestrator where hardware is limited, provides the scheduling and resilience the cloud used to.
Managed cloud services are swapped for self hosted equivalents — object storage for a local object store, managed databases for the same database running in the cluster — with the swap made behind an abstraction so application code does not change.
The on premises version needs to provide the same core features as the cloud version. Otherwise the two deployments start to diverge. That is easier when both deployment options are considered from the beginning rather than added later.
No external dependencies
A genuinely air gapped deployment makes no calls out.
No license checks against a remote server.
No telemetry.
No font or library loaded from a CDN.
No update check on startup.
No outbound call of any kind.
Each one can cause a failure that is difficult to spot from outside the network, so finding and removing those dependencies is part of the deployment work.