openshift-lb-envoy

Envoy in front of one public IP, fronting many OpenShift clusters. SSL passthrough — OpenShift owns its certs; Envoy just routes by SNI. All driven from docker-compose.yml — no custom image, no separate config file.

                                  ┌──► hammer2 SNO  (172.16.1.50)
Internet ──► <public_ip> ──► Envoy ──► anvil   SNO  (172.16.1.51)
              80 / 443 / 6443          └──► …

How it routes

Listener	Mechanism	Routes by
:443	TLS ClientHello SNI → TCP passthrough	*.apps.<cluster>
:6443	TLS ClientHello SNI → TCP passthrough	api.<cluster>
:80	HTTP Host header	*.apps.<cluster>

No TLS termination at Envoy. oc/kubectl work because mTLS isn't broken. Browsers see OpenShift's certs.

Architecture

Two services in docker-compose.yml:

• config — stock python:3.13-alpine. Decodes a base64-embedded Python script, reads the CLUSTERS env var, writes /etc/envoy/envoy.json into a shared Docker volume, exits.
• envoy — stock envoyproxy/envoy:v1.32-latest. Depends on config completing successfully, then reads the generated JSON.

No docker build. No bind-mounted source files. Ship docker-compose.yml to the host and run.

Usage

Edit the CLUSTERS env var in docker-compose.yml:

CLUSTERS: |
  [
    {"name": "hammer2", "base_domain": "hammer2.na-launch.com", "ingress_ip": "172.16.1.50"},
    {"name": "anvil",   "base_domain": "anvil.na-launch.com",   "ingress_ip": "172.16.1.51"}
  ]

Each cluster needs name, base_domain, ingress_ip. api_ip is optional and defaults to ingress_ip (SNO).

Then:

docker compose up -d --force-recreate     # apply / reapply
docker compose logs envoy
docker compose down

--force-recreate is what re-runs the config container so env-var changes take effect.

Deployment

1. NAT the public IP to the host running compose on ports 80, 443, 6443.
2. DNS per cluster:

   *.apps.<cluster>.<domain>   → <public_ip>
   api.<cluster>.<domain>      → <public_ip>
   

3. Reachability: the host must reach each cluster's ingress / API IPs on 80/443/6443.
4. docker compose up -d.

Verify

# OpenShift's cert should appear, not anything from Envoy:
openssl s_client -connect <public_ip>:443 \
  -servername console-openshift-console.apps.<cluster>.<domain> </dev/null 2>/dev/null \
  | openssl x509 -noout -issuer -subject

# Console:
curl -vk https://console-openshift-console.apps.<cluster>.<domain>/

# API via passthrough:
oc login -u kubeadmin https://api.<cluster>.<domain>:6443

Inspect the generated Envoy config:

docker run --rm -v $(basename $PWD)_envoy-config:/cfg alpine cat /cfg/envoy.json

Editing the generator

If you need to change the Envoy config schema (add headers, tweak ports, etc.), edit generate.py, then refresh the base64 in compose:

./embed.sh
docker compose up -d --force-recreate

generate.py and embed.sh are dev-time only — not needed at deploy time. The base64 in docker-compose.yml is the canonical script at runtime.

Constraints

• One cluster per base_domain — SNI can't disambiguate duplicates.
• Admin interface bound to 127.0.0.1:9901; SSH-tunnel for stats.
• Pure passthrough — no per-route HSTS, header rewrites, or cert termination at Envoy.

Layout

docker-compose.yml   the only runtime artifact (contains embedded script + clusters)
generate.py          source of truth for the embedded script
embed.sh             base64-embeds generate.py into docker-compose.yml