Taking a Dockerised API Live: Every Step, and Everything That Broke(Part 4&5)
Part 4 — What I would tell myself before starting
- Suspect the step before the one that failed. Nearly every error here appeared downstream of its cause.
- Read the status code, not just the body. A timeout is a firewall. "Refused" is nothing listening. 502 is a wrong upstream. An HTML 404 signed by your proxy means the proxy never forwarded. An empty body is not success — and curl -s hides all of it.
- Know what is fixed at build time. Build arguments and copied files need a rebuild. Environment variables need up -d, not restart. A named volume keeps whatever owner it had the first time it was mounted.
- Configuration on disk is not configuration in memory. nginx -T shows the file; only a reload applies it.
- Profiles hide services from docker compose build.
- "It works on my machine" usually works for a boring reason. My user ID happened to be 1000. My database happened to have hand-inserted rows. Port 8080 happened to be free. List the boring reasons before you deploy.
- Development conveniences don't carry over. Bind mounts, override files and dependencies installed on the host are exactly what production doesn't have.
- Shell redirection happens before the command runs.
- A health check must touch what it claims to check.
- Generate secrets on the server. Never copy a development .env, and never paste real secrets into screenshots or articles.
- Every value you repeat will drift. One port number broke three things in three different files. Define it once.
Part 5 — Before you share the URL
• Demo credentials are public by nature. Anyone with the URL and the demo password can sign in. Use a server and database holding only fake data, and consider basic authentication or an IP allowlist at nginx until the demonstration is over.
• Only 80 and 443 open to the internet. The database publishes no port; the app is on loopback.
• Reserve a static IP, so a restart doesn't break DNS and certificate renewal.
• Rotating application or encryption keys invalidates every issued token and stored credential. Do it before real data arrives, not after.
• Confirm the development override isn't loaded — dcp config should show no published database port.
• Keep the repository private if it contains your schema, infrastructure or security reasoning.
The go-live checklist
Before the server
- Production image builds locally
- Production mode boots and resolves every route, checked in CI
- Database migrates from empty and seeds successfully
- Reference data comes from migrations, not hand-inserted rows
- Scripts use set -euo pipefail and psql -v ON_ERROR_STOP=1
- Full end-to-end demo passes twice in a row
- No credentials baked into test tooling
- Code committed, staged files checked for secrets, first commit read before pushing
- Decided: the project's own database, never a shared superuser connection
- Checked what owns ports 80 and 443 on the server, and chose the front door accordingly
On the server
- docker ps works without sudo, after a fresh login
- Code cloned or copied without dependency folders, runtime state or .env
- .env created from the template before building
- Build-time user ID and group ID set to your own user before building
- Secrets generated on the server; production mode on, debug off
- Every secret the app requires in production is non-empty
- Images built; profile-gated services checked with config --services
- Started with both -f files; config shows the app on 127.0.0.1 and no database port
- Host port confirmed free and written down
- Migrations run from the production image, with administrator credentials
- Runtime volume owned by the app user; signing keys generated and listed
- /readyz returns 200 on the server, with the status code checked
- Static external IP reserved; dig matches ifconfig.me
- Firewall allows 80 and 443, and nothing else publicly
- Front door proxies to the loopback port, with X-Forwarded-Proto, body size and the ACME challenge location
- nginx -t passes, and nginx reloaded
- Certificate issued; health, readiness and key endpoints all return 200 over HTTPS
- Database seeded on the server; accounts confirmed with a query
- Client files generated on the server and fetched with scp
- Full demo run against the domain
- Access restricted if the demo credentials are public
None of these failures was exotic. Each was an ordinary assumption that happened to be true on my laptop and false on the server. The value of writing them down is that next time, they are checks instead of surprises.
Comments