A green build that deployed nothing
I spent an afternoon building a CI pipeline for this site, watched it go green, loaded the page, and called it done. It wasn't. The job had been succeeding and deploying nothing for hours, and everything I looked at told me it was working.
The setup
The site is built by Eleventy and served from a small nginx container on a home server. A
self-hosted Gitea runner picks up jobs on push, runs them in a throwaway node:lts container,
builds the site, and copies the output to the web root. The web root is a host directory
bind-mounted into the job container as /deploy.
The last step of the job is deliberately loud:
rm -rf /deploy/*
cp -r dist/. /deploy/
echo "--- deployed files ---"
ls -la /deploy
Push, watch it go green, ls shows the files. Site loads. Done.
Except the site never changed
The tell was a marker line I'd added to the page. It was in my source, it was in the job's
ls output, and it was not in the HTML the server returned. I'd been verifying the deploy by
loading the page in a browser and seeing a page, not by checking whether the bytes had
actually changed.
The file on disk was hours old. Nothing had written to it since I created it by hand.
The cause
The runner has a config option called valid_volumes: an allowlist of host paths that jobs
are permitted to bind-mount. Mine was set to the default:
valid_volumes: []
Empty list, so no bind mounts allowed. The -v flag mounting the web root into the job was
being silently dropped. The job container started without it, cp created an ordinary
directory called /deploy inside the container, copied the files there, and ls listed them
exactly as expected. Then the container was destroyed, taking the deploy with it.
Every command succeeded. Every step exited 0. The output looked perfect. It just had no effect on anything outside the container.
Why it was hard to see
The control was doing its job. valid_volumes exists because a runner in Docker mode can
otherwise mount anything on the host: /etc, the Docker socket, someone's SSH keys.
So it defaults to deny-all and makes you declare what's allowed. I'd hardened the runner
earlier and never completed the authorization half.
The failure mode of a good security control is often exactly this: the operation appears to succeed while quietly having no effect. That's usually what you want. It's also why the logs weren't helpful. Nothing failed, so nothing was logged as a failure.
The fix
List the one path explicitly:
valid_volumes:
- /home/user/containers/mabel-staging/site
options: "-v /home/user/containers/mabel-staging/site:/deploy"
Plenty of forum answers suggest valid_volumes: ['**']. Don't. That allows mounting any host
path and throws away the entire protection. Which, given the runner already has the Docker
socket, means any job can do anything to the host.
One more trap: the runner reads its config at startup. docker compose up -d did nothing,
because the compose file hadn't changed, only the config file it mounts had. It needs
docker compose restart.
What I actually changed about how I work
A green run means the commands succeeded. It does not mean they had the effect you wanted. Those are different claims, and CI only ever verifies the first one.
Now I verify at the destination:
- file timestamp and size on the target filesystem
- the served bytes, fetched in a way that can't hit a browser cache
- a unique marker in the content, so "it looks right" isn't the test