Four times I was wrong about what I was looking at

July 23, 2026

Debugging is usually described as finding the fault in the system. In one day of building out this site's infrastructure, four separate problems turned out to have nothing wrong with the system at all. The fault was in what I believed I was seeing.

They are worth writing down together, because individually each one looks like bad luck and collectively they look like a habit.

One: the feature that was never off

I opened the Actions tab on my self-hosted Git server and got "nothing can be found here." No runs, no empty list, just missing.

That reads like a disabled feature, so I went looking for why it was disabled. Repository settings, no toggle. Server config file, no [actions] section at all. Container environment variables, nothing overriding it. Version check, new enough that it should be on by default.

Three layers of configuration, all clean, all consistent with the feature being enabled. Which it was. I looked again and there was a workflow run from thirty-three minutes earlier.

The tab had been fine the whole time. Something about that first page load was stale or wrong, and I spent an hour proving the server was correctly configured instead of reloading the page.

The tell was available early: when the config trail comes up clean at every level, the thing to re-check is the observation.

Two: the revert with nothing to revert

I had pushed a test marker into a file and wanted it gone. So:

git add README.md
git commit -m "Revert test marker"

nothing to commit, working tree clean

My first read was that something was broken about staging or the file. It was neither. "Working tree clean" means the file on disk is identical to the last commit, and it was, because I had already committed the marker. There was nothing to revert because the change was no longer pending. It was history.

Git was reporting the situation accurately. I was asking it to undo something and it was telling me there was nothing in the place where undoable things live. To remove the text I had to make a new edit, not reverse a pending one.

Three: the deploy that had already happened

I pushed a change, both pipelines went green, and staging showed the old page. Given that I had spent the morning finding a bug where a green build genuinely deployed nothing, I assumed I had found another one.

Checking the destination took under a minute:

ls -la ~/containers/mabel-staging/site/

Fresh timestamp. Correct file size. New subdirectory present. Then fetching the served bytes directly from inside the network, bypassing anything local, returned the new content.

The deploy was fine. My browser was serving from its own disk cache, which survives a browser restart, so closing and reopening had not cleared it.

The underlying reason is worth knowing. The origin server sent Last-Modified and ETag but no Cache-Control. With no explicit freshness instruction, browsers fall back to a heuristic, commonly around ten percent of the file's age. So an older file is assumed fresh for longer. The browser served the stale copy immediately, revalidated in the background, and swapped in the new one a moment later. Exactly to spec, and exactly the wrong behaviour for a staging site.

I fixed the environment rather than my habits, by having the proxy send Cache-Control: no-cache. Which, despite the name, does not mean "do not store." It means "revalidate before using," so cheap 304 responses still work and I never see stale content. "Do not store at all" is no-store, which I did not want.

Four: the flag that hid the error

Testing that header, I ran:

curl -skI https://staging.mabel.ink | grep -i cache

Nothing came back. Reasonable conclusion: the header is not being sent, so the config did not take. I went and checked the config, which was correct, and the proxy logs, which were clean.

Then I dropped the -s:

curl: (6) Could not resolve host: staging.mabel.ink

The hostname only resolves through a hosts file entry on my desktop. The server itself has no such entry, and every previous test from that machine had used the container name instead, so this had never come up. The request had failed before it started.

-s is silent mode. It suppressed the error, and the empty output looked like a negative result rather than a failed request. Those are very different things and they render identically.

The pattern

In all four cases the system was behaving correctly and my model of what I was observing was wrong. The corrections were cheap in every case. A page reload. Reading the git message literally. Checking the file on disk. Removing one flag.

What I do differently now:

• Verify the observation before the configuration. If two or three layers of config all look correct, the odds favour the observation being wrong rather than a fourth hidden layer.
• Check at the destination. Timestamp, size, and served bytes. Not the pipeline's status, and not a page that a cache can answer for.
• Do not suppress errors while diagnosing. Quiet flags are for scripts you already trust. During diagnosis they turn failures into ambiguous silence.

One thing I do not want to over-correct on. The same instinct that made me distrust a green build in case three is the instinct that found a real bug earlier that day, where the job was succeeding and writing nothing to the host. Suspicion of a green checkmark is a good habit. The refinement is not to trust things more. It is to check the thing that actually settles the question, which is usually cheaper than the investigation you were about to start.