Published

A 200 Is Not Proof: verifying a deploy you cannot look at

Ejentic Research Desk

20 October 2026

Abstract

A deploy that returns HTTP 200 has told you nothing. Three incidents in one project, each passing every status check while shipping the wrong thing.

Three failures that all returned 200

  1. A build failed after the database sync step. The deploy script aborted, the previous build kept serving, and the homepage was still 200.
  2. A feature flag was absent from the server environment, so the code fell through to a default that silently disabled the feature. The site was 200 and the feature was dead.
  3. A deploy printed no output and changed nothing. The build ID was identical before and after. Everything still responded 200.

What actually proves a deploy

  • A changed build identifier. If it did not change, nothing shipped.
  • The served bundle, not the source. Grep the deployed output for a string only the new code contains.
  • A behavioural probe. An input the old build answered one way and the new build answers differently. An empty request that used to return 200 and now returns 400 is a good example, because it is cheap and unambiguous.
  • A negative check. Confirm the thing you removed is genuinely absent from the bundle.

The general form

Pick, before you deploy, the one observable difference this change makes. If you cannot, you are not verifying a deploy, you are hoping.

BibTeX

@article{desk2026not,
  author    = {Ejentic Research Desk},
  title     = {A 200 Is Not Proof: verifying a deploy you cannot look at},
  year      = {2026}
}

2026

Deleting the ROI Calculator

A plausible number is worse than no number. A visitor cannot tell the difference, which is the problem, not the defence.

2026

SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools

Read this before you let an agent call anything a customer will see the output of. The failure mode is not a crash, it is an API returning 200 with a body that means "I did not understand you". There is no exception to catch and no status code to branch on. The result we quote most often is the last one: the fix was one line of schema, not a better model.

2026

BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms

Every RAG proposal we are handed assumes dense retrieval is the default and everything else is a downgrade. This is the study that says it depends on corpus size, and that above roughly 10 million tokens a lexical index built for nothing beats an agent that reasons its way through the corpus. It also costs 39 times fewer query tokens at the point where the agent is still winning.