Published

Deleting the ROI Calculator

Ejentic Research Desk

6 October 2026

Abstract

We removed a working feature because it hardcoded a 75% improvement rate that nobody had measured. It converted well. That is exactly why it had to go.

The feature

An interactive calculator that estimated a client's return from AI adoption. It was one of the better-converting elements on the site.

It also contained a fixed 75% improvement rate, a fabricated baseline, and a third invented figure on the case studies page. None of them came from anything. They were chosen because they looked reasonable.

Why delete rather than soften

A disclaimer would leave the numbers doing their work while moving the responsibility to the reader. A range would be the same thing with extra steps. The only honest versions are a real measurement or nothing.

We had no measurement. So: nothing.

The standard we adopted

Only three numbers are permitted on the site: the ones we can actually count. Everything else is derived from them or omitted. The rule is in the repo guidance and it is enforced by review, not by tooling.

What it cost

Some conversion. We do not know how much, because we did not keep the feature long enough to measure it cleanly, and we consider that an acceptable price for not putting invented figures in front of prospects who might later audit them.

BibTeX

@article{desk2026deleting,
  author    = {Ejentic Research Desk},
  title     = {Deleting the ROI Calculator},
  year      = {2026}
}

2026

Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead

The desk already carries SWE-bench from 2023, and this is the paper you should read immediately after it. A benchmark that can no longer separate its leaders has not stopped being useful, it has changed jobs: it is a regression floor now, not a ranking. The 29.8-point within-model scaffold spread is the number to remember, because it is larger than the whole spread across the top thirty entries. The harness you wrap around a model moves the score more than the model does.

2026

SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools

Read this before you let an agent call anything a customer will see the output of. The failure mode is not a crash, it is an API returning 200 with a body that means "I did not understand you". There is no exception to catch and no status code to branch on. The result we quote most often is the last one: the fix was one line of schema, not a better model.

2026

BM25 Wins at Scale: A Scaling Study of Retrieval-Augmented Generation Paradigms

Every RAG proposal we are handed assumes dense retrieval is the default and everything else is a downgrade. This is the study that says it depends on corpus size, and that above roughly 10 million tokens a lexical index built for nothing beats an agent that reasons its way through the corpus. It also costs 39 times fewer query tokens at the point where the agent is still winning.