The AI Checked Its Math. Against the Wrong Source.

The AI Checked Its Math. Against the Wrong Source.

Seventeen articles for a numerology site, written in parallel by AI writers, each one full of worked examples. "Born April 9, 1980: 9 + 4 + 9 = 22." That kind of thing, dozens of times per article.

One wrong sum on a numerology site and the whole blog is a joke. So I asked every writer to verify its math before handing the article in. Every writer said it had.

They had. Against the wrong thing.

What "verified" meant

Each writer had checked its examples against its own implementation of the rules as I had described them in the brief. That is a real check, and it caught genuine arithmetic slips. But it means the writer was verifying the article against the brief, and the brief against itself.

If the brief had a mistake in it, every article would have reproduced that mistake faithfully, and every check would have passed.

The thing the articles actually needed to agree with was not the brief. It was the site's real calculator, the JavaScript file that runs when a visitor types in their birthday. If an article and the calculator disagree, the reader notices, and the article is wrong by definition.

Check against the code that ships

The fix took about an hour. A small harness loads the site's real numerology.js into a Node sandbox and runs every example from every article through it. Not a reimplementation. The actual file, byte for byte, that the site serves.

Then, because it was cheap, I brute-forced every date for a century through the same engine and looked at what came out.

That second step found things no single example could have.

A Life Path of 33 is impossible with the method this site uses. The parts top out at 9 + 9 + 9 = 27, so 33 can never be reached. Yet an already-published guide on the site listed 33 as a Life Path in its summary and gave it a whole section. Nobody had ever asked the calculator whether 33 could happen. The brief said master numbers exist, the writer wrote about master numbers, and the check confirmed the writer had described master numbers correctly.

And in 2026, nobody can have a Personal Year of 2 or 22. The year's digits make that arithmetically impossible with this method. The brief had never considered it, so neither had anyone else.

The compatibility claims got the same treatment, checked against the actual pairing matrix the paid report uses. No free article can now contradict the thing we charge for.

Examples prove what is reachable. Only exhaustion proves what is not.

This is the part I keep coming back to. A worked example is proof that one value can occur. Twenty worked examples are proof that twenty values can occur. None of them, however carefully checked, can tell you which values cannot occur.

And "cannot occur" is exactly where confident prose goes wrong. A writer who has been told master numbers exist will write a section on 33. Nothing in the examples will contradict them. The only thing that can is running the whole input space once and looking at the set of outputs.

For most product rules that run is tiny. Every date in a hundred years is about thirty-six thousand inputs. It took seconds.

The habit

When generated content makes checkable claims about your own product, verify it against the code that ships. Not a spec, not a restatement of the rules, not the writer's own understanding. Load the real thing and run the claims through it.

Then brute-force the inputs once, and read the list of outputs that never appear. That list is the set of things your content must not promise. It is also the list nobody will think to check by hand.