Work · Product · 01

Guidance Reconciliation Engine

Did the company hit the revenue guidance it gave last quarter?

In 30 seconds

Problem
Companies give revenue guidance every quarter. Checking it against what they later reported means reading hundreds of filings by hand.
Result
188 guidance items from 185 SEC releases reconciled with citations; only 7 (3.7%) needed the LLM. Accuracy against hand labels: pending.
My role
Sole developer: SEC data pipeline, rules, confidence gate, Claude layer and the live site.
  • measured 185 earnings releases read, 11 companies, Jan 2022 to Oct 2026
  • measured 7 of 188 items sent to the LLM (3.7%); the rest decided by rules
  • measured 23 unit tests on synthetic inputs
  • pending — accuracy against hand labels: labelling not done yet
Type
Product
When
October 2026
Role
Sole developer: data, rules, gate, LLM layer, site
Data
SEC 8-K earnings releases and XBRL company facts
Status
Live · accuracy pending

01In plain language

Every quarter, large companies say how much revenue they expect next quarter. Three months later they report what actually happened. This engine reads both from public SEC filings and labels each forecast MET, MISSED, RESTATED or UNVERIFIABLE.

Simple rules handle the clear cases. A language model only looks at the few cases the rules are unsure about, and its answer counts only if it quotes the filing word for word. When the evidence is not there, the engine says so instead of guessing.

02The question

Can a system decide most of these cases with cheap, transparent rules, hand only the unclear rest to a model, and still back every single label with a quote someone can check?

03Why the obvious approach fails

There are two obvious ways to build this. Both fail in ways that are easy to miss.

A script with regular expressions is cheap and fast. It also gives the same flat answer for a clean sentence and for a messy one, with no sign of which is which. It will happily compare a full-year forecast against a single quarter and report a miss.

An LLM on every filing reads messy text well. But it always returns an answer, written just as confidently when the guided period and the reported period do not match. And it costs a model call for every one of the 188 items, including the ones a rule gets right for free.

The filings themselves are less tidy than they look. NVIDIA guides "$8.10 billion, plus or minus 2%". Texas Instruments gives a range. Broadcom gives an approximate figure. Fiscal years end in different months. Many companies never report a fourth quarter on its own, only the full year.

04The gate

The rule layer does not just give a label. It scores how sure it is and writes down every reason it lost confidence. That score decides who answers.

Confidence gate routing 188 items enter the rule layer. 181 clear the 0.75 confidence threshold and are decided by rules: 167 met, 3 missed, 11 unverifiable. 7 fall below and go to the language model, which decides 6 as met, each with an exact quote, and declines 1 as unverifiable. INPUT 188 items RULE LAYER parses range, matches quarter, scores itself conf = min(parse, match) ≥ 0.75 ? yes · 181 DECIDED BY RULES 167 MET · 3 MISSED 11 UNVERIFIABLE no model call no · 7 LLM + QUOTE CHECK 6 MET, each with an exact quote 1 declined → UNVERIFIABLE
Run of 24 Oct 2026. The threshold is the default (0.75) until it can be calibrated on hand labels.

What lowers the rule layer's confidence

SignalDeductionWhy
No forward-looking word near the figure−0.30The number might be a past result, not a forecast
Adjusted basis (currency, exclusions)−0.25May not compare to reported total revenue
Reported figure within 0.5% of the low end−0.20MET versus MISSED depends on rounding
Period not named−0.15"Next quarter" has to be assumed
Point estimate ("approximately $X")−0.15No range given; met from 99% of X
Range written in reverse−0.10Unusual format, worth a second look

From extract.py and rules.py. Final confidence is the lower of parsing confidence and matching confidence.

In this run, 6 of the 7 escalations were AMD releases. AMD writes its outlook in a way the rule layer does not recognise as forward-looking, so each one lost 0.30 and went to the model. That is the gate doing its job: the unusual format got a closer read instead of a silent guess.

05Thought process

Left to right, the path one guidance item takes. The model only appears in the last card, and only for items the rules could not settle.

  1. 01

    Fetch

    • 8-K earnings releases, exhibit 99.1
    • XBRL company facts
    • SEC fair-access rules

    Public filings, nothing scraped from news.

  2. 02

    Extract

    • Outlook blocks, line by line
    • Total revenue only, segments skipped
    • Range, midpoint ± or point estimate

    Every lost point of confidence is written down.

  3. 03

    Match

    • First quarter ending after the filing
    • Five XBRL revenue tags merged
    • Q4 = full year − nine months

    Dates, never fiscal labels.

  4. 04

    Decide

    • MET if reported ≥ low end
    • Restatements checked
    • Confidence = min(parse, match)

    Rules settle 181 of 188.

  5. 05

    Adjudicate

    • Below 0.75 only
    • Typed tool call, four labels
    • Quote must be verbatim

    No quote, no label.

The rule layer and the model answer the same question with the same four labels. The difference is who is allowed to answer, and that is decided by confidence.

Step by step

  1. Fetch. Download 8-K earnings releases (Item 2.02, exhibit 99.1) and XBRL company facts from SEC EDGAR, following SEC fair-access rules.
  2. Extract. Find outlook blocks line by line, skip segment rows and full-year guidance, and parse the total-revenue range.
  3. Match. Find the reported revenue for the first roughly three-month period ending after the filing date. Merge the five XBRL revenue tags companies switch between. Derive Q4 as full year minus nine months.
  4. Decide. Rules label the item and score themselves.
  5. Gate. At or above 0.75 the rule decision stands. Below it, Claude gets the filing text and must answer through a typed tool call.
  6. Check the quote. Every citation the model gives must appear verbatim in the source. With no valid quote from the guidance, the label becomes UNVERIFIABLE. Answers are cached on disk, so re-runs cost nothing.

06Decisions, and what they cost

ChosenRules first, LLM for the rest

181 of 188 items were decided without a model call. Only 7 needed one.

Instead ofAn LLM on every item

188 calls, and no record of which answers came from easy cases and which from hard ones.

ChosenNo quote, no label

A model answer only counts if it quotes the filing exactly. Anything else becomes UNVERIFIABLE.

Instead ofTrusting the model's own confidence

A confident paragraph is not evidence. A quote someone can find in the filing is.

ChosenMatch on period end dates

Reported period must end after the filing date, within 130 days.

Instead ofMatch on fiscal labels

NVIDIA's "first quarter of fiscal 2023" ended in May 2022. Labels across companies do not line up; dates do.

ChosenCount out-of-scope cases

Full-year, growth-percentage and segment guidance are skipped and counted. Workday was dropped: its guidance is subscription revenue only.

Instead ofStretch the rules to cover them

Each stretch adds a way to be quietly wrong. v1 stays narrow: quarterly total revenue.

ChosenCalibrate the threshold on labels

Temperature scaling plus the lowest threshold at which rule-decided items reach 95% precision. Built, waiting for labels.

Instead ofPick a threshold that looks right

0.75 is a placeholder until hand labels say otherwise.

07What came out

The live engine, filtered to items the LLM decided. One AMD quarter is labelled MET with its reasoning; the next is labelled UNVERIFIABLE because the period could not be matched. Each item links to the filing on sec.gov.
The live site, filtered to the 7 items the LLM decided. The second one is the gate working: the period did not match cleanly, so it says UNVERIFIABLE instead of guessing.

Of 174 quarters with a reported figure, the engine found 2 where revenue came in below the low end of guidance. Both are well-known misses: NVIDIA's second quarter of fiscal 2023 (guided $8.10B ± 2%, reported $6.70B) and Micron's fourth quarter of fiscal 2022 (guided $7.2B ± $0.4B, reported $6.64B).

Reported revenue against the low end of guidance Of 174 quarters, 2 came in below the low end of guidance. The median quarter came in 5.4% above it, and the tallest bar is 0 to 2% above. 0 10 20 30 40 -16% to -14%: 1 quarters -4% to -2%: 1 quarters 0% to 2%: 48 quarters 2% to 4%: 23 quarters 4% to 6%: 18 quarters 6% to 8%: 31 quarters 8% to 10%: 22 quarters 10% to 12%: 16 quarters 12% to 14%: 4 quarters 14% to 16%: 3 quarters 16% to 18%: 2 quarters 20% to 22%: 1 quarters 24% to 26%: 2 quarters 26% to 28%: 1 quarters 30% to 32%: 1 quarters low end of guidance median +5.4% -16%-8%0%+8%+16%+24%+32% 2 misses reported revenue vs low end of guided range
174 quarters with a reported figure. Excludes 3 spurious items and 11 quarters not yet reported. System output, not yet checked against hand labels.

Everything else landed at or above the low end. 48 quarters landed within 2% above it. For these companies, the low end of the guided range is rarely broken.

11 items are UNVERIFIABLE because their quarter had not been reported yet when the engine ran on 24 October 2026. One more is UNVERIFIABLE because the model declined it (see section 09).

08What it refuses to claim

Accuracy. The engine has not been scored against hand labels yet, so no accuracy number appears anywhere. The labelling sheet is blind: it hides the system's labels so they cannot bias the person checking.

A "beat rate" for these companies. The chart shows the engine's output. Until the labels exist, it is a description of what the engine decided, not a finding about the companies.

09What broke

Three items that should not exist

While writing this page I checked all 188 items by hand. Three come from releases that contain a second dollar figure next to the quarterly guidance. The extractor turned each into an extra item:

ReleaseWhat it readLabel givenRule conf.
Cisco, May 2024Full-year revenue guidance of $53.6–53.8B, compared against one quarterMISSED0.80
Broadcom, Sep 2024Full-year AI revenue of $12B, compared against total quarterly revenueMET0.80
Broadcom, Mar 2026A $10B share buyback, read as revenue guidanceMET0.85

All three cleared the gate. The confidence deductions only catch the problems they were written for, and "second figure in the same release" was not one of them. The fix is a structural rule, not another deduction: one quarterly total-revenue item per release. These are exactly the errors hand labelling is for, and why accuracy stays pending until it is done.

The model caught an extractor bug

For AMD's October 2024 release, the extractor attached the wrong period label to correct guidance. The model noticed that the label and the reported period did not agree and returned UNVERIFIABLE instead of MET. The rule layer had said MET. Refusing was the right call.

10Still open

  • Hand-label the items, run gre eval, and replace the pending accuracy with a measured one.
  • Enforce one quarterly item per release, and fix AMD's period label.
  • Split UNVERIFIABLE into "cannot tell" and "not reported yet".
  • RESTATED never fired in this run. The detection exists but has not been seen working on real data.
  • Calibrate the 0.75 threshold once labels exist.

11Reproduce it

Every number on this page comes from these commands, run in the repository.

git clone https://github.com/omkar-k0/guidance-reconciliation-engine && cd guidance-reconciliation-engine
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
cp .env.example .env      # SEC contact e-mail, Anthropic key
set -a && source .env && set +a
pytest                    # 23 unit tests on synthetic inputs
gre fetch                 # releases and XBRL facts from SEC
gre run --no-llm          # rules only, free
gre run                   # with Claude on the low-confidence rest
gre eval                  # needs data/labels/worksheet.csv filled in
FileWhat it holds
data/processed/results.jsonEvery reconciled item with its route, confidence and citation
data/labels/worksheet.csvBlind hand labels for the accuracy run (in progress)
tests/The 23 unit tests