Work · Product · 03
Werkbank
Which workshop should take this furniture order?
In 30 seconds
- Problem
- A furniture shop with three workshops routes every incoming order by hand, and misrouted jobs cost days.
- Result
- On 60 hand-written requests it routed 40% on its own at 95.8% accuracy and sent 23.3% to a review queue instead of guessing. Real-shop accuracy: pending.
- My role
- Sole developer: classifier, calibrated gate, review queue, app, auth and eval.
- measured 95.8% accuracy on the 40% of hand-written requests it routed on its own
- measured 23.3% of hand-written requests sent to the review queue instead of guessed
- synthetic 96.8% top-1 accuracy after a new shop corrects 100 orders (from 75.5%)
- pending — accuracy on real shop orders: no shop has used it yet
- Type
- Product
- When
- October 2026
- Role
- Sole developer: classifier, gate, app, auth, eval
- Origin
- A real client brief from a furniture shop with three workshops
- Status
- Live demo · EN and DE · 57 tests
01In plain language
A furniture shop with its own workshops gets requests like "sofa cushion torn, needs new foam" or "Tisch wackelt, Bein lose". Someone has to decide which bench each job goes to: carpentry, upholstery, polish and paint, or metalwork.
Werkbank reads the request the way the owner would write it on a job slip and sends it to the right workshop. It also labels the job type, urgency and effort. When it is not sure, it does not guess: the order waits in a review queue. Every correction the owner makes teaches that shop's model.
It started as a real brief from a furniture shop with three in-house workshops. I turned it into a product any shop can sign up for, in English and German.
02The question
How many orders can go straight to the right workshop without a person, and can the system tell which ones those are?
03Why the obvious approach fails
Keyword rules are easy to explain to a shop owner, but customers do not use the shop's words. "Recliner mechanism stuck" is a metalwork job with no metal word in it.
A classifier that always answers sends every unclear order somewhere. A wrong routing is worse than a slow one: the job sits at the wrong bench until someone notices.
An LLM on every order costs money per order, needs an API key in every shop, and still answers confidently when the request is vague.
04The gate
Two cheap layers run in under a millisecond: a keyword lexicon in English and German, and a naive Bayes model trained on seed data plus the shop's own corrections. Their blended score goes through a gate.
| Outcome | Condition (balanced setting) |
|---|---|
| Auto-route | confidence ≥ 0.85 and margin ≥ 0.5 |
| Route, flagged "check" | confidence ≥ 0.55 |
| Review queue | Lower confidence, almost no known words, the two layers strongly disagree, or the shop has no workshop for the predicted job |
Naive Bayes is known to be over-confident, so its scores are tempered by a temperature fitted on held-out phrasings (2.5). The interface never shows more than "99%+". Shops can move the gate between strict, balanced and relaxed.
An optional language model only sees orders that are already in review. It must quote the request word for word, or its answer is thrown away. An LLM decision is never marked as automatic.
05Thought process
Left to right, the path one order takes, from the owner's typing to the shop's next model.
- 01
Read
- Free text, English or German
- Live preview while typing
- Due date taken into account
The owner writes the way they talk.
- 02
Score
- Keyword lexicon, EN + DE
- Naive Bayes: seed + corrections
- Temperature 2.5 calibration
Two cheap opinions, in under a millisecond.
- 03
Gate
- Auto ≥ 0.85, margin ≥ 0.5
- Check ≥ 0.55
- Disagreement → review
Sure, fairly sure, or ask.
- 04
Review
- Best guess shown to the owner
- Optional LLM, quote required
- Never marked automatic
Unsure orders wait for a person.
- 05
Learn
- Every reassignment adds counts
- No training job needed
- Per-shop model
Corrections make tomorrow easier.
06Decisions, and what they cost
One correction just adds counts. The shop's model updates instantly, with no training job.
Every correction would wait for a training run before the shop saw any benefit.
Thresholds were set before looking at test results.
The search on validation data collapsed to the minimum threshold, so it was dropped and the thresholds were fixed by hand.
They depend on a few explicit words and the due date. "No rush" is checked before "rush".
A shop owner can read a rule and see why a job was marked urgent.
The synthetic test set uses sentence templates and synonyms that never appear in training, plus 60 messy requests written by hand.
That would test whether it remembers its own templates.
07What came out
| Test set | Auto-routed | Auto accuracy | To review | Accuracy of all routed |
|---|---|---|---|---|
| Hand-written, messy (n = 60) | 40.0% | 95.8% | 23.3% | 93.5% |
| Synthetic, unseen phrasing (n = 800) | 18.9% | 95.4% | 36.1% | 84.0% |
The gate holds: whatever skips review is right about 95% of the time on both sets. What changes is how much it dares to skip. On phrasing it has never seen, it sends more to review instead of guessing.
Strict or relaxed: the trade-off a shop chooses
| Auto threshold | Auto-routed | Auto accuracy |
|---|---|---|
| 0.50 | 71.8% | 82.8% |
| 0.70 | 43.6% | 86.5% |
| 0.85 (default) | 18.9% | 95.4% |
| 0.95 | 4.4% | 100.0% |
The feedback loop
A simulated shop that phrases things in ways the seed model never saw corrects its first orders. Then the model is measured on 400 later orders:
| Corrections | Top-1 accuracy | Auto-routed | Review queue |
|---|---|---|---|
| 0 | 75.5% | 20.8% | 34.8% |
| 50 | 92.0% | 63.7% | 12.5% |
| 100 | 96.8% | 82.5% | 6.3% |
| 200 | 98.5% | 87.5% | 4.0% |
All from reports/eval.md, generated 26 Oct 2026. Seed model trained on 2,400 synthetic requests.
08What it refuses to claim
Real-world accuracy. There is no public dataset of furniture-shop tickets, so training data is synthetic, and synthetic test data is easier than real orders. The true number is unknown until a shop uses it.
The 400-correction result. It reaches 100%, but by then the test orders are close to what it trained on. The evaluation report says not to quote it, so it is not quoted here.
09What broke
Three hand-written orders went to the wrong bench
| Request | Right | Routed to |
|---|---|---|
| "need a study table built into the corner, 120x60, white laminate top" | Carpentry | Finishing (auto, 0.93) |
| "old teak chest, grandma's, wants it restored. finish is black and sticky" | Finishing | Carpentry (check, 0.74) |
| "recliner mechanism stuck, footrest won't come out" | Metalwork | Upholstery (check, 0.66) |
Only the first one skipped review entirely. "White laminate top" pulled it towards finishing. The other two were routed but flagged for the owner to check.
German is the weaker language
The keyword lexicon is mostly English. In a small manual check, 3 of 5 German example sentences went to the review queue. They were not routed wrongly, but the shop gets less automation in German.
Job type is mediocre
Repair, custom, refurbish or assembly is right only 70% of the time on the synthetic set (75% hand-written). It is shown as a label, not used for routing.
10Still open
- A real shop using it, and accuracy measured on real orders.
- A bigger German lexicon.
- Login rate limiting, and ending open sessions when a password is reset.
- Payments: picking a plan records it, but billing is not connected.
11Reproduce it
Every number on this page comes from these commands, run in the repository.
git clone https://github.com/omkar-k0/werkbank && cd werkbank
npm install
npm test # classifier, tenant isolation, split orders, auth, i18n
npm run model # regenerate synthetic data, retrain, re-evaluate
npm run dev # http://localhost:3000, Postgres in-process via PGlite | File | What it holds |
|---|---|
| reports/eval.md | Accuracy, routed share and review share per test set |
| reports/eval.json | The same numbers, machine-readable |
| scripts/eval.ts | The script that writes both reports |
All training data is synthetic. The 60 hand-written requests were not used for training or thresholds.