Entry 189-1 Build in Public 2 min ↩ back to the timeline

The examples outvoted the instructions

Severo's oldest bug finally falls: the prompt said 'the user is learning English' but the few-shot examples answered in English, and the model believed the examples, flipping native and target for exactly the Spanish-to-English learners the app was born for. The fix: example banks per native language, verified with automated tests.

video fuente → Source video thumbnail
Source transmission · “0 to 1 Million” diary

// trace: where this idea came from

Severo isn’t forgotten, “no he olvidado Severito,” and today its most stubborn bug gets diagnosed and fixed, with automated tests to prove it ▸ 6:14. The archaeology explains everything: he wrote the original prompt for himself, an English speaker practicing Chinese, so every few-shot example answered in English, “you’re right, you only had this mistake…” ▸ 9:06. For any pairing routed through English that worked fine. But for the app’s core user, a Spanish speaker learning English, the model faced a contradiction: the instruction said English is the target, while the example answers were in English, and it resolved it by trusting the examples, “si le tengo que responder en inglés es porque la persona está estudiando español”, silently swapping native and target ▸ 9:52, “volteaba el nativo con el target” ▸ 10:30.

en un prompt, los ejemplos votan más fuerte que las instrucciones →

The fix respects the diagnosis: don’t argue with the model, change the evidence. The prompt now assembles itself by the user’s native language, a Spanish speaker gets example feedback written in Spanish, a Portuguese speaker in Portuguese, generated banks for the eight or nine main languages ▸ 10:41. Julia spots the industry rhyme immediately: Duolingo’s giant catalog is the from-English one, and every other native language gets the partial shelf ▸ 7:18, the English tax reproduced inside his own prompt by accident. The same session localizes the interface, where the assistant translated the eight main languages and then quietly wrote a script that filled every remaining language with English ▸ 11:35, laziness he caught by reading the diff. The lesson generalizes past Severo: in a prompt, examples are load-bearing, and when they disagree with the instructions, the examples win…

// continued in

no entry has continued this idea yet: the arc is still open

la lista es el activo →

Continues

One distilled teaching per entry, straight from the video diary. Leave your email and get the next one the day it lands.