The generalist beat the specialists
The investor complained that Severo's speaking check hallucinates, you can say anything and it marks you right. To fix pronunciation scoring, Juan benchmarked phoneme models that turn audio into IPA symbols. Allosaurus, a well-known one, was about 40% wrong; Allophant, newer, got to roughly 25-30%, better but still too high, and heavy. Then he tried just handing the audio to Gemini with the sole instruction to extract the phonemes: 15% error, beating both specialized models built years ago for exactly that task. That's the headline, a general-purpose model beat two purpose-built ones, the retort to 'AI is useless'. Two tails to the story: Gemini 3.1 Flashlight (15% error) versus the 2.5 they were on (80%), so the model upgrade alone was decisive; and a prompt-engineering trick, asking Gemini for the transcript and the phonetics at once yields a more detailed transcript, as if the extra task's pressure sharpens the main one. The shipped fix feeds Gemini the audio plus a transcript so it stops hallucinating the answer, the 'elephant' when the exercise was about the cat.
// trace: where this idea came from
- ↳ video diary @ 26:29 (el inversor, puedo decir cualquier cosa y me lo marca bien, alucina)
- ↳ video diary @ 32:29 (darle el audio a Gemini para fonemas, 15% de error, mejor que los modelos específicos)
- ↳ video diary @ 36:19 (pedirle transcript y fonética a la vez da un transcript más detallado)
- ↳ Entry 254-2: The best of both worlds (la misma colaboración con el modelo, anclarlo para que no alucine)
The investor pointed at a real flaw: Severo’s speaking check hallucinates, you can say anything and it marks you correct, or invent that you gave the right answer when you gave half of it ▸ 26:29. To fix pronunciation scoring, Juan benchmarked phoneme models, the kind that turn audio into IPA symbols. Allosaurus, a well-known one, came back about 40% wrong; Allophant, newer, reached roughly 25-30%, better but still too high, and heavy to run. Then he tried the obvious thing nobody expects to work: hand the audio to Gemini with the sole instruction to extract the phonemes. 15% error, beating both specialized models that were built years ago for exactly that task ▸ 32:29.
un modelo general le ganó a dos especialistas hechos para la tarea →
That’s the headline, and the retort to everyone saying AI is useless: a general-purpose model beat two purpose-built ones, the work of who knows how many people, now passed by ▸ 33:04. Two tails hang off it. The model matters, Gemini 3.1 Flashlight scored that 15% where the 2.5 they’d been running hallucinated at 80%, so the upgrade alone was decisive. And a prompt trick worth keeping: asking Gemini for the transcript and the phonetics at once produced a more detailed transcript than asking for either alone, as if the pressure of the extra task sharpens the main one ▸ 36:19. The version that shipped is the same anchoring move: feed Gemini the audio plus a transcript as a base, so it stops hallucinating, no more marking “the elephant” correct when the exercise was about the cat…
// continued in
no entry has continued this idea yet: the arc is still open