One word, two models
| Model | Transcript |
|---|---|
| Stock Parakeet v2 | I want to go to the Micollette today to buy my groceries. |
| This fine-tune | I want to go to the makolet today to buy my groceries. |
Recorded in the demo Space, 2026-10-15. Makolet (מכולת) is the Israeli corner grocery shop. Stock v2 has never heard it, so it guesses an English-looking name.
What it is
Stock Parakeet v2 is excellent at English, but it has never seen romanized Hebrew. A Hebrew word in English speech comes out as whatever English words sound closest. Its tokenizer has no Hebrew characters, so the fine-tune teaches it one romanized spelling per Hebrew word, from read-aloud sentences recorded on the phone that runs the model.
- Single-speaker proof of concept. All training and test audio is one speaker: Irish-accented English, living in Jerusalem. Gains on other voices are untested.
- Output is English with romanized Hebrew, never Hebrew script. Transcripts are cased and punctuated, like v2.
- Built on v2, not v3, deliberately: in the speaker's own use, v2 was more accurate for English dictation.
- About 70% code-switching, 30% English in training, so the model keeps its English.
Project links
Which file do I want?
| You want to… | Get | Run it with |
|---|---|---|
| Dictate on an Android phone | GGUF Q8_0 (730 MB) | Offline Voice Input |
| Android with ≤ 4 GB RAM | GGUF Q4_K_M (475 MB) | Offline Voice Input |
| Dictate on a Linux / macOS / Windows desktop | GGUF Q8_0 | Handy |
| Desktop dictation with sherpa-onnx apps | sherpa-onnx int8 | Voxtype, any app that loads sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8 |
| Python on CPU, onnx-asr | ONNX (int8 or fp32) | onnx-asr |
| GPU inference, or fine-tune it further | .nemo (2.47 GB) | NVIDIA NeMo |
If unsure, take Q8_0. On five held-out clips it matched the full-precision model word for word; Q4_K_M differed on one word. On an 8-core desktop CPU, Q8_0 was also faster (about 3× realtime against 1.4×).
Formats
GGUF, ONNX, sherpa-onnx and .nemo are model v1.0 (run r6-en30, 2026-10-19), the first release after the proof-of-concept checkpoints PoC-1 to PoC-5 (published as v1 to v5); MLX is still PoC-3. Grouped in the Hugging Face collection.
Versions
Download links above always point to the latest version. These links are pinned to each version's commit and never change.
Project test set
219 held-out code-switching clips, term-disjoint: the Hebrew words in them never appear in training, so these numbers measure generalisation, not memorised sentences. Every version is re-scored on the same test revision (b47310621f79), so the bars compare like with like. WER is lowercase with punctuation removed.
Word error rate by test set
Lower is better. The last two rows are the forgetting check: English must not get worse.
Show as table
Hebrew terms transcribed correctly
Higher is better. Share of term occurrences in test sentences with the canonical spelling (176 unseen, 153 seen).
Show as table
Charts show checkpoints up to PoC-3. PoC-4, PoC-5 (2026-10-18) and v1.0 (2026-10-19) have not been benchmarked yet.
Reading it
- Code-switching WER halved (10.9% → 5.3% with v3) and long-form dictation improved by almost as much (9.0% → 5.0%).
- Hebrew words the model never heard in training went from 27% to about 48% correct. It learned how this speaker's Hebrew maps to the romanization, not only the training words.
- Each version helped words it has recordings of (57% → 60% → 66% from PoC-1 to PoC-3). Unseen words did not move after PoC-1 (47%, 47%, 48% of 176): more takes per word help, more words alone do not.
- No measurable forgetting. Plain English and LibriSpeech stay at or below stock v2.
Public benchmark
A separate, frozen test set scores this model against stock Parakeet and Whisper with the same normaliser and term matcher: leaderboard, test set, runner. Clean subset: 84 clips, 172 Hebrew term occurrences.
Hebrew term recall by model
Higher is better. This project's models in blue. One speaker reading aloud: treat gaps of a few points as noise.
Show as table
⚠ Whisper-Hebrish has known overlap with the test data. Benchmark v1 also labels 22 English tech words as terms; ranks are unaffected, absolute recall runs 2–6 points high.
Where stock v2 starts
Stock v2 already gets 258 of 1,139 Hebrew terms in the training recordings right every time: names, well-known places and words that are already English (Shabbat, Pesach, Bamba). It never gets 706 right: challah, cheshbonit, sabich, Motzei Shabbat, osek patur, tachana merkazit. (Term map from 2026-10-18, dataset revision b47310621f79.) Recording now concentrates on that second group.