README.md
15.8 KB · 372 lines · markdown Raw
1 ---
2 license: other
3 license_name: nvidia-open-model-license
4 license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
5 base_model: nvidia/parakeet-unified-en-0.6b
6 base_model_relation: quantized
7 library_name: transcribe.cpp
8 pipeline_tag: automatic-speech-recognition
9 language:
10 - en
11 tags:
12 - gguf
13 - transcribe.cpp
14 - asr
15 - speech-to-text
16 - parakeet
17 - conformer
18 - rnnt
19 transcribe_cpp:
20 schema_version: 2
21 wer_fleurs_en:
22 q8_0: 3.99
23 wer_librispeech_test_clean:
24 f32: 1.59
25 f16: 1.59
26 q8_0: 1.6
27 q6_k: 1.61
28 q5_k_m: 1.58
29 q4_k_m: 1.62
30 rtf_m4_max:
31 cpu: 37.32
32 metal: 208.06
33 rtf_ryzen_4750u:
34 cpu: 13.96
35 vulkan: 25.28
36 streaming: true
37 translate: false
38 lang_detect: false
39 timestamps: token
40 ---
41
42 # parakeet-unified-en-0.6b: transcribe.cpp GGUF
43
44 GGUF conversions of [nvidia/parakeet-unified-en-0.6b](https://huggingface.co/nvidia/parakeet-unified-en-0.6b) for use
45 with [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).
46
47 Ported from upstream commit
48 [d4ac992](https://huggingface.co/nvidia/parakeet-unified-en-0.6b/commit/d4ac992),
49 pinned 2026-05-10.
50 Validated against the NeMo reference at transcribe.cpp commit
51 [42528dd](https://github.com/handy-computer/transcribe.cpp/tree/42528dd)
52 on 2026-05-10.
53
54 English speech-to-text with punctuation and capitalization. A FastConformer encoder with an RNN-T transducer decoder, trained as a 'unified' streaming/offline model. This port runs the model in both offline and buffered streaming modes.
55
56
57 ## Downloads
58
59 | Quantization | Download | Size | WER (LibriSpeech test-clean, offline) |
60 | --- | --- | ---: | ---: |
61 | F32 | [parakeet-unified-en-0.6b-F32.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-F32.gguf) | 2.47 GB | 1.59% |
62 | F16 | [parakeet-unified-en-0.6b-F16.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-F16.gguf) | 1.24 GB | 1.59% |
63 | Q8_0 | [parakeet-unified-en-0.6b-Q8_0.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-Q8_0.gguf) | 731 MB | 1.60% |
64 | Q6_K | [parakeet-unified-en-0.6b-Q6_K.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-Q6_K.gguf) | 602 MB | 1.61% |
65 | Q5_K_M | [parakeet-unified-en-0.6b-Q5_K_M.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-Q5_K_M.gguf) | 541 MB | 1.58% |
66 | Q4_K_M | [parakeet-unified-en-0.6b-Q4_K_M.gguf](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/resolve/main/parakeet-unified-en-0.6b-Q4_K_M.gguf) | 477 MB | 1.62% |
67
68 WER on the full LibriSpeech test-clean split (2,620 utterances), batch size 1, timestamps none. Figures without a commit were published before provenance was recorded.
69
70 Greedy RNN-T decoding, no external LM. F32 reference baseline: 1.59%. NVIDIA's
71 self-reported number on the same split is 1.63%.
72
73
74 ## Usage
75
76 Build transcribe.cpp from source:
77
78 ```bash
79 git clone git@github.com:handy-computer/transcribe.cpp.git
80 cd transcribe.cpp
81 cmake -B build && cmake --build build
82 ```
83
84 Run on a 16 kHz mono WAV:
85
86 ```bash
87 build/bin/transcribe-cli \
88 -m parakeet-unified-en-0.6b-Q8_0.gguf \
89 input.wav
90 ```
91
92 If your audio isn't already 16 kHz mono WAV, convert it first:
93
94 ```bash
95 ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav
96 ```
97
98 See the [transcribe.cpp model page](https://github.com/handy-computer/transcribe.cpp/blob/main/docs/models/parakeet-unified-en-0.6b.md) for performance
99 numbers, numerical validation, and reproduction steps.
100
101 ## License
102
103 Inherited from the base model: **NVIDIA Open Model License**. See the
104 [upstream model card](https://huggingface.co/nvidia/parakeet-unified-en-0.6b) for full terms.
105 This repository includes the license text
106 ([`LICENSE`](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/blob/main/LICENSE)) and a
107 [`NOTICE`](https://huggingface.co/handy-computer/parakeet-unified-en-0.6b-gguf/blob/main/NOTICE) recording
108 how these files were derived from the original.
109
110 ---
111
112 ## Original Model Card
113
114 > The section below is reproduced from
115 > [nvidia/parakeet-unified-en-0.6b](https://huggingface.co/nvidia/parakeet-unified-en-0.6b) at commit
116 > `d4ac992` for offline reference. The upstream card is the
117 > authoritative source.
118
119 # 🦜Parakeet-unified-en-0.6b: Unified ASR model for offline and streaming inference
120
121 | [Model architecture](#model-architecture) | [Model size](#model-architecture) | [Language](#datasets) |
122 |---|---|---|
123
124 Parakeet-unified-en-0.6b is an English automatic speech recognition (ASR) model based on transducer architecture (RNN-T) combining both offline and streaming inference (with a minimum latency of 160ms) in one model [1]. It is trained mostly on the English part of the Granary dataset [4], which contains approximately 250,000 hours of US English (en-US) speech across diverse acoustic conditions. The model transcribes speech to English alphabet, spaces, and apostrophes with punctuation and captalization support.
125
126 <figure align="center">
127 <img src="figures/wer_comparison.png" width="1250" />
128 <figcaption>
129 Average WER comparison on the HF ASR Leaderboard datasets including offline and streaming inference with different latency values.
130 </figcaption>
131 </figure>
132
133 Why Choose nvidia/parakeet-unified-en-0.6b?
134
135 - **One model for both tasks:** You need to utilize only one unified model for both offline and streaming inference with a minimum latency of 160ms.
136 - **Better accuracy performance:** The unified model achieves better accuracy performance on the HF ASR Leaderboard datasets compared to the previous transducer-based offline and streaming only models.
137 - **Streaming chunk size flexibilty:** Enables you to choose the optimal streaming latency (chunk + right context) from 2080ms to 160ms with step of 80ms.
138 - **Punctuation & Capitalization:** Built-in support for punctuation and capitalization in output text
139
140 This model consists of a 🦜 Parakeet (FastConformer) encoder (jointly trained in offline and streaming modes) with an RNN-T decoder. It is designed for offline and streaming speech-to-text applications where latency can be as low as 160ms, such as voice assistants, live captioning, and conversational AI systems. The current inference pipeline supports only buffered streaming (left context is recomputed for each chunk) that can be longer than cache-aware streaming.
141
142 This model is ready for commercial/non-commercial use.
143
144 ## License/Terms of Use:
145
146 Governing Terms: Use of the model is governed by the [NVIDIA Open Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/).
147
148 ## Deployment Geography:
149
150 Global
151
152
153 ## Use Case:
154
155 This model is for transcription of English audio in offline and streaming modes.
156
157
158 ## Release Date:
159
160 - Hugging Face [04/07/2026] via [https://huggingface.co/nvidia/parakeet-unified-en-0.6b](https://huggingface.co/nvidia/parakeet-unified-en-0.6b)
161
162
163 ## Model Architecture
164
165 **Architecture Type:** Unified-FastConformer-RNNT
166
167 The unified model architecture is presented in [1]. The model is based on the FastConformer encoder architecture [2] with 24 encoder layers and an RNNT (Recurrent Neural Network Transducer) decoder. The model was trained jointly in offline and streaming modes. In the offline mode we used standard offline training with full-context self-attention and non-causal convolutions. In the streaming mode we applied chunked self-attention masks (incluing left, middle/chunk and right context) together with Dynamic Chunked Convolutions inside each FastConformer layer [3] to adapt the model to both decoding scenarios. We also introduced a novel mode-consistency regularization loss to further reduce the gap between offline and streaming performance. All the model parameters are shared between offline and streaming modes (encoder, predictor, and joint networks), including initial x8 subsampling with non-causal convolutions.
168
169 **Network Architecture:**
170
171 - Encoder: Unified FastConformer with 24 layers
172 - Decoder: RNNT (Recurrent Neural Network Transducer)
173 - Parameters: 600M
174
175 ## NVIDIA NeMo
176
177 ## How to Use this Model
178
179 For now, we provide only inference support for the unified model. We will release the unified training pipeline soon.
180
181 ### Loading the Model
182
183 ```python
184 import nemo.collections.asr as nemo_asr
185 asr_model = nemo_asr.models.ASRModel.from_pretrained(model_name="nvidia/parakeet-unified-en-0.6b")
186 ```
187
188 ### Offline Inference
189
190 ```python
191 output = asr_model.transcribe([wav_file_path])
192 print(output[0].text)
193 ```
194
195 ### Streaming Inference
196
197 For streaming inference you can use statfull chunked RNN-T decoding script from NeMo - [/NeMo/blob/main/examples/asr/asr_chunked_inference/rnnt/speech_to_text_streaming_infer_rnnt.py](https://github.com/NVIDIA-NeMo/NeMo/blob/main/examples/asr/asr_chunked_inference/rnnt/speech_to_text_streaming_infer_rnnt.py)
198
199 ```bash
200 cd NeMo
201 python examples/asr/asr_chunked_inference/rnnt/speech_to_text_streaming_infer_rnnt.py \
202 model_path=<model_path> \
203 dataset_manifest=<dataset_manifest> \
204 output_filename=<output_json_file> \
205 left_context_secs=<left_context_secs> \ # left context in seconds, 5.6s by default
206 chunk_secs=<chunk_secs> \ # chunk size in seconds, 0.56s by default
207 right_context_secs=<right_cintext_secs> \ # right context in seconds, 0.56s by default
208 att_context_size_as_chunk=true \ # set to true to use chunked self-attention masks
209 batch_size=<batch_size>
210 ```
211
212 You can also run streaming inference through the pipeline method, which uses [NeMo/examples/asr/conf/asr_streaming_inference/buffered_rnnt.yaml](https://github.com/NVIDIA-NeMo/NeMo/blob/main/examples/asr/conf/asr_streaming_inference/buffered_rnnt.yaml) configuration file to build end‑to‑end workflows with punctuation and capitalization (PnC), inverse text normalization (ITN), and translation support.
213
214 ```python
215 from nemo.collections.asr.inference.factory.pipeline_builder import PipelineBuilder
216 from omegaconf import OmegaConf
217
218 # Path to the buffered rnnt config file downloaded from above link
219 cfg_path = 'buffered_rnnt.yaml'
220 cfg = OmegaConf.load(cfg_path)
221
222 # Pass the paths of all the audio files for inferencing
223 audios = ['/path/to/your/audio.wav']
224
225 # Create the pipeline object and run inference
226 pipeline = PipelineBuilder.build_pipeline(cfg)
227 output = pipeline.run(audios)
228
229 # Print the output
230 for entry in output:
231 print(entry['text'])
232 ```
233
234 ---
235
236 ### Setting up Streaming Configuration
237
238 Latency is defined as the sum of the chunk size (middle part) and the right context.
239 For the left context we use 5.6s by default (5.6s was used during the model training), but you can try to find the optimal value for better accuracy/speed trade-off.
240
241 We would recommend to use the following context parameters for different latencies:
242
243 | Left, s | Chunk, s | Right, s | Latency (C+R), s |
244 | :---: | :---: | :---: | :---: |
245 | 5.6 | 1.04 | 1.04 | 2.08 |
246 | 5.6 | 0.56 | 0.56 | 1.12 |
247 | 5.6 | 0.16 | 0.40 | 0.56 |
248 | 5.6 | 0.08 | 0.24 | 0.32 |
249 | 5.6 | 0.08 | 0.16 | 0.24 |
250 | 5.6 | 0.08 | 0.08 | 0.16 |
251
252 ### Input
253
254 - Input Type(s): Audio
255 - Input Format(s): wav
256 - Input Parameters: One-Dimensional (1D)
257 - Other Properties Related to Input: Maximum Length in seconds specific to GPU Memory, No Pre-Processing Needed, Mono channel is required. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
258
259
260 ### Output
261
262 - Output Type(s): Text String in English
263 - Output Format(s): String
264 - Output Parameters: One-Dimensional (1D)
265 - Other Properties Related to Output: No Maximum Character Length, transcribe punctuation and capitalization. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
266
267
268 ## Datasets
269
270 ### Training Datasets
271
272 The majority of the training data comes from the English portion of the Granary dataset [4]:
273
274 - YouTube-Commons (YTC) (109.5k hours)
275 - YODAS2 (102k hours)
276 - Mosel (14k hours)
277 - LibriLight (49.5k hours)
278
279 In addition, the following datasets were used:
280
281 - Librispeech 960 hours
282 - Fisher Corpus
283 - Switchboard-1 Dataset
284 - WSJ-0 and WSJ-1
285 - National Speech Corpus (Part 1, Part 6)
286 - VCTK
287 - VoxPopuli (EN)
288 - Europarl-ASR (EN)
289 - Multilingual Librispeech (MLS EN)
290 - Mozilla Common Voice (v11.0)
291 - Mozilla Common Voice (v7.0)
292 - Mozilla Common Voice (v4.0)
293 - People Speech
294 - AMI
295
296 **Data Modality:** Audio and text
297
298 **Audio Training Data Size:** 530k hours
299
300 **Data Collection Method:** Human - All audios are human recorded
301
302 **Labeling Method:** Hybrid (Human, Synthetic) - Some transcripts are generated by ASR models, while some are manually labeled
303
304 ### Evaluation Datasets
305
306 The model was evaluated on the HuggingFace ASR Leaderboard datasets:
307
308 - AMI
309 - Earnings22
310 - Gigaspeech
311 - LibriSpeech test-clean
312 - LibriSpeech test-other
313 - SPGI Speech
314 - TEDLIUM
315 - VoxPopuli
316
317 ## Performance
318
319 ## ASR Performance (w/o PnC)
320
321 ASR performance is measured using the Word Error Rate (WER). Both ground-truth and predicted texts are processed using [whisper-normalizer](https://pypi.org/project/whisper-normalizer/) version 0.1.12. The obtained results for other models can be slightly different from the official HF model cards because of the different evaluation machines.
322
323 The following table show the WER on the [HuggingFace OpenASR leaderboard](https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) datasets including offline and streaming inference with different latency values:
324
325
326 | Model setup | Offline | 2.08s | 1.12s | 0.56s | 0.40s | 0.32s | 0.24s | 0.16s | 0.08s |
327 | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
328 | nvidia/parakeet-tdt-0.6b-v2 | 6.04 | 7.99 | 22.83 | 69.55 | 95.12 | — | — | — | — |
329 | nvidia/nemotron-speech-streaming-en-0.6b | 6.92 | 7.46 | 6.92 | 7.09 | 9.52 | 7.64 | 8.01 | **7.84** | **8.70** |
330 | nvidia/parakeet-unified-en-0.6b | **5.91** | **6.14** | **6.29** | **6.52** | **6.70** | **6.92** | **7.35** | 8.44 | 15.63 |
331
332
333 Parakeet-unified-en-0.6b model outperforms previous NVIDIA transducer-based models in offline and streaming (up to 240ms latency) inference modes. At 160ms latency, the unified model start to degrade because of the ansence of enough right context, yielding slightly to the strong streaming baseline. For 80ms latency we would recommend to use nemotron-speech-streaming-en-0.6b model instead.
334
335 ## Software Integration
336
337 **Runtime Engine:** NeMo 2.7.3
338
339 **Supported Hardware Microarchitecture Compatibility:**
340
341 - NVIDIA Ampere
342 - NVIDIA Blackwell
343 - NVIDIA Hopper
344 - NVIDIA Volta
345
346 **Test Hardware:**
347
348 - NVIDIA V100
349 - NVIDIA A100
350 - NVIDIA A6000
351 - DGX Spark
352
353 **Preferred/Supported Operating System(s):** Linux
354
355 ## Ethical Considerations
356
357 NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
358
359 Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).
360
361 ## References
362
363 [1] [Reducing the Offline-Streaming Gap for Unified ASR Transducer with Consistency Regularization](https://arxiv.org/abs/2604.19079)
364
365 [2] [Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition](https://arxiv.org/abs/2305.05084)
366
367 [3] [Dynamic Chunk Convolution for Unified Streaming and Non-Streaming Conformer ASR](https://arxiv.org/abs/2304.09325)
368
369 [4] [NVIDIA Granary](https://huggingface.co/datasets/nvidia/Granary)
370
371 [5] [NVIDIA NeMo Framework](https://github.com/NVIDIA/NeMo)
372