Speech to Text

To transcribe audio into text, NobodyWho provides an integration with the Whisper models in ONNX format.

import 'package:nobodywho/nobodywho.dart' as nobodywho;

// ... after NobodyWho.init().
final stt = nobodywho.Stt(source: 'onnx-community/whisper-base');

final text = await stt.transcribeFile('recording.mp3').completed();
print(text);

If the audio is not coming from a file, but instead directly from a buffer, transcribePcm is available:

final text = await stt.transcribePcm(samples, 16000).completed();

In order to make this work, the buffer needs to be mono i16 PCM samples. The sample rate can be anything - NobodyWho resamples internally to what Whisper expects.

As with classic Chat models, streaming is available, so the transcription can be consumed token by token:

await for (final piece in stt.transcribeFile('recording.mp3')) {
  stdout.write(piece);
}

Supported models

NobodyWho only supports Whisper models in ONNX format. source is a Hugging Face repo ID or a local directory containing such a model, e.g. onnx-community/whisper-base. Browse the Whisper ONNX models on Hugging Face to pick a size that fits your accuracy and speed needs.

You can also pick a quantization variant of the model to download and load. Lower-precision variants are smaller and faster, but can lose some transcription accuracy. Supported values are default, fp16, int8, uint8, bnb4, q4, q4f16, and quantized. Defaults to default.

final stt = nobodywho.Stt(
  source: 'onnx-community/whisper-base',
  quantization: 'int8',
);

Improving performance

By default, Whisper auto-detects the spoken language, which costs a bit of extra processing. If you already know the language, pass its ISO 639-1 code as language to skip detection and improve performance:

final stt = nobodywho.Stt(
  source: 'onnx-community/whisper-base',
  language: 'en',
);