How to Use Faster-Whisper: Turn Audio Into Accurate Transcripts

What It Is

Faster-whisper is a Python library that turns speech into text quickly and cheaply. It is a reimplementation of OpenAI’s Whisper model using CTranslate2, a fast inference engine for Transformer models. According to the README, it is up to 4 times faster than openai/whisper for the same accuracy while using less memory, and efficiency can be improved further with 8-bit quantisation on CPU and GPU. On GitHub, the repository SYSTRAN/faster-whisper has about 25,916 stars and is released under the MIT licence. For a small business, it is a practical building block: feed it an audio file, get back a transcript with timestamps, and use that text for notes, captions, searchable records, or follow-up drafts.

Who It’s For / SMB Use Case

Faster-whisper fits any business that creates a lot of spoken content. A consultancy can transcribe client interviews and turn them into summaries. A clinic, agency, or trades business can transcribe recorded calls or site walkthroughs into written records. A content creator can generate captions and show notes for videos and podcasts. A team that runs meetings can transcribe recordings and keep a searchable archive instead of relying on memory. The main requirement is comfort with Python, since faster-whisper is a library, not a finished app. If you need a point-and-click tool, the README lists community projects built on faster-whisper, including graphical interfaces. If you have a developer or a technically confident team member, the library gives you the most control at the lowest running cost.

Installation

The README lists Python 3.9 or greater as the requirement. Unlike openai-whisper, FFmpeg does not need to be installed on the system, because audio is decoded with the PyAV library, which bundles the FFmpeg libraries. Install the package from PyPI:

pip install faster-whisper

For GPU execution, the README requires Nvidia libraries: cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12. It notes that the latest versions of CTranslate2 only support CUDA 12 and cuDNN 9, and that older CUDA setups need a downgraded CTranslate2 version. On Linux, those libraries can be installed with pip, with LD_LIBRARY_PATH set before launching Python:

pip install nvidia-cublas-cu12 nvidia-cudnn-cu12==9.*

Alternatively, the README points to an official Nvidia CUDA Docker image that already contains the libraries. For most small-business tests, start on CPU. Move to GPU only when transcription volume or turnaround time justifies it.

How to Use

1. Transcribe a file on CPU. The README’s basic example loads a model and transcribes an audio file. This version runs on CPU with INT8 quantisation, which keeps memory use low:

   from faster_whisper import WhisperModel
   
   model = WhisperModel("large-v3", device="cpu", compute_type="int8")
   
   segments, info = model.transcribe("audio.mp3", beam_size=5)
   
   print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
   
   for segment in segments:
       print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

   Note the README’s warning: segments is a generator, so transcription only starts when you iterate over it. Collect the segments in a list or a for loop to run it to completion.

2. Run on GPU when available. Swap the model setup for GPU with FP16 precision, as shown in the README:

   model = WhisperModel("large-v3", device="cuda", compute_type="float16")

   Keep the rest of the code the same. Start with a smaller model, such as small or medium, if the large model is slow or runs out of memory on your machine.

3. Speed up batches. For many files, the README shows a batched pipeline that is a drop-in replacement for the standard transcribe method:

   from faster_whisper import WhisperModel, BatchedInferencePipeline
   
   model = WhisperModel("turbo", device="cuda", compute_type="float16")
   batched_model = BatchedInferencePipeline(model=model)
   
   segments, info = batched_model.transcribe("audio.mp3", batch_size=16)

   The README reports that batching cuts transcription time sharply in its benchmarks. Test with a small batch first, since larger batches use more memory.

4. Clean up the transcript. Add a voice-activity filter to skip silence, and request word-level timestamps when you need captions:

   segments, _ = model.transcribe("audio.mp3", vad_filter=True)
   
   segments, _ = model.transcribe("audio.mp3", word_timestamps=True)

   The README notes that the VAD filter integrates the Silero VAD model, and that its default behaviour only removes silence longer than 2 seconds. Save the segments to a text or subtitle file so the transcript is reusable.

Pricing / Limitations

Faster-whisper is free, open-source software under the MIT licence, so there is no licence or per-minute fee for the library itself. Your costs are the computer or server you run it on, plus electricity or cloud compute. When you load a model by size, such as WhisperModel("large-v3"), the corresponding CTranslate2 model is downloaded automatically from the Hugging Face Hub, per the README. The limits are practical. Accuracy depends on audio quality, background noise, accents, and the model size you choose. Large models need significant RAM or VRAM. The library does not include speaker diarization, which identifies who said what, though the README lists community projects that add it, such as WhisperX and whisper-diarize. Finally, transcription is only the first step: you still need to review the text, especially for names, numbers, and anything you plan to publish or act on.

Bottom Line for SMB

Faster-whisper is the cheapest reliable way to turn your audio into text at volume. Start with one real recording, transcribe it on CPU with a small or medium model, and check the quality. If the transcript is good enough for notes and captions, build a simple repeatable workflow around it before spending anything on a GPU.

FAQ

Do I need FFmpeg to use faster-whisper?

No. The README states that unlike openai-whisper, FFmpeg does not need to be installed, because audio is decoded with PyAV, which bundles the FFmpeg libraries.

Which model size should I start with?

Start small and move up only if accuracy is not good enough. Smaller models are faster and use less memory, while larger models such as large-v3 are more accurate but heavier. The README’s benchmarks compare implementations on the same audio, so test on your own recordings before committing.

Can faster-whisper tell speakers apart?

Not on its own. The README lists community integrations that add speaker diarization, including WhisperX and whisper-diarize. For exact setup, check those projects’ docs.

Related reading: Best AI Meeting Assistants and Otter.ai Review.