Most AI hardware advice assumes you are chasing frontier LLMs. Whisper is the opposite case: the largest speech-to-text model in the family is small by AI standards, and that flips the buying logic entirely. This guide covers what large-v3 actually needs, what an old gaming card delivers, and when buying anything is a mistake.
How much VRAM does Whisper large-v3 need?
Whisper large-v3 is a 1.54-billion-parameter encoder-decoder transformer released in November 2023. Per the fit page's verified architecture data, it needs about 4.4 GB of VRAM at FP16 — 3.1 GB of weights plus roughly 1 GB of runtime overhead for the Mel spectrogram and beam-search buffers — and only 2.0 GB at Q4_K_M quantization. The architecture is the reason it is so light: audio is processed in 30-second windows, so unlike an LLM there is no growing KV cache. The fit page sets the minimum at any 4 GB GPU at FP16 (or CPU-only via whisper.cpp) and recommends an 8 GB card of the RTX 3060/4060 class for faster-than-real-time batch transcription. For contrast, Mixture-of-Experts LLMs live in a different universe of requirements — see our guide to what DeepSeek V4 Flash needs to run locally. The whole Whisper family, for scale:
| Model | Parameters | VRAM at typical batch |
|---|---|---|
| tiny | 39 M | ~1 GB (estimated) |
| base | 74 M | ~1 GB (estimated) |
| small | 244 M | ~1–2 GB (estimated) |
| medium | 769 M | ~2–3 GB (estimated) |
| large-v3 | 1.54 B | 4.4 GB FP16 incl. overhead (verified); 2.0 GB at Q4_K_M |
Parameter counts come from OpenAI's Whisper repository; the large-v3 VRAM figures are from the fit page. The smaller models' VRAM entries are estimates scaled from parameter count — none are stated on the source pages.
What can you expect from an older card?
More than you might expect. The buying guide's core observation: Whisper's largest model fits comfortably on mainstream cards from several generations back, so VRAM capacity is rarely the deciding factor it is for LLMs or image generation. Software choice matters more here than hardware age — Faster-Whisper, SYSTRAN's reimplementation on CTranslate2, cuts compute cost substantially versus the original implementation, and the cheaper the card, the more that matters. Community Faster-Whisper setups report comfortable real-time factors on a previous-generation RTX 3060. One honesty note: the guide's benchmark database holds no Whisper-specific timings, so card comparisons use SDXL Turbo throughput as a compute reference — useful for batch planning, not a stopwatch.
Is CPU-only transcription viable?
Yes, with a throughput trade-off. The fit page calls CPU-only inference practical at roughly real-time speed with whisper.cpp, the heavily optimized C++ port; the original implementation is slower, with long recordings taking multiples of the audio duration in practice. That makes CPU fine for occasional files and painful for archives. If you already own an Apple Silicon Mac, try its native speech stack before buying any GPU — the guide's words — since optimized builds run through Apple's own frameworks. Only regular transcription volume justifies shopping for a card at all.
Which GPUs give the best value for transcription?
The guide's budget pick is the RTX 3060: cheap on the used market, enough memory and compute for the largest model, and the classic entry-level transcription card. Buyers wanting new hardware with documented specifications have the Arc B580 at a $249 MSRP with 12 GB, or the RTX 4060 Ti 16GB at $499 — whose 160 W TDP drops into most desktops without a PSU upgrade — and both exceed Whisper's requirements by a wide margin. The mid-range RTX 4070 Super suits batch workers processing podcast and meeting archives, while the RTX 5090 is only for buyers parallelizing huge audio libraries; its $1,999 price and 575 W draw buy capacity most Whisper users never touch. The guide's closing logic inverts usual GPU advice: the cheapest tier that covers your batch volume is the correct one. Full tier-by-tier detail is in the guide to the best GPUs for Whisper transcription. And if you want one card that also pulls LLM duty, a used 24 GB card is the flexible pick — our used RTX 3090 buying checklist covers what to inspect before paying.
Frequently asked questions
Can an old GTX 1070-class card run Whisper large-v3?
Yes, qualitatively: the fit page's minimum is any 4 GB GPU at FP16, and 8 GB Pascal-era cards clear that bar with room to spare. Expect slower batch throughput than modern cards, and run Faster-Whisper — optimized implementations benefit older hardware most.
Should I just use the small model instead of large-v3?
The family scales from 39 M to 1.54 B parameters, and smaller models trade accuracy for speed and memory. For clear audio and casual use, small runs on nearly anything; for demanding transcription, large-v3 still fits modest hardware, so the accuracy trade is often unnecessary.
Why does Whisper need so little VRAM compared to LLMs?
Two reasons: 1.54 billion parameters is tiny next to 32B-plus LLMs, and the 30-second audio window means no growing KV cache — memory use stays flat regardless of recording length.
Is a flagship GPU ever worth it for transcription?
Only for batch archives. Running many parallel workers turns transcription into a queue problem, and throughput drains the queue — but even 16 GB cards host multiple workers with room to spare, so flagships mostly buy unused capacity.
Sources and further reading
- Whisper large-v3 fit page on Compare AI Hardware (data cited above) — VRAM by quantization from the verified HF config (openai/whisper-large-v3: d_model 1280, 32 encoder + 32 decoder layers, 128 Mel bins)
- Best GPU for Whisper & Speech-to-Text AI in 2026 on Compare AI Hardware (linked above) — GPU picks, MSRP pricing, and the SDXL Turbo throughput reference table (Tom's Hardware and TechPowerUp measurements, June 2025)
- OpenAI Whisper repository — parameter counts per model size (tiny 39M, base 74M, small 244M, medium 769M, large 1550M): github.com/openai/whisper
Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.
No comments:
Post a Comment