Best larger-system route
Team 32
The smallest system in the range that reaches this model's preferred working allowance.
Explore Team 32OpenAI model
Whisper Large V3 GPU Requirements & Server Fit
Whisper Large V3 is not memory-bound on this range. All products can host it. Select hardware from audio hours per day, simultaneous streams, latency, diarisation and any post-processing model.
- Model version
openai/whisper-large-v3- Family and variant
- Whisper Large V3 · whisper-large-v3
- Source version
06f233fe06e7- Source updated
- 12 August 2024
OpenAI Whisper Large V3
openai/whisper-large-v3 - Minimum GPU memory
- 6GB GPU planning floor for an optimised inference runtime
- Recommended hardware
- 12GB or more for useful batch and concurrency headroom
- Licence
- Apache License 2.0
- Useful for
- Speech & audio
Buyer verdict
Where Whisper Large V3 is a sensible fit
Whisper Large V3 remains a useful baseline for multilingual transcription and English translation because it is well documented, Apache-licensed and supported by several optimised runtimes.
These figures apply to the named model version. Quantisation, fine-tuning, context length, image resolution, batch size and serving software can materially change the hardware needed.
Hardware requirements
- Minimum
- 6GB GPU planning floor for an optimised inference runtime
- Recommended
- 12GB or more for useful batch and concurrency headroom
Compute type, batch and timestamps change the working requirement.
The current 32GB entry product already exceeds this memory envelope.
Technical specification
Whisper Large V3 model and hardware facts
Specifications shown for source version
06f233fe06e710322aca913c1bc4249a0d71fce1, updated
12 August 2024.
- Parameters
- 1.55B
- Task
- Multilingual transcription + translation
- Core weight format
- FP16 variants available
- Audio window
- 30-second model segments
- Ecosystem
- Transformers, CTranslate2, whisper.cpp
- Licence
- Apache 2.0
Product compatibility
Whisper Large V3 compatibility across all 11 GPU systems
- Systems that fit without model splitting
- 11
- Systems needing multi-GPU validation
- 0
A system is listed as fitting when its GPU memory meets the requirement shown above. Speed, usable context, batch size and concurrent users still need testing with the final model and software configuration.
sme workstation
Team 32
- Per GPU
- 32GB
- Total GPU memory
- 32GB
- GPU count
- 1
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Company 64
- Per GPU
- 32GB
- Total GPU memory
- 64GB
- GPU count
- 2
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Studio 96
- Per GPU
- 96GB
- Total GPU memory
- 96GB
- GPU count
- 1
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Studio 192
- Per GPU
- 96GB
- Total GPU memory
- 192GB
- GPU count
- 2
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Value Rack 128
- Per GPU
- 32GB
- Total GPU memory
- 128GB
- GPU count
- 4
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Value Rack 256
- Per GPU
- 32GB
- Total GPU memory
- 256GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Enterprise 384
- Per GPU
- 96GB
- Total GPU memory
- 384GB
- GPU count
- 4
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Enterprise 768
- Per GPU
- 96GB
- Total GPU memory
- 768GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
H200 1.1TB
- Per GPU
- 141GB
- Total GPU memory
- 1,128GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
frontier partner
Frontier Native 2.3TB
- Per GPU
- 288GB
- Total GPU memory
- 2,304GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
frontier partner
Frontier Rack 20TB
- Per GPU
- 288GB
- Total GPU memory
- 20,736GB
- GPU count
- 72
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
Deployment reality
Strengths, limits and runtime route
What it is good at
- Mature ecosystem including Transformers, faster-whisper and whisper.cpp.
- Multilingual transcription and translation.
- Apache 2.0 licence.
- Small enough for several workers on larger products.
Where to be cautious
- The repository includes multiple framework representations, so its total file sum is not loaded GPU memory.
- Speaker diarisation is not provided by the base checkpoint.
- Word-level timestamps, streaming and batching vary by runtime.
- Accuracy can fall with noise, overlap, accents and specialist vocabulary.
Serving software
- faster-whisper
- Transformers
- whisper.cpp
- OpenAI Whisper
Before installation
- Choose one serving implementation and pin its compute type.
- Measure real-time factor and word error rate on representative audio.
- Record channel count, sample rate, average duration and concurrent streams.
- Add a separate diarisation and redaction evaluation where required.
System requirements
- GPU layout
- The minimum can fit on one GPU; multiple GPUs may still be useful for replicas or throughput.
- Context and cache
- The model context ceiling is not a guaranteed serving target. KV cache, batch size and concurrent sessions need separate capacity tests.
- System RAM
- Size system memory for model loading, runtime overhead, preprocessing and any CPU offload used by the final configuration.
- Storage
- Allow space for the pinned checkpoint, runtime images, caches, logs and at least one rollback version.
- Serving software
- Validate the exact checkpoint with faster-whisper, Transformers, whisper.cpp, OpenAI Whisper before acceptance.
- Representative workload
- Benchmark representative prompts or media at the required context, quality, latency and concurrency.
Performance: speed, usable context and concurrency depend on the selected system, software and workload. No benchmark is quoted on this page.
Commercial and legal boundary
Apache License 2.0
Commercial use: permitted
Apache 2.0 permits commercial use. Audio rights, consent and retention remain the operator's responsibility.
Always retain the applicable notices and recheck the live terms for the intended organisation, territory, use and distribution route. Obtain legal advice where required; the official licence governs use.
Read the official licenceTechnical questions
Whisper Large V3 deployment FAQ
Which product should I buy for Whisper Large V3?
Team 32 is already sufficient for one substantial worker. Move up for parallel workers, combined services or much larger audio queues.
Does Whisper identify speakers?
Not by itself. Speaker diarisation requires a separate component and evaluation.
Can Whisper run without a GPU?
Yes, especially with optimised or quantised runtimes, but a GPU improves turnaround and concurrency for sustained use.
What a complete Whisper Large V3 deployment needs
GPU memory is only one part of the system. Storage, data access, serving software, monitoring and administrator handover also affect a reliable deployment.
