Best larger-system route
Team 32
The smallest system in the range that reaches this model's preferred working allowance.
Explore Team 32BAAI model
BGE-M3 Hardware Requirements & RAG Server Fit
BGE-M3 is small enough that model memory is not the buying constraint for this range. Every product can host it; corpus size, indexing throughput, reranking, vector storage and the language model endpoint determine the wider system.
- Model version
BAAI/bge-m3- Family and variant
- BGE-M3 · bge-m3
- Source version
5617a9f61b02- Source updated
- 3 July 2024
BAAI BGE-M3
BAAI/bge-m3 - Minimum GPU memory
- 4GB GPU planning floor, or CPU for light use
- Recommended hardware
- 12GB or more for sustained indexing and queries
- Licence
- MIT License
- Useful for
- Embeddings & RAG
Buyer verdict
Where BGE-M3 is a sensible fit
BGE-M3 is useful for multilingual RAG and hybrid retrieval where one model should produce dense, sparse and ColBERT-style representations.
These figures apply to the named model version. Quantisation, fine-tuning, context length, image resolution, batch size and serving software can materially change the hardware needed.
Hardware requirements
- Minimum
- 4GB GPU planning floor, or CPU for light use
- Recommended
- 12GB or more for sustained indexing and queries
Batch and long inputs can raise working memory.
The full RAG pipeline, not BGE-M3 alone, usually determines the product.
Technical specification
BGE-M3 model and hardware facts
Specifications shown for source version
5617a9f61b028005a4858fdac845db406aefb181, updated
3 July 2024.
- Embedding dimensions
- 1,024
- Input length
- 8,192 tokens
- Retrieval modes
- Dense + sparse + multi-vector
- Repository weights
- About 2.27GB
- Languages
- 100+
- Licence
- MIT
Product compatibility
BGE-M3 compatibility across all 11 GPU systems
- Systems that fit without model splitting
- 11
- Systems needing multi-GPU validation
- 0
A system is listed as fitting when its GPU memory meets the requirement shown above. Speed, usable context, batch size and concurrent users still need testing with the final model and software configuration.
sme workstation
Team 32
- Per GPU
- 32GB
- Total GPU memory
- 32GB
- GPU count
- 1
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Company 64
- Per GPU
- 32GB
- Total GPU memory
- 64GB
- GPU count
- 2
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Studio 96
- Per GPU
- 96GB
- Total GPU memory
- 96GB
- GPU count
- 1
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
sme workstation
Studio 192
- Per GPU
- 96GB
- Total GPU memory
- 192GB
- GPU count
- 2
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Value Rack 128
- Per GPU
- 32GB
- Total GPU memory
- 128GB
- GPU count
- 4
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Value Rack 256
- Per GPU
- 32GB
- Total GPU memory
- 256GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Enterprise 384
- Per GPU
- 96GB
- Total GPU memory
- 384GB
- GPU count
- 4
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
Enterprise 768
- Per GPU
- 96GB
- Total GPU memory
- 768GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
pcie rack
H200 1.1TB
- Per GPU
- 141GB
- Total GPU memory
- 1,128GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
frontier partner
Frontier Native 2.3TB
- Per GPU
- 288GB
- Total GPU memory
- 2,304GB
- GPU count
- 8
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
frontier partner
Frontier Rack 20TB
- Per GPU
- 288GB
- Total GPU memory
- 20,736GB
- GPU count
- 72
Also meets this model page's recommended working allowance.
Available GPU memory exceeds the calculated or estimated requirement. Confirm the final precision, software, workload and performance before purchase.
Deployment reality
Strengths, limits and runtime route
What it is good at
- Dense, sparse and multi-vector retrieval from one checkpoint.
- More than 100 languages and 8,192-token input.
- MIT licence.
- A mature Sentence Transformers and FlagEmbedding ecosystem.
Where to be cautious
- The model card's benchmark results do not predict retrieval quality on a buyer's corpus.
- Long inputs reduce throughput and may not improve retrieval.
- Hybrid retrieval adds index and operational complexity.
- Reranking and answer generation need separate models and capacity.
Serving software
- Sentence Transformers
- FlagEmbedding
- Text Embeddings Inference
- ONNX
Before installation
- Evaluate dense, sparse and hybrid modes against the same relevance set.
- Record chunking, token length, batch and index configuration.
- CPU serving may be viable at low volume; a GPU is useful for sustained indexing and lower latency.
- Do not size the server from document count alone.
System requirements
- GPU layout
- The minimum can fit on one GPU; multiple GPUs may still be useful for replicas or throughput.
- Context and cache
- The model context ceiling is not a guaranteed serving target. KV cache, batch size and concurrent sessions need separate capacity tests.
- System RAM
- Size system memory for model loading, runtime overhead, preprocessing and any CPU offload used by the final configuration.
- Storage
- Allow space for the pinned checkpoint, runtime images, caches, logs and at least one rollback version.
- Serving software
- Validate the exact checkpoint with Sentence Transformers, FlagEmbedding, Text Embeddings Inference, ONNX before acceptance.
- Representative workload
- Benchmark representative prompts or media at the required context, quality, latency and concurrency.
Performance: speed, usable context and concurrency depend on the selected system, software and workload. No benchmark is quoted on this page.
Commercial and legal boundary
MIT License
Commercial use: permitted
MIT permits commercial use subject to inclusion of the copyright and licence notice.
Always retain the applicable notices and recheck the live terms for the intended organisation, territory, use and distribution route. Obtain legal advice where required; the official licence governs use.
Read the official licenceTechnical questions
BGE-M3 deployment FAQ
Which GPU Servers product fits BGE-M3?
All of them. Team 32 is already much larger than the model's memory requirement, so choose from the complete RAG workload.
Can BGE-M3 run on CPU?
Yes for lighter workloads. GPU acceleration becomes useful for sustained indexing, long inputs or stricter latency.
Does BGE-M3 replace a reranker?
Not necessarily. A separate reranker can improve precision, but adds latency and another model to operate.
What a complete BGE-M3 deployment needs
GPU memory is only one part of the system. Storage, data access, serving software, monitoring and administrator handover also affect a reliable deployment.
