A system is listed as fitting when its GPU memory meets the requirement
shown above. Speed, usable context, batch size and concurrent users
still need testing with the final model and software configuration.
The first system with enough installed GPU memory when supported software divides the model across its GPUs. We confirm software support, GPU layout, context and performance during sizing.
Explore Enterprise 768 Larger-memory system recommended - Per GPU
- 32GB
- Total GPU memory
- 32GB
- GPU count
- 1
This system provides 32GB per GPU and 32GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 32GB
- Total GPU memory
- 64GB
- GPU count
- 2
This system provides 32GB per GPU and 64GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 96GB
- Total GPU memory
- 96GB
- GPU count
- 1
This system provides 96GB per GPU and 96GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 96GB
- Total GPU memory
- 192GB
- GPU count
- 2
This system provides 96GB per GPU and 192GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 32GB
- Total GPU memory
- 128GB
- GPU count
- 4
This system provides 32GB per GPU and 128GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 32GB
- Total GPU memory
- 256GB
- GPU count
- 8
This system provides 32GB per GPU and 256GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Larger-memory system recommended - Per GPU
- 96GB
- Total GPU memory
- 384GB
- GPU count
- 4
This system provides 96GB per GPU and 384GB in total. The model requires 416GB aggregate across at least 4 GPUs.
For this model, explore Enterprise 768 .
Multi-GPU route available - Per GPU
- 96GB
- Total GPU memory
- 768GB
- GPU count
- 8
Total installed memory is sufficient, but the model must be divided across GPUs. Confirm that the serving software supports this GPU layout and test the required context and speed.
Multi-GPU route available - Per GPU
- 141GB
- Total GPU memory
- 1,128GB
- GPU count
- 8
Total installed memory is sufficient, but the model must be divided across GPUs. Confirm that the serving software supports this GPU layout and test the required context and speed.
Multi-GPU route available - Per GPU
- 288GB
- Total GPU memory
- 2,304GB
- GPU count
- 8
Total installed memory is sufficient, but the model must be divided across GPUs. Confirm that the serving software supports this GPU layout and test the required context and speed.
Multi-GPU route available - Per GPU
- 288GB
- Total GPU memory
- 20,736GB
- GPU count
- 72
Total installed memory is sufficient, but the model must be divided across GPUs. Confirm that the serving software supports this GPU layout and test the required context and speed.