- Home
- IA locale
HARDWARE SIZING TOOL · 2026-11-05
What hardware for local Mistral AI?
A first estimate — no account and no document upload. Describe your workload to get a hardware profile to review with an engineer.
02 / Indicative result
Recommended model
03 / SOURCES
Open Mistral models: weights, context and licences
Architecture details from config.json/params.json on Hugging Face. Verified 2026-11-05. API-retired models are marked where weights remain downloadable.
| Modèle / dépôt source | Parameters | Layers | KV heads | head_dim / MLA | Maximum context | Licence |
|---|---|---|---|---|---|---|
| Ministral 3 8B Instruct (2512) | 8.9B | 34 | 8 | 128 | 256k | Apache-2.0 |
| Ministral 3 14B Instruct (2512) | 13.9B | 40 | 8 | 128 | 256k | Apache-2.0 |
| Mistral Small 3.2 24B Instruct (2506)Retired from Mistral's API on 2026-07-31 (replaced by Mistral Small 4); open weights remain downloadable. | 24.0B | 40 | 8 | 128 | 128k | Apache-2.0 |
| Devstral Small 2 24B Instruct (2512) | 24.0B | 40 | 8 | 128 | 256k | Apache-2.0 |
| Mistral Small 4 119B (2603) — MoE | 119.4B6.5B actifs | 36 | — | MLA rank 256 + rope 64 | 256k | Apache-2.0 |
| Devstral 2 123B Instruct (2512) | 125.0B | 88 | 8 | 128 | 256k | Modified MITNot usable if the company's (or employer's) consolidated monthly revenue exceeds USD 20 million; commercial licence from Mistral AI otherwise. |
| Mistral Medium 3.5 128B | 127.7B | 88 | 8 | 128 | 256k | Modified MITNot usable if the company's (or employer's) consolidated monthly revenue exceeds USD 20 million; commercial licence from Mistral AI otherwise. |
| Mistral Large 3 675B Instruct (2512) — MoEModel card: FP8 runs on one 8×H200 (or B200) node; NVFP4 on one node of H100/A100. Parameter count from the model card (params.json has no total). | 675.0B41B actifs | 61 | — | MLA rank 512 + rope 64 | 256k | Apache-2.0 |
Apache 2.0 generally permits commercial use subject to its terms. Modified licences may include exceptions: read the linked text for your specific use.
04 / MÉTHODE
How we calculate
Reproducible sizing from published configs and verifiable hardware specifications. No fabricated prices.
1. Weights and KV cache
poids (GB) = paramètres × bits / 8 / 10⁹KV GQA = 2 × couches × têtes KV × head_dim × octets × tokens × utilisateursKV MLA = couches × (kv_lora_rank + qk_rope_head_dim) × octets × tokens × utilisateursWeights ≈ parameters × effective bits / 8. GQA KV = 2 (K+V) × layers × KV heads × head_dim × bytes per element × context × users. FP8 uses 1 byte for K/V; llama.cpp uses 2 bytes. Total = weights + KV + 10% runtime margin + fixed 2 GB. Assumed 50% context fill per user.
2. Throughput and hardware
Estimated decode throughput, not benchmarked: memory bandwidth × assumed efficiency (GPU 50–75%, Apple 45–70%) / weights read and active KV-cache. Simplified bandwidth, batch and kernel model; real-world results vary widely.
GPU usable capacity assumed 95%; Apple unified memory 75%. Multiple GPUs require model sharding; estimated 15% interconnect penalty. System RAM ≈ provisioned GPU memory, disk ≥ 2× weights + 500 GB, PSU ≈ 1.25× (GPU + 350 W). No prices: prices and availability change.
05 / IA PRIVÉE
Why local AI?
A local installation can avoid sending prompts and documents to a cloud AI provider. Mistral AI is a French company founded in Paris. Local deployment may reduce certain data flows; it does not replace legal review or security controls.
No legal, GDPR, professional-secrecy, HDS or SecNumCloud guarantee is implied.
- RGPD · contrôle des flux à analyser
- Secret professionnel · configuration et accès
- HDS / SecNumCloud · certifications distinctes
- Mistral AI · entreprise française
06 / FAQ
Frequently asked questions
Is a local server enough for GDPR?
No. It limits some transfers to cloud providers, but compliance depends on all processing, security, access, processors and retention periods.
What are HDS and SecNumCloud?
They are separate certification frameworks. A local model or workstation is not automatically HDS or SecNumCloud certified. Validate applicable requirements.
Which quantisation should I choose?
Q4 reduces memory and may affect some tasks; Q8 is usually closer to native weights and uses more memory. Evaluate with representative data and check redistributed quantisation terms.
Why multiply KV cache by users?
Each concurrent sequence has its own KV cache. The formula uses the requested context and simultaneous users.
Are tokens/s guaranteed?
No. They are theoretical ranges based on bandwidth and assumed efficiency. Server, kernels, batching, quantisation and output length change throughput.
Which data stays local?
Prompts and documents can stay on your server if the system does not call cloud services. Logs, telemetry, backups and integrations must also be audited.
Mac or NVIDIA GPU?
Unified memory fits models larger than one GPU, but is shared with the system. Dedicated GPUs often suit CUDA/vLLM and multi-user service better.
Can I process health data?
Do not infer HDS eligibility from local installation alone. Map data and verify hosting and sector obligations before processing.
Discuss this configuration
Your configuration is attached to the contact form for a contextual discussion.