Best Local AI Server: questions & answers

Our editorial choice is Olares One. It ships ready with Olares OS, separate accounts and remote access, built around an RTX 5090 Mobile with 24 GB of VRAM. If your team needs 70B models, a 128 GB machine such as DGX Spark or a Ryzen AI Max+ 395 box is the better buy.

By an approximate rule, a 4-bit model needs 0.5-0.6 GB per billion parameters plus a few GB for context. That is about 5 GB for 8B, 19-20 GB for 32B and 40 GB for 70B.

It still offers 24 GB of VRAM at 936 GB/s on CUDA. SpecPicks estimated a full tower at 3,800 USD in its November 2026 update, without a case, and about 515 W at sustained full load. You build and maintain it yourself.

Olares One for models up to about 32B, image and video work and a ready multi-user server, at a 4,999 USD list price. DGX Spark for 70B and larger models in 128 GB of unified memory, at 6,950 USD on NVIDIA's store (prices read 12-13 Nov 2026).

Yes, if the model runner batches requests. In Olares' own test, Olares One served Qwen3-30B-A3B at about 81 tokens/s per request with 8 concurrent requests. Give each person their own account.

Not comfortably. A dense 70B model needs about 40 GB at 4-bit by the approximate rule, more than its 24 GB of VRAM. Models that do not fit spill into the 96 GB of system memory and slow down. Olares' FAQ gives about 36 tokens/s for gpt-oss-120b run that way.

Yes. The Olares Market lists both as one-click installs. LM Studio cannot be installed on the box, and according to Olares' FAQ it can connect to models on Olares One through the API.

It depends on the machine. In Olares' lab test, Olares One measured 19 dB at idle and under 39 dB at full load; Olares lists 30 W in standby and a 330 W adapter. NVIDIA rates the desktop RTX 5090 card alone at 575 W.