eRacks Systems Tech Blog
Open Source Experts Since 1999
Private AI, Sized and Priced: How to Spec a Server to Run LLMs On-Premise (from $5,995)
Running large language models (LLMs, the AI models behind chat assistants) on your own hardware, often called “private” or “on-premise” AI, keeps your data inside your building, replaces per-user cloud fees with a one-time purchase, and removes vendor lock-in. The catch is sizing it right. We just published a full, vendor-neutral guide to doing exactly that, and here is the short version.
Why run AI on your own hardware?
Three reasons come up again and again. Privacy and compliance: protected health information (HIPAA), attorney-client material, controlled government data, and source code often cannot legally or contractually leave your control. Predictable cost: a one-time purchase instead of per-seat or per-token billing that grows with every user and every query. Control: your models, your uptime, no rate limits, and no vendor quietly deprecating the model your workflow depends on. For light or occasional use a cloud API is cheaper and simpler; private AI wins when you have data you cannot send out, or when usage is steady and everyday.
The first number: GPU memory (VRAM)
A model has to fit in GPU memory (VRAM, the fast memory on the graphics card) to run at full speed. How much you need is set by the model’s parameter count and its quantization (compressing the weights to fewer bits each: Q4 is about 4 bits per weight and near-lossless for most tasks, Q8 is about 8 bits, fp16 is full precision).
| Model size | Q4 (4-bit) | Q8 (8-bit) | Good for |
|---|---|---|---|
| 7 to 8B (Llama 3.1 8B, Mistral) | ~6 GB | ~10 GB | chat, RAG, coding assist |
| 32 to 34B (Qwen 2.5 32B) | ~22 GB | ~38 GB | strong reasoning, agents |
| 70B (Llama 3.3 70B) | ~42 GB | ~80 GB | frontier-class open models |
| 120B+ or several at once | 70 GB+ | 140 GB+ | heavy or multi-tenant |
Updated July 2026: the mid-tier was relaunched as the AISLING/AILEEN Good, Better, Best ladder, and the AINE and HIGHLANDER flagships joined the line. Full story in our relaunch post.
A quick rule: VRAM in GB is roughly the parameter count in billions times 0.6 for Q4, or times 1.1 for Q8, with context headroom included. (RAG, or retrieval-augmented generation, feeds the model your own documents at query time.)
It is not only VRAM: system RAM and CPU matter too
System RAM stages models into the GPUs, runs the model server and your data pipeline, and spills over when a model is slightly too big for VRAM. Size it at roughly 1.5 to 2 times your total VRAM. CPU and PCIe lanes: the processor feeds the GPUs through PCIe lanes, so a multi-GPU server needs enough lanes to drive every card at full bandwidth. That is why we build on server-class AMD EPYC and Intel Xeon processors rather than desktop chips: far more PCIe lanes, and support for ECC (error-correcting) memory.
When self-hosting beats the cloud
The arithmetic is direct. A cloud subscription such as ChatGPT Team runs about $30 per user per month. For a 30-person team that is roughly $10,800 a year, every year, with your prompts on someone else’s servers. An on-premise eRacks AILSA at $5,995 covers the same everyday inference on hardware you own, and pays for itself in under a year. In practice, self-hosting wins at roughly 5 to 10 or more regular users, or any privacy mandate.
The GPUs: VRAM without the NVIDIA tax
You do not need flagship NVIDIA silicon to run these models. You need VRAM.
- Intel Arc Pro B50 16GB (low-profile, about $349 to $399): the value pick. Four give 64 GB for well under $8,000 of GPU.
- Intel Arc Pro B70 32GB (about $949): roughly half the price per gigabyte of comparable NVIDIA professional cards. Four give 128 GB.
- NVIDIA RTX PRO 4000 Blackwell SFF 24GB: when you need the CUDA ecosystem and ECC memory in a small, 70-watt card.
The eRacks AI lineup
| Server | GPU memory | Comfortably runs | From |
|---|---|---|---|
| AILSA (2U) | up to 96 GB | Llama 3.3 70B (Q4), Qwen 2.5 32B | $5,995 |
| AIDAN (2U) | 32 GB | 32 to 34B models, 8B at full precision | $13,895 |
| AISLING (4U) | up to 96 GB | 70B-class quantized, fine-tuning | $16,995 |
| AILEEN (4U) | up to 128 GB | 70B with room for long context, redundant power | $21,995 |
| AISHA (4U) | up to 256 GB | 70B at Q8, or several models, multi-tenant | $30,995 |
| AINE (4U) | up to 192 GB | Multi-70B serving, 200B-class quantized | $84,995 |
| HIGHLANDER (4U) | up to 768 GB | The largest open models, entirely on-premise | $154,995 |
Every eRacks AI server ships with Ubuntu LTS (long-term-support Linux) and a complete open-source AI stack (Ollama, Open WebUI, vLLM, llama.cpp, PyTorch) pre-installed and tested. Staff reach the AI from a browser on day one. No per-seat or per-token fees, you own the hardware, and your data never leaves the building.
Bottom line
Start from the model, not the GPU: decide the largest model you will run and at what quantization, size the VRAM (about params times 0.6 for Q4), then add system RAM at 1.5 to 2 times that, and choose a server CPU with the lanes for your GPU count. If privacy is the driver, on-premise is the answer and the only question is which size. And the entry is lower than people expect: a 70B-class model, private, from $5,995.
Configure an AI server → or read the full sizing guide.
Want us to size one to your exact models and user count, at no charge? Reply to this post, a real engineer will help.
joe July 27th, 2026
Posted In: AI, AI Servers, Deep Learning, News
Tags: AI inference server, AILSA, GPU VRAM, HIPAA AI, Intel Arc Pro B50, Intel Arc Pro B70, Llama 70B, local LLM, on-premise AI, open source AI server, private AI, Qwen 32B, self-hosted LLM, system RAM
The Price You See Is the Price: Transparent Server Pricing in a Shortage
This month, one of our distributors repriced a GPU by 29 percent. Overnight. The same card, quoted to us on a Thursday, cost $280 more on Friday. Server-grade DDR5 ECC memory (error-correcting code memory, the kind servers use to detect and fix bit errors) is running $24 to $28 per gigabyte on the street, roughly triple what it cost before the 2026 memory shortage. Enterprise hard drives jumped too.
In a market like this, most server vendors reach for the same three tricks. We think you should know what they are, because we built our whole pricing system to do the opposite.
The three tricks
The teaser config. The advertised price quietly assumes last year’s components: less memory than the workload needs, a boot drive standing in for storage, a processor two generations back. The real configuration costs thousands more, which you discover after you are invested.
“Call for pricing.” If the vendor will not put a number on the page, the number depends on who is asking. That is not pricing, that is negotiation, and it always favors the side with more information. That is never the buyer.
The silent quote. A price quoted in a volatile market is a bet. Some vendors let you carry a stale quote right up to the purchase order, then “revise” it when backing out is hardest.
What we do instead
The configurator price is the price. Every configuration on eracks.com produces a real, orderable number, computed from what the components actually cost us. No “starting at” games: change the memory, the drives, the GPUs, and watch the price move in both directions, up and down, with the parts market.
Robots watch the market so the catalog stays true. Automated trackers re-check memory, drive, and GPU street prices daily and weekly, and feed re-pricing that a human reviews before it ships. When a component spikes, we re-price openly rather than let the old number quietly stop being true.
When a build goes underwater, we fix the build in the open. This week our own margin checks flagged two NAS models whose entry configurations no longer made sense at current drive prices. We did not pad the price and we did not gut the spec quietly. We changed the default drive count, published the same price, and left every larger configuration exactly where it was. The entry price stayed real.
Quotes tell you everything up front. As of this month, every formal eRacks quote carries five things: your discounts by name, a validity window, a component-volatility note, an estimated lead time (in this market, chassis allocation is a real thing, and you deserve to know it before you order), and a configuration that matches a product we actually ship today. That last one sounds obvious. It is not industry practice.
When we cannot source a part at a price we can stand behind, we say so. Some configurations show “get a quote” instead of a number. That is not coyness. It means the market for that part is moving too fast for a printed price to stay true, so we price it fresh, with a validity window, when you ask.
Why bother
Earlier this year a prospect told us one of our configurations looked misleading. We did not enjoy hearing it, and he had a point: a stale page had drifted out of sync with the parts market. That critique turned into the system described above: automated price tracking, margin floors, re-verified defaults, and the five-line quote rule. The fix for being wrong is not better wording. It is better plumbing.
We have been building open-source servers since 1999. No venture capital, no lock-in, no per-seat fees, and nothing in the business model that needs a pricing trick to work. In a shortage, transparent pricing is slower and occasionally embarrassing. It is also the only kind worth publishing.
Configure anything, from a ZFS NAS to a private AI server, and the number you see is the number. If your workload does not fit a page, ask us and you will get a quote with all five lines on it.
joe July 20th, 2026
Posted In: News
Tags: AI server, DRAM shortage, NAS, no lock-in, Open Source, quotes, server pricing, supply chain, transparent pricing
We Re-Verified Every NAS Default Against the 2026 Drive Shortage. Here’s What Changed.
The 2026 drive and memory shortage has quietly broken a lot of storage price lists. Drives that vendors still advertise as standard are backorder-only at every national retailer. Memory prices moved 50 percent in a quarter. A configurator that hasn’t been re-checked against real supply this year is, politely, fiction.
So we re-verified our entire NAS line: every default component, checked against multiple named suppliers, for both price and actual in-stock availability. Here is what changed, and what it means if you’re shopping for storage this summer.
The 30TB drive problem
Our large NAS models used to default to the 30TB Seagate IronWolf Pro. It’s a fine drive, and it has become effectively unobtainable: backorder-only at the one national supplier still listing it, dropped from Seagate’s own current lineup, unavailable everywhere else we checked. Continuing to default to it would mean quoting systems we could not ship this week.
The fix: our high-capacity systems now default to the 24TB IronWolf Pro, in stock at multiple national retailers at about $35 per terabyte, and we added the 32TB IronWolf Pro, also in stock, at the top of the ladder at nearly the same cost per terabyte. Every drive in our NAS configurators remains CMR (conventional magnetic recording); we do not use shingled SMR drives, whose write performance collapses during the sustained writes of a RAID rebuild, exactly when you can least afford it.
ZFS RAIDZ2, now the default everywhere
Every eRacks storage system, from the 1U four-bay NAS4 to the petabyte-class NAS100, now defaults to a ZFS RAIDZ2 pool: dual parity, meaning any two drives can fail without losing data. RAIDZ3 (triple parity) and striped mirrors (the performance-first layout) are right there in the dropdown, and traditional hardware RAID remains available for shops that require it, just never as a silent default.
Under every pool is an IT-mode HBA (a host bus adapter that passes drives directly to the operating system), because ZFS wants to see raw drives to do its end-to-end checksumming and self-healing. If you’re deciding between RAIDZ2, RAIDZ3, and mirrors, our free ZFS layout guide works through the math.
Prices that match what parts cost
Component costs moved, so prices moved: mid-size and large models are repriced to current reality (the NAS12 now starts at $8,995, the NAS72 at $25,995), and every price in the configurator reflects a component list we verified we can buy this week. The entry line held: the NAS4 still starts at $1,995 and the NAS6 at $2,995, now with 16GB of DDR5 standard and ECC memory available as an upgrade.
This is the same discipline we described on our refreshed Components We Use page: new parts only, authorized US distribution, multi-source price-and-availability checks before anything gets quoted, and a 72-hour burn-in before anything ships.
Browse the re-verified line at eracks.com/products/rackmount-nas-servers, or ask us to spec one for your workload, a human answers.
joe July 9th, 2026
Tags: CMR, drive shortage, IronWolf Pro, IT-mode HBA, NAS, RAIDZ2, supply chain, transparent pricing, TrueNAS, ZFS
The Gray Market Got Smarter. Our Answer Hasn’t Changed: New Parts Only.
In 2015 we published a page called Components We Use, prompted by the broker emails that arrive in our inbox four or five times a week: “clean pull” hard drives by the pallet, bank wire only, 90-day warranty if you’re lucky. The message of that page was one sentence: every new eRacks system is built from 100% new, factory-fresh components, full stop.
A decade later the emails still come, but the merchandise has evolved. This week’s version offered current-generation AMD EPYC server processors, “brand new”, at 50 to 70 percent below any plausible new-part price. So we rewrote the page for 2026, and the new material is worth a blog post of its own, because knowing this trick can save you real money and real pain.
The vendor-locked CPU trick
Modern AMD EPYC processors include a security feature called Platform Secure Boot (PSB for short). When a large vendor like Dell, or a cloud data center, first powers a chip in one of their servers, PSB burns one-time fuses inside the processor that bind it permanently to that vendor’s firmware. That chip will now boot only on that vendor’s motherboards. Forever. There is no unlock.
When those servers are decommissioned, the pulled CPUs flow into the gray market, where brokers list them as “brand new” at half price. Buy one for a custom build and you own a very expensive paperweight: it will never POST (pass the power-on self-test) on a Supermicro, ASUS, or any other standard board. The discount is the tell. A price that looks impossible through authorized distribution usually is.
What we do instead
The refreshed page spells out our sourcing practice in full, and it is the same one we have followed since 1999:
- Authorized US distribution only, so every manufacturer warranty passes through to you intact.
- Multi-source price and availability verification before we quote or set a default. When a part goes supply-constrained, as several high-capacity drives have in the current shortage, we mark it subject to availability or move it to quote-only instead of pretending it ships tomorrow.
- Quality brands, qualified by us: Supermicro and ASUS boards, Intel Xeon and AMD EPYC/Ryzen/Threadripper processors, Seagate, WD, and Toshiba drives (CMR only in our NAS lines), Samsung enterprise SSDs, Intel Arc and NVIDIA GPUs, Broadcom/LSI controllers.
- 72-hour burn-in under full load on every system before it ships. A component that fails burn-in gets replaced, and the clock starts over.
None of this is new policy. What is new is the page saying it plainly for 2026, with the vendor-lock mechanics explained, so that when one of those emails lands in YOUR inbox, you know exactly what is being sold.
Read the full page, including the vintage 2015 broker email we preserved for posterity: eracks.com/components-we-use
CPU photographs by smial via Wikimedia Commons, Free Art License.
joe July 6th, 2026
Posted In: Behind the Scenes, News
Tags: AMD PSB, burn-in, components, gray market, hardware, sourcing, transparency, vendor-locked CPU, warranty
Private AI Servers, Restructured: The New eRacks Good, Better, Best Ladder
We have restructured the eRacks private AI server line into a clear Good, Better, Best ladder. The goal is simple: whatever size model you want to run, there is one obvious system for it, at a price that reflects what is actually inside the box. Prices on the mid-tier dropped substantially from the old line, because the new systems use current-generation parts and are sized honestly for the work.
First, the framing. Private AI means running large language models (LLMs, the software behind AI assistants) on hardware you own, inside your own building. Your prompts, your documents, and your model outputs never touch a cloud provider. You buy the system once, and there are no per-seat or per-token fees afterward. Every system in the line is air-gap ready (able to operate with no internet connection at all), which matters for legal, medical, financial, and government work.
Good: AILSA 2U, from $5,995
The AILSA is the entry point. It is a 2U system (a U is 1.75 inches of rack height) assembled, burned in, and certified by eRacks, with Intel Arc GPUs (graphics processing units, the chips that run AI models). The base build carries two Arc B50 low-profile cards for 32GB of VRAM (the GPU’s onboard memory, which holds the model), with larger Arc Pro options available in the configurator. It handles 70B-class models (roughly 70 billion parameters), which covers most private chat, coding, and RAG workloads (retrieval-augmented generation, where the model answers from your own documents).
Better: AISLING 4U, from $16,995
The new AISLING is the workhorse of the line. It pairs a 24-core AMD Threadripper 9960X with 128GB of ECC memory (error-correcting code memory, which detects and fixes memory errors) and a single 1600W power supply. The 4U chassis takes up to three dual-slot GPUs, which means 96GB of total VRAM with Intel Arc Pro B70 cards. That is enough headroom to run a 70B-class model at higher precision, serve more simultaneous users, or hold longer context windows.
Best: AILEEN 4U, from $21,995
The new AILEEN steps up to server-class silicon: a 32-core AMD EPYC 9355 with 12 memory channels and 192GB of ECC memory. The extra memory bandwidth feeds the GPUs and speeds up CPU-side work like document indexing. It takes up to four GPUs for 128GB of VRAM, and it has redundant 1+1 power (two supplies, either one can run the system alone). AILEEN also ships in custom colors: blue, black, white, or red. The blue unit is the one pictured here.
| Model | Form factor | GPU memory (VRAM) | From |
|---|---|---|---|
| AILSA | 2U | 32GB base, larger Arc Pro options | $5,995 |
| AISLING | 4U | up to 96GB (3 x Arc Pro B70) | $16,995 |
| AILEEN | 4U | up to 128GB (4 GPUs) | $21,995 |
The enterprise tier
Some buyers need validated OEM server systems rather than our eRacks-Certified workhorse builds, usually because their ops teams require out-of-band management (a dedicated channel for remote hardware control, such as IPMI, that works even when the operating system is down). For them, the AIDAN 2U EPYC starts at $13,895, and the AISHA 4U starts at $30,995 with support for up to 10 GPUs.
What every system includes
All five models ship with Ubuntu LTS and the open-source AI stack pre-installed: Ollama, Open WebUI, vLLM, llama.cpp, and PyTorch. Each unit is burned in (run under sustained load before shipping) and tested. You get browser access to your own models on day one: unbox, rack, log in.
Not sure which tier fits? Start with our private AI sizing guide, which walks through how much GPU memory and system RAM a given model actually needs. Then configure the system that matches.
joe July 5th, 2026
Posted In: AI Servers, News
Tags: AI server, AMD EPYC, AMD Threadripper, GPU server, Intel Arc Pro, Ollama, open source AI, private AI, self-hosted LLM, vLLM