Budget Multi-GPU Builds for Local AI: Dual 3090s and Beyond

Quick answer: The cheapest path to serious local-AI VRAM is not one new flagship card — it is two used ones. A single RTX 5090 costs $1,999 MSRP for 32 GB; two used RTX 3090s pool 48 GB for a fraction of that, and they remain the best used-card pairing for VRAM per dollar in Compare AI Hardware's tracked data. Here is when a second card actually pays off, what a dual-3090 budget build fits, and where the pitfalls are.

Why multi-GPU at all?

A second GPU helps in exactly two situations: the model does not fit on one card, or your jobs are independent and can run one per card. It does not speed up a single interactive chat whose model already fits — splitting one model across cards adds per-token coordination that a single bigger card never pays. The full decision framework is laid out in the multi-GPU guide on Compare AI Hardware. The short version: go multi-GPU for 48 GB+ of combined VRAM or for training and batch throughput; stay single-GPU for simplicity and latency. For calibration, the guide's single-card speed references (llama.cpp, Llama-3-8B Q4) record 150 tokens per second on the RTX 3090, 220 on the RTX 4090, and 320 on the RTX 5090 — doubling cards doubles capacity and parallel throughput, not those single-stream numbers.

Dual used 3090s: what fits

Two RTX 3090s give you 24 GB per card and 48 GB combined, with 936 GB/s of memory bandwidth each per the manufacturer datasheet. That 48 GB is enough for 70B-class models at 4-bit quantization — the same pooled capacity as the guide's $6,500 dual-RTX-4090 reference build, reached through the used market instead. The 3090 is also the last GeForce card with NVLink bridge support, though modern multi-GPU AI is SLI-free: the cards cooperate through software, and pipeline parallelism over plain PCIe is the practical default. In llama.cpp, -sm layer splits layers across both cards and -ts tunes the ratio; Ollama distributes across detected GPUs automatically; for image generation you run one ComfyUI or WebUI instance per GPU. The tracked configurations for context:

Configuration	Combined VRAM	Interconnect	Tracked cost
Single RTX 5090	32 GB	—	$1,999 MSRP
Dual used RTX 3090	48 GB	PCIe (NVLink optional)	used-market prices
DIY dual RTX 4090	48 GB	PCIe peer-to-peer	$6,500 (Threadripper 7980X, 1600 W PSU)
System76 dual RTX 6000 Ada	96 GB	NVLink bridges	$18,000

Before you buy two cards, run the inspection checklist in our used RTX 3090 buyer's guide — a mismatched or thermally abused pair undermines the whole value proposition. And to see what that pooled 48 GB means for a specific model, our Llama 3.3 70B hardware guide works through the full VRAM math that makes the dual-card route necessary in the first place.

What about 3-4 card stacks?

They exist mostly in the cloud. The guide's rental data records four RTX 4090s at $1.20 per hour on Vast.ai and eight RTX 3090s at $1.80 per hour — snapshots that fluctuate with supply, so re-check before budgeting a long run. At home, the binding constraint past two cards is PCIe lanes: consumer platforms have limited lane budgets, so populating three or four slots often narrows one or more links, which hurts model loading and tensor-parallel traffic (layer-split inference and data-parallel training tolerate it far better). The tracked multi-GPU builds use Threadripper-class CPUs precisely because they feed multiple slots at full width. The guide's rule of thumb: if your multi-GPU need is measured in a few hundred hours, rent — it avoids the hardware outlay, the power draw, and the cooling engineering, and rented nodes arrive pre-configured.

Risks and pitfalls

The big one: two GPUs do not merge into one VRAM pool. Each card keeps its own memory, total capacity is bounded by the sum, and any single layer must fit on one card — layer-split inference just makes the pair behave like one big pool. Second, size the power supply by summing both cards' TDP plus the rest of the system: the tracked dual-4090 build carries 900 W of GPU TDP and uses a 1600 W supply against an estimated 1200 W peak; two 350 W (datasheet) RTX 3090s are lighter but still demand headroom. Third, cooling is part of the build — adjacent high-TDP cards heat each other, so leave an empty slot where the board allows, keep strong front-to-back airflow, and watch both cards under sustained load; if the top card throttles, the fix is spacing or airflow, not software. Finally, verify both cards appear with nvidia-smi at full negotiated link width, then measure tokens per second and time-to-first-token against your single-card baseline so you know what the second card actually bought.

Frequently asked questions

Does SLI help AI workloads?

No. SLI is a gaming frame-rendering technology that AI frameworks ignore. Multi-GPU AI uses software parallelism — layer splits, tensor parallelism, or data parallelism — over PCIe or NVLink.

Do two GPUs double LLM inference speed?

No. For a single request, splitting one model adds per-token communication. The wins are capacity (larger models fit) and throughput (parallel requests or jobs), not single-stream latency.

Can you mix different GPU models in one build?

You can, but matched cards are simpler: equal VRAM keeps layer splits balanced and identical architectures avoid driver quirks. With unequal cards, llama.cpp's tensor-split ratio option compensates.

What power supply do two GPUs need?

Sum both cards' TDP plus roughly the rest of the system. The tracked dual RTX 4090 build (900 W of GPU TDP) uses a 1600 W supply with an estimated 1200 W peak — match your own cards' TDP to that pattern.

Sources and further reading

• Multi-GPU guide on Compare AI Hardware (linked above) — tracked workstation configurations (DIY dual 4090: Threadripper 7980X, 128 GB DDR5 ECC, 1600 W PSU, ~1200 W peak; ArsenalPC dual RTX 5090 $11,999, 64 GB; System76 dual RTX 6000 Ada $18,000, 96 GB NVLink; HP Z8 Fury), updated August 14, 2026
• Single-GPU speed references recorded on the guide: llama.cpp, Llama-3-8B Q4 — RTX 3090 150 tok/s, RTX 4090 220, RTX 5090 320, per Puget Systems, Tom's Hardware, and TechPowerUp measurements
• Cloud rental rates recorded on the guide (snapshot): Vast.ai 4× RTX 4090 $1.20/hr and 8× RTX 3090 $1.80/hr; RunPod single RTX 4090 $0.34/hr

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

What GPU Do You Need For FLUX.1? Image Generation Requirements

Quick answer: FLUX.1 dev needs about 10 GB of VRAM at Q4/NF4 quantization for 1024×1024 image generation, so a 12 GB GPU such as the GeForce RTX 3060 12GB or RTX 4070 runs it. The FP16 checkpoint needs about 28.4 GB, which requires a 32 GB GPU, and FP8 checkpoints need about 17 GB.

FLUX.1 dev is Black Forest Labs' 12-billion-parameter image model, and unlike its closed cousins it runs locally. The practical question is not "can my GPU load it" — that depends entirely on which checkpoint format you pick. A 12 GB card and a 32 GB card can both generate images, just with very different quality and speed ceilings. Here is the verified memory math.

How much VRAM does FLUX.1 dev need?

The published figures, computed from the model's verified architecture: FLUX.1 dev is a 12.00B-parameter rectified-flow transformer (August 2024). At the popular Q4_K_M quantization the total requirement is 9.9 GB, and the fit page's method is transparent — weights = parameters × bits-per-weight, plus 10% loading overhead, plus a 2 GB activation overhead for 1024×1024 generation (latent buffers, VAE, text encoders partially offloadable). Because this is a diffusion transformer there is no KV cache, which is why the same 9.9 GB holds at long contexts. The live GPU fit lists, grouped by headroom, are maintained on the FLUX.1 dev fit page — that is where to check before buying hardware.

Which quantizations fit smaller cards?

The quantization ladder is the whole story for a 12 GB class card, because the jumps are large:

Variant / quant	Total VRAM @1024×1024	Example GPU that fits
NF4 / GGUF Q4_K_M	9.9 GB	RTX 3060 12GB (minimum tier)
Q6_K	12.9 GB	RTX 4060 Ti 16GB
Q8_0	16.0 GB	RTX 4080 16GB
FP8	~17 GB	RTX 3090 / 4090 24GB (comfortable)
FP16	28.4 GB	RTX 5090 32GB; A100 40GB

Practical reading: Q4_K_M/NF4 is the entry point for 12 GB cards at full 1024×1024 resolution, and the fit page classifies those as the minimum tier with little headroom. Q6_K and Q8_0 move you to 16 GB cards, and FP16 is datacenter or 32 GB territory. FP8 sits in the middle: about 15 GB for weights alone and about 17 GB with activation overhead, which makes 16 GB the practical FP8 floor with encoder offload and 24 GB comfortable.

What speed can you expect?

Speed is best described qualitatively, because the fit page publishes capacity math, not benchmark timings. Generation time on FLUX.1 dev is bound by memory bandwidth rather than raw compute: the page notes that bandwidth, measured in GB/s, matters as much as capacity — and flags 16 GB cards at FP8 as the "fast, high-quality" tier, while minimum-tier 12 GB cards fit the model but leave little headroom. The honest expectation is that a 12 GB card at Q4 generates, a 16 GB card at Q8_0 is meaningfully faster, and a 24 GB card at FP8 is the fastest consumer configuration — but actual seconds per image depend on the pipeline, resolution, and step count, and numbers should not be quoted without measuring on your own card.

Best value GPUs for FLUX right now

Value splits cleanly along two lines. If you already own a 12 GB card (RTX 3060 12GB, RTX 4070), the fit page's answer is: it runs at Q4/NF4 — no upgrade required. If you are buying, 16 GB is the sweet spot: the fit page's comfortable tier (25%+ headroom) starts there, with the RTX 4060 Ti 16GB and RTX 4080 both listed, and 16 GB also unlocks FP8 with encoder offload. The 24 GB tier (RTX 3090, RTX 4090) is where FP8 becomes truly comfortable and where the fit page stops flagging headroom concerns entirely — which is exactly why the used used RTX 3090 remains one of the best budget cards for local AI in 2026. For perspective on how the same cards behave on LLM workloads, our Qwen 3 32B local GPU guide covers the VRAM math for a text model of similar weight.

Frequently asked questions

Can an 8 GB GPU run FLUX.1 dev?

Not comfortably. Q4 weights plus activation overhead need about 10 GB, so an 8 GB GPU must offload the text encoders to system RAM and use GGUF Q3/Q4 with reduced resolution — which degrades quality and adds minutes per image. The fit page's own FAQ says this explicitly.

Is 12 GB enough for FLUX.1 dev?

Yes, at Q4/NF4 quantization: 9.9 GB total at 1024×1024, and the fit page lists 12 GB cards (RTX 3060 12GB, RTX 4070) as the minimum tier — the model fits, but with little headroom for larger resolutions or heavier pipelines.

What does FP8 need?

About 15 GB for FP8 weights alone and about 17 GB with activation overhead at 1024×1024. A 16 GB GPU is the practical FP8 floor with encoder offload; a 24 GB GPU runs FP8 comfortably.

Can a Mac run FLUX.1 dev?

Yes. The fit page confirms Apple Silicon machines fit FLUX.1 dev at Q4_K_M, since macOS lets the GPU use about 75% of unified memory — and notes generation speed is bound by memory bandwidth, so the GB/s figure matters as much as capacity.

Sources and further reading

• FLUX.1 dev fit page on Compare AI Hardware (linked above) — quantization table computed from parameters × bits-per-weight plus 10% loading overhead plus 2 GB activation overhead for 1024×1024; source architecture: 12.00B-parameter rectified-flow transformer, HF model card (black-forest-labs/FLUX.1-dev); FP8/NF4/GGUF variants assumed per checkpoint format
• GPU fit lists computed live from CompareAIHardware's GPU database; Mac figures assume macOS exposes ~75% of unified memory

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Can You Run Llama 3.3 70B Locally? The Real Hardware Requirements

Quick answer: Llama 3.3 70B needs about 47.9 GB of VRAM at Q4_K_M with a 4k context — a 48 GB workstation card is the minimum, and an 80 GB H100-class GPU is the comfortable recommendation. No 24 GB consumer card fits it; pairs of 32 GB cards or a Mac with 64 GB unified memory are the consumer routes.

Llama 3.3 70B is Meta's most-downloaded open model class, and the first question every builder asks is whether their hardware can hold it. The answer is more forgiving than "70B" sounds — if you pick the right quantization — and harsher than most expect if you own a 24 GB card. Here is the full memory math, verified against the model's published architecture.

How much VRAM does Llama 3.3 70B need?

The verified numbers: Llama 3.3 70B is a 70.60B-parameter dense model (December 2024) with a 131,072-token context, and its architecture is identical to Llama 3.1 70B — so every figure here applies to both. At the popular Q4_K_M quantization, weights alone are 42.4 GB and total VRAM lands at 47.9 GB with a 4k-token context; push context to 32k and the requirement grows to 57.3 GB. The method behind these numbers is transparent: parameters × bits-per-weight, plus 10% loading overhead, plus KV cache computed from the verified config (80 layers, 8 KV heads, 128 head dim). The live fit lists — grouped by headroom — are maintained on the Llama 3.3 70B fit page, which is where to check before buying.

Which quantization should you use?

The quantization ladder decides everything, because the jumps are large at this parameter count:

Quantization	Weights only	Total @4k ctx	Total @32k ctx	Example hardware that fits
Q4_K_M (4.8 bpw)	42.4 GB	47.9 GB	57.3 GB	RTX A6000 / RTX 6000 Ada 48 GB (minimum); 2× RTX 5090
Q5_K_M (5.7 bpw)	50.3 GB	56.7 GB	66.1 GB	H100 / A100 80 GB (recommended tier)
Q6_K (6.6 bpw)	58.3 GB	65.4 GB	74.8 GB	H100 80 GB; 2× RTX PRO 6000 Blackwell 96 GB
Q8_0 (8.5 bpw)	75.0 GB	83.9 GB	93.3 GB	H100 80 GB (tight at 32k); H200 141 GB
FP16 (16 bpw)	141.2 GB	156.7 GB	166.1 GB	H200 141 GB (tight); AMD Instinct MI300X 192 GB

Practical reading: Q4_K_M is the entry point and the quality most casual users expect; Q5_K_M–Q6_K is where the fit page points for serious use; Q8_0 and FP16 are datacenter territory. Every figure above is from the fit page's verified architecture math — nothing here is estimated.

What GPUs actually fit it?

The single-card minimum list is entirely 48 GB professional silicon: RTX A6000, RTX 6000 Ada, AMD Radeon Pro W7900, and RTX PRO 5000 Blackwell — all classified as minimum fits with little headroom, so keep contexts short. Comfort begins at 80 GB (H100, A100) and the RTX PRO 6000 Blackwell at 96 GB. The notable absence: no 24 GB consumer card appears anywhere on the fit lists, because 47.9 GB is nearly double what a used RTX 3090 or RTX 4090 holds. That is the hard truth for the most common budget build — and it is why the dual-GPU and Mac routes below exist.

Is a Mac or dual-GPU setup better?

Both work; they optimize for different things. The dual-GPU consumer path is a pair of RTX 5090s (32 GB each, 64 GB pooled) — the fit page assumes roughly 90% efficiency for tensor- or pipeline-parallel pairs, which covers the 47.9 GB requirement with room for context. Pro pairs like 2× RTX PRO 4500 Blackwell also qualify. The Mac path trades peak speed for simplicity: macOS exposes about 75% of unified memory to the GPU, so an M4 Pro with 64 GB (48 GB usable) just clears Q4_K_M, while M3 Max and M4 Max machines with 128 GB expose 96 GB comfortably. Bandwidth is the honest caveat — 273 GB/s on the M4 Pro versus 1,792 GB/s on a 5090 — and generation speed is bound by memory bandwidth, so the Mac wins on simplicity and loses on tokens per second. If your heart is set on the used market anyway, note that a used 24 GB card still runs the smaller sibling beautifully: our Qwen 3 32B hardware guide maps that model onto exactly these cards. And if you buy used hardware for a dual-GPU build, our used RTX 3090 inspection checklist covers the checks that matter before money changes hands.

Frequently asked questions

Can a used RTX 3090 run Llama 3.3 70B?

No, not solo — its 24 GB is roughly half the 47.9 GB Q4_K_M requirement, and the fit page lists no 24 GB card in any tier. Two 24 GB cards also fall short of the pooled requirement at ~90% efficiency. It remains a great card for 32B-class models.

Is Llama 3.3 70B different from Llama 3.1 70B in hardware terms?

No. Per the fit page, 3.3 uses the same 70.6B architecture as 3.1, so every VRAM figure and GPU-fit result applies to both — the difference is training data and tuning, not memory math.

How much does context length change the requirement?

Significantly: Q4_K_M grows from 47.9 GB at 4k tokens to 57.3 GB at 32k — about 9.4 GB of KV cache from 80 layers and 8 KV heads. Minimum-fit cards should keep contexts short.

What is the cheapest realistic build?

A used 48 GB workstation card (RTX A6000 or RTX 6000 Ada class) at Q4_K_M with short contexts, or a 64 GB M4 Pro Mac for simplicity over speed. Both are minimum fits — comfortable headroom starts at 80 GB.

Sources and further reading

• Llama 3.3 70B fit page on Compare AI Hardware (linked above) — quantization table computed from parameters × bits-per-weight plus 10% loading overhead plus KV cache from the verified HF config (unsloth/Llama-3.3-70B-Instruct mirror; 80 layers, 8 KV heads, head dim 128, 131,072 context; identical architecture to Llama 3.1 70B per Meta model card)
• GPU fit lists computed live from CompareAIHardware's GPU database; Mac figures assume macOS exposes ~75% of unified memory; paired-GPU figures assume ~90% efficiency

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Cloud vs Local LLM: The Break-Even Math Nobody Shows You

Quick answer: For an 8B model at Q4, local electricity costs roughly $0.05 per million output tokens — against about $0.53 per million blended on a cheap API. Break-even on a $499 GPU lands near one billion tokens: casual users never get there; heavy daily users pass it in months.

Every "buy a GPU and save on API bills" thread skips the actual arithmetic: what a token really costs once you own the hardware, and how many tokens it takes to repay the card. This post works the math end to end with every assumption stated — so you can swap in your own numbers and see your own break-even point.

What does local inference actually cost per million tokens?

Worked example, assumptions first: an RTX 4060 Ti 16GB ($499 launch MSRP per CompareAIHardware's GPU database) running an 8B model at Q4 quantization at about 95 tokens per second (TechPowerUp measurement in CompareAIHardware's benchmark database), drawing roughly 70% of its 160 W board power under sustained inference — about 112 W, an assumption — at $0.15 per kWh, an assumed rate; your bill differs.

The arithmetic: one million output tokens at 95 tok/s takes 1,000,000 ÷ 95 ≈ 10,526 seconds ≈ 2.93 hours. Energy: 2.93 h × 0.112 kW ≈ 0.33 kWh, which costs 0.33 × $0.15 ≈ $0.05 per million output tokens. In a $0.30/kWh market (typical of Germany), the same tokens cost about $0.10. That is electricity only — the $499 hardware is amortized separately, and that amortization is where the break-even lives. Note what is absent: any claim that this is a benchmark. It is a worked example, and the utilization assumption is the honest variable the whole calculation swings on.

What do API providers charge?

Per DeepSeek's published pricing page (checked August 2026), deepseek-v4-flash bills $0.22 per 1M input tokens on cache miss and $0.66 per 1M output tokens at off-peak rates, with peak hours priced at exactly double; cached input drops to $0.007 per 1M. Premium closed APIs cost multiples of that — we deliberately cite no specific figure for them, since tiers change and this post's job is the method, not the price list. For a blended worked example, assume a 30% input / 70% output mix at off-peak: (0.3 × $0.22) + (0.7 × $0.66) ≈ $0.53 per 1M tokens. Your workload's mix moves this number; long-context workloads with heavy cached input move it a lot.

Where is the break-even point?

CompareAIHardware's cloud vs local AI cost comparison deliberately publishes no break-even multiplier — electricity rates, utilization, and street prices differ per user, so the page records the facts (rental rates, MSRPs, board power) and leaves the multiplication to your usage profile. Respect that framing: what follows is a worked example, not a verdict. Savings per 1M tokens: $0.53 − $0.05 ≈ $0.48. Hardware to repay: $499. Break-even: 499 ÷ 0.48 ≈ 1.04 billion tokens. Now the utilization question that decides everything:

Monthly volume (worked example)	API cost @ $0.53/1M	Local electricity @ $0.05/1M	Months to repay $499 GPU
10M tokens	$5.28	$0.49	~104 (never, realistically)
100M tokens	$52.80	$4.90	~10.4
500M tokens	$264.00	$24.50	~2.1

There is a third path the per-token framing hides: renting GPUs by the hour. CompareAIHardware's cloud instance database records a RunPod RTX 3090 at $0.22 per hour on-demand ($0.10 spot) and a single RTX 4090 at $0.34 — recorded rates for bursty workloads that need real capacity occasionally without owning anything.

When does cloud still win?

The comparison guide's list is blunt, and the break-even math backs it. Cloud wins for sporadic use (hourly billing, no idle hardware), bursty team demand (rent an eight-GPU node for a day, then release it), and frontier-model access — hosted endpoints serve models no consumer GPU can hold, and the gap is widening: our guide to what DeepSeek V4 Flash needs to run locally shows frontier-class memory requirements climbing past single consumer cards. Cloud also wins on zero maintenance — no drivers, cooling, or failed fans. Local wins on privacy, unlimited usage without meter-watching, and sustained daily workloads where the hardware earns its price every month. If you go local, buy well: our used RTX 3090 inspection guide covers the secondhand route that stretches the break-even math in your favor.

Frequently asked questions

Does the math change for bigger models?

The structure holds; the numbers move. Larger models need more VRAM (frontier-class models exceed single consumer cards entirely), often run slower per token, and can demand costlier hardware — all of which pushes break-even further out unless your volume is genuinely high.

What if my electricity is expensive?

At $0.30/kWh the worked example's per-token cost roughly doubles to about $0.10 per 1M output tokens. That is still far below the API anchor, but it stretches the break-even proportionally — the sensitivity is linear, which is why the assumption must be stated.

Is renting a GPU by the hour a middle path?

Yes. At recorded rates like $0.22 per hour for an RTX 3090, a bursty workload that needs real capacity a few days a month rents instead of buys — no maintenance, no idle capital, and spot rates lower still for batch-tolerant jobs.

Why won't the comparison guide just publish a break-even multiplier?

Because one multiplier would be wrong for most readers: electricity rates, utilization, and street prices all vary. The guide publishes recorded rates, MSRPs, and board power — the inputs — and leaves the arithmetic to your usage profile, which is exactly the habit this post applies.

Sources and further reading

• Cloud vs Local AI on Compare AI Hardware (linked above) — cloud instance database rental rates (RunPod, Lambda, Vast.ai, AWS, GCP, Azure), GPU launch MSRPs and board power from its GPU database; updated August 14, 2026
• DeepSeek Models & Pricing page (checked August 2026): deepseek-v4-flash $0.22/1M input (cache miss, off-peak), $0.66/1M output off-peak, peak = 2× off-peak, cached input $0.007/1M — api-docs.deepseek.com/quick_start/pricing
• 8B throughput reference: ~95 tokens/second on Llama-3-8B Q4 (TechPowerUp measurement in CompareAIHardware's benchmark database, via its Best GPU for LLMs guide)

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Whisper Large-v3 on Your Old Gaming PC: What It Really Needs

Quick answer: Whisper large-v3 needs only about 4.4 GB of VRAM at FP16 — roughly 2 GB quantized — so almost any modern GPU runs it, and even a 4 GB card qualifies as minimum. The real question is throughput: casual files run on CPU, but batch archives reward an 8 GB-class card.

Most AI hardware advice assumes you are chasing frontier LLMs. Whisper is the opposite case: the largest speech-to-text model in the family is small by AI standards, and that flips the buying logic entirely. This guide covers what large-v3 actually needs, what an old gaming card delivers, and when buying anything is a mistake.

How much VRAM does Whisper large-v3 need?

Whisper large-v3 is a 1.54-billion-parameter encoder-decoder transformer released in November 2023. Per the fit page's verified architecture data, it needs about 4.4 GB of VRAM at FP16 — 3.1 GB of weights plus roughly 1 GB of runtime overhead for the Mel spectrogram and beam-search buffers — and only 2.0 GB at Q4_K_M quantization. The architecture is the reason it is so light: audio is processed in 30-second windows, so unlike an LLM there is no growing KV cache. The fit page sets the minimum at any 4 GB GPU at FP16 (or CPU-only via whisper.cpp) and recommends an 8 GB card of the RTX 3060/4060 class for faster-than-real-time batch transcription. For contrast, Mixture-of-Experts LLMs live in a different universe of requirements — see our guide to what DeepSeek V4 Flash needs to run locally. The whole Whisper family, for scale:

Model	Parameters	VRAM at typical batch
tiny	39 M	~1 GB (estimated)
base	74 M	~1 GB (estimated)
small	244 M	~1–2 GB (estimated)
medium	769 M	~2–3 GB (estimated)
large-v3	1.54 B	4.4 GB FP16 incl. overhead (verified); 2.0 GB at Q4_K_M

Parameter counts come from OpenAI's Whisper repository; the large-v3 VRAM figures are from the fit page. The smaller models' VRAM entries are estimates scaled from parameter count — none are stated on the source pages.

What can you expect from an older card?

More than you might expect. The buying guide's core observation: Whisper's largest model fits comfortably on mainstream cards from several generations back, so VRAM capacity is rarely the deciding factor it is for LLMs or image generation. Software choice matters more here than hardware age — Faster-Whisper, SYSTRAN's reimplementation on CTranslate2, cuts compute cost substantially versus the original implementation, and the cheaper the card, the more that matters. Community Faster-Whisper setups report comfortable real-time factors on a previous-generation RTX 3060. One honesty note: the guide's benchmark database holds no Whisper-specific timings, so card comparisons use SDXL Turbo throughput as a compute reference — useful for batch planning, not a stopwatch.

Is CPU-only transcription viable?

Yes, with a throughput trade-off. The fit page calls CPU-only inference practical at roughly real-time speed with whisper.cpp, the heavily optimized C++ port; the original implementation is slower, with long recordings taking multiples of the audio duration in practice. That makes CPU fine for occasional files and painful for archives. If you already own an Apple Silicon Mac, try its native speech stack before buying any GPU — the guide's words — since optimized builds run through Apple's own frameworks. Only regular transcription volume justifies shopping for a card at all.

Which GPUs give the best value for transcription?

The guide's budget pick is the RTX 3060: cheap on the used market, enough memory and compute for the largest model, and the classic entry-level transcription card. Buyers wanting new hardware with documented specifications have the Arc B580 at a $249 MSRP with 12 GB, or the RTX 4060 Ti 16GB at $499 — whose 160 W TDP drops into most desktops without a PSU upgrade — and both exceed Whisper's requirements by a wide margin. The mid-range RTX 4070 Super suits batch workers processing podcast and meeting archives, while the RTX 5090 is only for buyers parallelizing huge audio libraries; its $1,999 price and 575 W draw buy capacity most Whisper users never touch. The guide's closing logic inverts usual GPU advice: the cheapest tier that covers your batch volume is the correct one. Full tier-by-tier detail is in the guide to the best GPUs for Whisper transcription. And if you want one card that also pulls LLM duty, a used 24 GB card is the flexible pick — our used RTX 3090 buying checklist covers what to inspect before paying.

Frequently asked questions

Can an old GTX 1070-class card run Whisper large-v3?

Yes, qualitatively: the fit page's minimum is any 4 GB GPU at FP16, and 8 GB Pascal-era cards clear that bar with room to spare. Expect slower batch throughput than modern cards, and run Faster-Whisper — optimized implementations benefit older hardware most.

Should I just use the small model instead of large-v3?

The family scales from 39 M to 1.54 B parameters, and smaller models trade accuracy for speed and memory. For clear audio and casual use, small runs on nearly anything; for demanding transcription, large-v3 still fits modest hardware, so the accuracy trade is often unnecessary.

Why does Whisper need so little VRAM compared to LLMs?

Two reasons: 1.54 billion parameters is tiny next to 32B-plus LLMs, and the 30-second audio window means no growing KV cache — memory use stays flat regardless of recording length.

Is a flagship GPU ever worth it for transcription?

Only for batch archives. Running many parallel workers turns transcription into a queue problem, and throughput drains the queue — but even 16 GB cards host multiple workers with room to spare, so flagships mostly buy unused capacity.

Sources and further reading

• Whisper large-v3 fit page on Compare AI Hardware (data cited above) — VRAM by quantization from the verified HF config (openai/whisper-large-v3: d_model 1280, 32 encoder + 32 decoder layers, 128 Mel bins)
• Best GPU for Whisper & Speech-to-Text AI in 2026 on Compare AI Hardware (linked above) — GPU picks, MSRP pricing, and the SDXL Turbo throughput reference table (Tom's Hardware and TechPowerUp measurements, June 2025)
• OpenAI Whisper repository — parameter counts per model size (tiny 39M, base 74M, small 244M, medium 769M, large 1550M): github.com/openai/whisper

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Qwen 3 32B Locally: What GPU Do You Actually Need?

Quick answer: Qwen 3 32B needs about 22.7 GB of VRAM at Q4_K_M with a 4k-token context — a used 24 GB card like the RTX 3090 is the minimum fit, and longer contexts push that past 30 GB. A 16 GB card only works in a pair; single-card comfort starts at 32 GB.

Qwen 3 32B sits in the sweet spot of local LLMs: big enough to feel like a frontier-adjacent model, small enough to fit on hardware you can actually buy. This guide walks through the real VRAM math at each quantization level, which cards fit and which merely scrape by, and what laptops and NPUs can honestly deliver.

What is Qwen 3 32B?

Qwen 3 32B is a dense 32.8-billion-parameter model from Alibaba's Qwen family, released in April 2025 with a native 40,960-token context window. Dense is the operative word: unlike Mixture-of-Experts designs — see our guide to what DeepSeek V4 Flash needs to run locally for that contrast — every parameter activates on every token, so the full 32.8B weights must sit in fast memory. The fit page's architecture source is the verified Hugging Face config: 64 layers, 8 KV heads, 128 head dimension. That config is what makes the memory numbers below reproducible math rather than folklore.

How much VRAM does it need?

The calculation has three parts: weights (parameters × bits-per-weight), a 10% loading overhead, and the KV cache computed from the real architecture. At the popular Q4_K_M quantization (4.8 bits per weight), weights alone are 19.7 GB and total VRAM lands at 22.7 GB with a 4k-token context. Push context to 32k and the KV cache grows the requirement to 30.2 GB. The full ladder, per the fit page:

Quantization	Weights only	Total @4k ctx	Total @32k ctx	Example GPUs that fit
Q4_K_M (4.8 bpw)	19.7 GB	22.7 GB	30.2 GB	Used RTX 3090 24 GB (minimum, short contexts); RTX 4090 24 GB
Q6_K (6.6 bpw)	27.1 GB	30.8 GB	38.4 GB	RTX 5090 32 GB (comfortable); 2× RTX 4060 Ti 16GB pooled
Q8_0 (8.5 bpw)	34.9 GB	39.4 GB	46.9 GB	RTX 6000 Ada or RTX A6000 48 GB; 2× 24 GB cards
FP16 (16 bpw)	65.6 GB	73.2 GB	80.8 GB	H100 80 GB; RTX PRO 6000 Blackwell 96 GB

The live GPU fit lists — grouped by headroom, from "runs comfortably" to "needs two GPUs" — are computed from the site's GPU database on the Qwen 3 32B fit page, which is the destination to check before you buy anything.

Which used-market card is the sweet spot?

For a single-card build, the used RTX 3090 is the entry point: its 24 GB holds the 22.7 GB Q4_K_M footprint, but the fit page classifies it as a minimum fit — little headroom, so keep contexts short. The RTX 4090 offers the same 24 GB with more speed at a used premium. Two 16 GB cards in parallel also cover the requirement (the fit page assumes roughly 90% efficiency for paired cards), and a used RTX 4060 Ti 16GB pair is the cheap way there — but each card's 288 GB/s bandwidth keeps token speed modest. If you go the used route, read our guide to inspecting a used RTX 3090 before buying first. Prefer new? CompareAIHardware's best-GPUs-under-$500 guide ranks the RX 7600 XT 16 GB ($329), RTX 4060 Ti 16 GB ($499), and Arc B580 ($249) by MSRP — all below the single-card Q4_K_M bar, which is exactly why the used 24 GB market stays hot.

What about laptops and NPUs?

Apple Silicon is the credible laptop path: macOS lets the GPU use about 75% of unified memory, so a 32 GB M4 machine exposes roughly 24 GB — just enough for Q4_K_M at short contexts. A 64 GB M4 Pro exposes about 48 GB comfortably, and bandwidth rises from 120 GB/s on the base M4 to 273 GB/s on the M4 Pro, 400–546 GB/s on Max chips, and 819 GB/s on the M3 Ultra. Since generation speed is bound by memory bandwidth, the GB/s column matters as much as capacity. Windows NPUs and edge devices mostly cannot play: devices like the Jetson AGX Orin (32–64 GB, 204 GB/s) or Snapdragon X Elite (64 GB, 135 GB/s) have enough memory for Q4_K_M, but the fit page warns to expect a fraction of desktop generation speed — treat them as embedded deployments, not daily drivers.

Frequently asked questions

Can a used RTX 3090 run Qwen 3 32B?

Yes — it is a classified minimum fit. The 22.7 GB Q4_K_M footprint at 4k context squeezes into 24 GB with little headroom; at 32k context the 30.2 GB requirement exceeds a single 24 GB card.

Can two 16 GB cards run it?

Yes. Paired cards cover the 22.7 GB requirement at roughly 90% efficiency — a used RTX 4060 Ti 16GB pair is the budget route — but 288 GB/s per card keeps token generation slow.

Does Qwen 3 32B fit on a Mac?

Yes, at Q4_K_M, if usable unified memory clears about 23 GB — a 32 GB M4 machine's roughly 24 GB usable just qualifies. More memory and bandwidth (M4 Pro, Max, Ultra) buy headroom and speed.

Why does context length change VRAM so much?

The KV cache scales with context: with 64 layers and 8 KV heads, going from 4k to 32k tokens adds about 7.5 GB at Q4_K_M (22.7 to 30.2 GB, per the fit page's math).

Sources and further reading

• Qwen 3 32B fit page on Compare AI Hardware (linked above) — quantization table computed from parameters × bits-per-weight plus 10% loading overhead plus KV cache from the verified HF config (Qwen/Qwen3-32B: 64 layers, 8 KV heads, head dim 128, 40,960 native context)
• Best GPU for AI Under $500 on Compare AI Hardware (named above, not linked) — MSRP pricing for RX 7600 XT, RTX 4060 Ti 16 GB, Arc B580; benchmark figures from TechPowerUp data in its database

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Used RTX 3090 in 2026: Still the Best Budget Local AI Card?

Quick answer: Yes, for most budget builders: a used RTX 3090 is still the cheapest route to 24 GB of VRAM with full CUDA support. Capacity decides which models fit, and 24 GB still covers large quantized LLMs. The trade-off is speed — newer cards are faster — and used cards need inspection.

Three GeForce generations later, the RTX 3090 keeps showing up in budget local-AI builds, and the reason has not changed: memory capacity. This guide covers why 24 GB still matters, what breaks on used cards, how to inspect before paying, and how it compares with newer options.

Why does 24 GB of VRAM still win per dollar?

For local AI, VRAM capacity matters first, bandwidth second, compute third — capacity decides which models fit, bandwidth how fast tokens come out. Its 24 GB of GDDR6X across a 384-bit bus is the core asset: large quantized models needing more than 16 GB do not fit mid-range cards, which is why the card stays relevant years after its 2020 launch at a $1,499 MSRP. The card is end-of-life, so nearly all units trade second-hand — exactly where the value sits. It is also the last consumer GeForce card with NVLink: two 3090s pool their 24 GB into a 48 GB pool for models too large for one card. The 936 GB/s of memory bandwidth drives token generation speed, since single-GPU LLM inference is largely a memory-bandwidth problem.

What are the real risks of a used 3090?

The most common fault is degraded thermal pads on the VRAM modules, followed by worn fan bearings and dried thermal paste — all serviceable, and many buyers plan for the maintenance up front. The GDDR6X modules run hot under sustained AI workloads, and the factory pads dry out over years. Typical symptoms: memory temperature climbs during long inference runs, the card throttles, and generation slows or artifacts appear. History matters less than condition: miners often undervolted their cards and kept them in cool rooms; gaming cards see fewer hours but more thermal cycling. No history is disqualifying — buy on load-test evidence, not the story. Red flags that end the deal immediately: burnt smell, visible board damage, a failed memory test, a missing serial-number sticker, or a price far below every comparable listing — that is not a bargain, it is the scam signal. The full inspection checklist lives in CompareAIHardware's used RTX 3090 buying guide.

What should you check before buying?

Three checks catch most problem cards. First, ask the seller what the card ran — gaming, mining, or AI — and whether it was ever opened, repasted, or repaired. Second, request an nvidia-smi screenshot showing the full 24 GB of memory and healthy error counters. Third, test memory under sustained load: about ten minutes watching memory temperatures, then a GPU memory stress test — artifacts, crashes, or errors mean failing VRAM; decline the card. Physically, both fans must spin freely without grinding, with no burnt smell, no PCB discoloration, and no bent display-output pins. Buy through a platform with buyer protection and prefer local pickup; if buying shipped untested, ask for a load-test video plus serial-number photos and keep payment inside the platform's own checkout. Sizing the card against the models you actually want to run? Know what frontier-class open models like Kimi K3 need before you spend.

How does it compare with newer budget cards?

Card	VRAM	Memory bandwidth	Typical price	Local-AI verdict
Used RTX 3090	24 GB GDDR6X	936 GB/s	market-dependent (no used-price estimates published)	Best budget capacity pick; fits models 16 GB cards cannot
RTX 4060 Ti 16GB	16 GB GDDR6	288 GB/s	$499 launch MSRP (new)	Holds mid-size quantized models; bandwidth-limited for token speed
RTX 5060 Ti 16GB	16 GB GDDR7	448 GB/s (vendor spec)	$429 launch MSRP (new, vendor)	Faster and efficient, but the 16 GB ceiling excludes larger quantized models
Used RTX 4090	24 GB GDDR6X	1008 GB/s	market-dependent (no used-price estimates published)	Same capacity, faster on every other spec; carries a premium

The 3090-versus-4090 question comes down to price: both hold 24 GB, and the 4090 is faster everywhere else — the 1008 vs 936 GB/s bandwidth gap suggests only a modest single-GPU LLM inference difference, while image generation favors the 4090's newer architecture more clearly. Whether that speed is worth the premium depends on current used listings. Against the 16 GB cards the decision is simpler: models needing more than 16 GB are excluded regardless of speed. And for frontier-class open weights, 24 GB is where consumer cards end — our DeepSeek V4 Pro vs closed models breakdown shows the other end of the spectrum.

Frequently asked questions

Is a used RTX 3090 good for running local LLMs?

Yes. Its 24 GB of VRAM fits large quantized models that 16 GB cards cannot hold, and CUDA support keeps it compatible with the standard local inference tools.

What power supply does an RTX 3090 need?

Plan comfortable headroom above the card's 350 W total board power rating plus the rest of the system's draw — sustained AI loads hold the card at its power limit for hours.

Can I run two RTX 3090 cards together?

Yes. NVLink lets two cards pool 24 GB each into 48 GB for larger models — the last consumer GeForce generation with that feature.

Should I replace the thermal pads immediately after buying?

If memory temperatures run high under sustained load, yes. Pad replacement is standard maintenance on this card, and many buyers plan for it up front.

Sources and further reading

• Used RTX 3090 Buying Guide on Compare AI Hardware (linked above) — specs from its GPU database; benchmark attribution: Tom's Hardware results tracked for the RTX 4090 (220 tok/s Llama-3-8B Q4, 80 SDXL Turbo images/min), none tracked for the RTX 3090
• RTX 5060 Ti 16GB figures are vendor specifications; used prices are market-dependent (the guide publishes no estimates)

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Can Your PC Run DeepSeek V4 Flash? VRAM Requirements Explained

Quick answer: DeepSeek V4 Flash is a Mixture-of-Experts model: all 284 billion parameters must be stored, so combined RAM plus VRAM — not GPU compute — decides whether it fits. No single consumer GeForce card holds the full model; self-hosting lives on high-unified-memory Macs or multi-GPU professional servers.

DeepSeek V4 Flash is the release that made "just run it at home" a serious question again. The catch is architectural: as a Mixture-of-Experts model, its barrier is storage, not compute. This guide walks through the actual memory math, what fits at each budget, and where the honest limits sit.

What is DeepSeek V4 Flash?

V4 Flash is DeepSeek's compact, efficiency-focused model in the V4 generation, and two design choices define it. First, it is a Mixture-of-Experts model: every expert sits in memory, but only a few activate per token, which keeps inference fast while shifting the bottleneck to storage. Second, deployment maturity: the production checkpoints ship quantization-aware, so the official weights are already compact rather than shrunk as an afterthought, and a hybrid attention design compresses the KV cache — DeepSeek reports roughly a tenth of its predecessor's cache size at very long contexts. For the bigger sibling's standing, see our write-up on how DeepSeek V4 Pro stacks up against closed models.

How much VRAM does it need?

Here is the counterintuitive part: combined RAM plus VRAM matters more than VRAM alone. Because a Mixture-of-Experts model must store every expert, the question is not "does my GPU have enough" but "do my GPU and system RAM together hold the weights." CompareAIHardware's models database lists V4 Flash at 284.00B total parameters with 13.00B active per token, and the DeepSeek V4 Flash fit page draws the practical conclusion: no single consumer GeForce card holds the full model, so self-hosting lives on high-unified-memory machines or multi-GPU servers.

What about quantized versions?

An honest caveat first: DeepSeek has not published official quantized download sizes as a simple gigabyte figure, and real sizing depends on which community build you fetch. The fit page — citing DeepSeek's published self-hosting notes — describes quantized community builds spanning roughly a hundred gigabytes at heavy compression up to around a hundred and seventy for lossless builds; treat those endpoints as order-of-magnitude guides and verify any specific artifact before you commit disk. The VRAM calculator on Compare AI Hardware computes weights plus KV cache from published model configurations and labels anything it cannot verify as architecture-estimated, which is the right tool for a specific case.

Quant level	VRAM/RAM needed	Example hardware
Heavy quantization	order of ~100 GB combined (fit page, citing DeepSeek's self-hosting notes)	Mac Studio M3 Ultra (up to 512 GB unified); 2× RTX 6000 Ada workstation
Mid quantization	between the endpoints (estimated)	Mac Studio 512 GB; multi-GPU RTX 6000 Ada / A6000
Lossless	order of ~170 GB combined (fit page, citing DeepSeek's self-hosting notes)	H200 (141 GB) plus offload; H100 pair; Mac Studio 512 GB
Partial offload	large system RAM + 48 GB GPU	RTX 6000 Ada or RTX A6000 workstation
CPU-only (heavy quant)	high system RAM, no GPU	high-memory desktop; slow token speed

Which hardware fits at each budget?

Budget tier: a high-memory desktop with no discrete GPU can run heavily quantized builds on the processor alone — the small active-parameter count keeps that viable — but token speed will test your patience. Mid tier: a workstation with large system RAM plus one professional GPU (RTX 6000 Ada or RTX A6000, 48 GB each per the fit page's database) holds the weights in RAM and offloads the busiest layers to the card, trading some setup complexity for a large speed gain over processor-only inference. No-compromise tier: the Mac Studio with M3 Ultra (up to 512 GB unified, 819 GB/s) is the simplest single-box option and holds any quantization comfortably, while datacenter nodes — H100 (80 GB HBM3) or H200 (141 GB HBM3e) — run production serving under vLLM, with DSpark speculative decoding for responsive performance. If you want the research context behind these efficiency gains, our piece on DeepSeek's self-improving harness research covers where the field is heading.

Frequently asked questions

Can a single RTX 4090 run DeepSeek V4 Flash?

No. Its 24 GB is far below the footprint of even heavily quantized community builds — order of a hundred gigabytes per the fit page's citation of DeepSeek's self-hosting notes — and the fit page is explicit that no single consumer GeForce card holds the full model. Realistic options are CPU offload (slow), multi-GPU professional workstations, or a high-unified-memory Mac.

Why does combined RAM matter more than VRAM for this model?

Mixture-of-Experts weights must be stored entirely even though only a few experts activate per token. Storage — RAM plus VRAM together — is the barrier, not compute.

Is V4 Flash better than V4 Pro?

For agentic and coding work, DeepSeek's vendor-reported results say yes, at lower API pricing. Independent verification was still pending as of August 2026, per the fit page.

Can I run it without a GPU?

Yes, slowly. A high-memory desktop can run heavily quantized builds on the processor; the small active-parameter count keeps it viable, but token speed will test your patience.

Sources and further reading

• DeepSeek V4 Flash fit page on Compare AI Hardware (linked above) — memory math, hardware tiers, and self-hosting notes compiled from DeepSeek's published documentation
• VRAM calculator on Compare AI Hardware (linked above) — per-model weights plus KV-cache math from published configurations

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Qwen3.8-Flash-Next: The Free AI Catching The Giants

Updated: 2026-08-28

Quick answer: Qwen3.8-Flash-Next is Qwen's open-weights preview of the Qwen4 architecture: a 125B-parameter MoE with 6B active per token, plus a 51B N-gram embedding table that can offload to host memory. Qwen reports training cost around 1/9 of Qwen3.7-Plus with stronger coding results.

Two days ago Qwen opened the weights of Qwen3.8-Flash-Next, and the interesting part is not the benchmark chart — it is the architecture preview. This release exists to let the community examine the design Qwen4 will be built on, the same way Qwen3-Next previewed Qwen3.5. Here is what the official repository actually documents, and what it means for anyone hoping to run it locally.

What is Qwen3.8-Flash-Next?

Qwen3.8-Flash-Next is a multimodal Mixture-of-Experts model with open weights, released on 2026-08-26. The repository describes it as an early preview of the Qwen4 architecture, upgrading four things at once: attention, residual connections, embeddings, and optimization. The headline numbers: a 125B-parameter main model supplemented by 51B N-gram embeddings, with only 6B parameters activated per token. Weights are published on Hugging Face Hub and ModelScope, and the model powers the new "Standard" mode on QwenWork.

Why does the N-gram embedding matter?

Because it is the second major lab bet on lookup-based memory this month. The embedding module looks up a table using the local context to scale model capacity with very little extra computation — and, critically for self-hosters, the table can be offloaded to host memory and overlapped with model computation through asynchronous prefetching. DeepSeek's Engram module pursues the same broad idea from a different angle, which we covered in our explainer of DeepSeek's fix for AI's biggest problem. When two frontier labs ship lookup-based memory in the same month, that is a direction, not a coincidence.

What does it mean for VRAM and local hardware?

Honest answer first: the model is too new for verified VRAM figures, and nothing is measured on the model fit database yet — check the model fit database before planning any build, since that is where verified per-model numbers land. What the official specs do tell us: the storage floor is large, because all 176B parameters (125B main + 51B embeddings) must live somewhere; a rough 4-bit estimate for the main model alone starts near 63 GB (estimated — derived from parameter count, not from a published artifact size). The offsets are real too: only 6B parameters activate per token, which keeps compute light, and the N-gram table is explicitly designed to sit in host memory rather than VRAM. Treat every specific figure here as unconfirmed until the fit page publishes measured numbers.

How does it compare with prior Qwen releases?

Model	Architecture	Parameters	Active / token
Qwen 3 32B (Apr 2025)	Dense	32.80B	all (dense)
Qwen3.7-Plus	hybrid GDN + Gated Attention line	not stated in repo (unconfirmed)	not stated
Qwen3.8-Flash-Next (Aug 2026)	MoE, GDN + QSA hybrid, N-gram embedding	125B main + 51B embeddings	6B

Against Qwen3.7-Plus, the repository claims training cost of only about 1/9 with superior capabilities in coding and office tasks — a vendor claim, pending independent verification. Against the older dense Qwen 3 32B, the contrast is architectural: that model activates every parameter on every token and fits a 24 GB card at Q4_K_M, which our Qwen 3 32B hardware guide covers in detail. Flash-Next inverts the trade: tiny active compute, large stored capacity.

Frequently asked questions

Is Qwen3.8-Flash-Next free to use?

The weights are open and downloadable from Hugging Face Hub and ModelScope; Qwen also serves it through QwenWork's Standard mode and the QwenCloud API. License terms live on the model page.

Can I run it locally today?

Unconfirmed. The architecture is published and the weights are out, but no verified VRAM figures exist yet. The storage floor is large and the N-gram table is designed for host-memory offload — wait for measured fit data before buying anything.

What is QSA?

Qwen Sparse Attention: a compressed lightweight indexer selects the important context at micro-block granularity, substantially reducing the cost of attention on long sequences. It works alongside Gated DeltaNet, which compresses history efficiently.

Why is it called "Flash-Next"?

Naming lineage: it plays the role Qwen3-Next played for Qwen3.5 — architectural changes released early so the community can examine them before the full Qwen4 model family is built on top.

Sources and further reading

• Qwen3.8-Flash-Next official repository and tech report: https://ift.tt/EoCt8aK
• Qwen blog announcement: https://ift.tt/dGzK3T1
• Episode sponsor — Weights & Biases: https://wandb.me/papers

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

DeepSeek's Self-Improving AI Explained (And What It Needs To Run)

Updated: 2026-08-28

Quick answer: DeepSeek researchers demonstrated a harness where a model improves its own reasoning by iterating on tasks and keeping what works. It runs on open-weight models — meaning the technique, and models like it, can run on your own hardware if you have enough VRAM.

DeepSeek's researchers demonstrated a harness that lets a model upgrade itself — iterating on tasks, keeping the strategies that work, and discarding the rest. The technique runs on open-weight models, which makes it reproducible outside a closed lab. Here is what was actually shown, and what hardware you need if you want to run models of this class yourself.

What did DeepSeek actually build?

A harness: scaffolding around a model that automates improvement. Instead of a human curating what the model learns from, the system attempts tasks, evaluates the results, and keeps the reasoning strategies that succeed. The harness and the accompanying paper are linked in the sources below. Two details make this worth attention: the loop is automated end-to-end, and it is demonstrated on open-weight models — nothing in the technique requires a closed, proprietary base. It is also another step in a rapid run of releases; we covered the previous one in DeepSeek's previous open-weight release.

How does self-upgrading work?

At a high level, the harness closes a feedback loop. The model attempts a task, the outcome is scored, and successful reasoning paths are retained for future attempts while failed ones are dropped. Over many iterations the system accumulates strategies that work — improvement by selection rather than by manual tuning. The paper describes the exact mechanics, and the video walkthrough above shows the demonstration. This is the same broad pattern behind DeepSeek's inference speed hack, where architectural cleverness rather than raw scale drove the gains.

Can you run models like this locally?

Yes, with caveats. The harness runs on open-weight models, so reproducing the technique is a software problem, not an access problem. The hard part is the same one that governs every local LLM: memory. Quantized weights plus the KV cache must fit in VRAM or unified memory, and reasoning-heavy workloads generate long chains that inflate that cache. A 70B-class model at 4-bit quantization needs roughly 40 GB before the cache even starts filling; smaller models run on far less but trade away capability.

What hardware does it take?

GPU	VRAM	Bandwidth	Role for local LLMs
GeForce RTX 5090	32 GB GDDR7	1,792 GB/s	Best overall; largest consumer VRAM pool
GeForce RTX 4090	24 GB GDDR6X	1,008 GB/s	Strong all-rounder
GeForce RTX 3090 (used)	24 GB GDDR6X	936 GB/s	Best value pick
Intel Arc B580	12 GB GDDR6	456 GB/s	Cheapest usable new card ($249 MSRP)

The table condenses Compare AI Hardware's ranked picks for local LLM inference, using their GPU database specifications. The RTX 5090's 32 GB frame buffer is the guide's best-overall pick and the memory pool that 70B-class quantized models want; a used RTX 3090 remains the value option at 24 GB. For the full tier-by-tier ranking from 12 GB to 96 GB, see the guide to the best GPUs for running DeepSeek models locally.

Frequently asked questions

What is a self-upgrading AI harness?

Scaffolding around a model that automates improvement: the system attempts tasks, scores outcomes, and retains the reasoning strategies that succeed. The linked paper describes the exact mechanism.

Is the DeepSeek harness open?

The harness and paper are publicly linked in the sources below, and the technique is demonstrated on open-weight models — so it can be studied and reproduced without access to a closed lab.

How much VRAM do I need for models like this?

It depends on model size and quantization. A 70B-class model at 4-bit needs roughly 40 GB or more; 24 GB cards handle smaller models well. The table above maps model classes to cards.

Does self-upgrading rewrite the model's weights?

The demonstrated loop improves reasoning by iterating on tasks and keeping what works; whether and how weights change is described in the paper. The safe summary: improvement by selection, not by hand-tuning.

Where can I read the paper?

Both the harness and the paper are linked in the sources section below.

Sources and further reading

• DeepSeek Harness: https://ift.tt/qBlnL6M
• Accompanying paper: https://ift.tt/fcmt96V
• Episode sponsor — Lambda GPU Cloud: https://ift.tt/XQJqdaP

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

This Small AI Will Change Everything

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/VogMa70 📝 The Qwen3.8-27b is available here: https://ift.tt/LQIMDJU Sources: https://ift.tt/QAaTSNt https://ift.tt/PGzDeWM https://ift.tt/Pg38yCa https://ift.tt/Yqgbr1F https://ift.tt/dpVD1cm 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

DeepSeek V4 Pro vs Closed AI: What Open Weights Now Deliver

Updated: 2026-08-28

Quick answer: DeepSeek's V4 Pro 0813 release closes most of the gap with closed frontier models while staying open-weight. Anyone with sufficient VRAM — or a Mac with large unified memory — can run DeepSeek-class models locally without an API bill. Here is what changed, and what hardware it takes.

Two Minute Papers recently covered DeepSeek's V4 Pro 0813 release, and the headline is blunt: an open-weight model is now trading blows with closed frontier systems. This post unpacks what the release actually changes, how close it really is, and — the practical part — what kind of hardware can run a model of this class at home.

What is DeepSeek V4 Pro 0813?

V4 Pro 0813 is DeepSeek's August snapshot of its V4-generation model family, and two design choices stand out. First, the production checkpoints ship quantization-aware, so the official weights are already compact rather than shrunk as an afterthought. Second, DeepSeek publishes a hybrid attention design that keeps long-context serving tractable. Like its siblings, it is a Mixture-of-Experts model: every expert sits in memory but only a few activate per token, which shifts the bottleneck from compute to storage.

How close is it to closed models?

Benchmark chatter around 0813 focuses on agentic and coding work, where DeepSeek's vendor-reported results put it within striking distance of closed frontier systems. Independent replication is still catching up, so treat specific scores as provisional. The trajectory is not provisional, though: our earlier DeepSeek V4 benchmark breakdown showed the previous snapshot closing gaps that looked permanent months ago. The cost angle matters just as much — we also covered how DeepSeek tackled AI's billion-dollar inference problem with architectural efficiency rather than brute force.

Can your PC run it?

Here is the sobering part. Because a Mixture-of-Experts model must store every expert, combined RAM plus VRAM decides what fits: quantized community builds span roughly 100 GB at heavy compression up to around 170 GB for lossless weights, according to Compare AI Hardware's DeepSeek V4 hardware requirements breakdown. No single consumer GeForce card holds the full model even at heavy quantization. Self-hosting therefore lives on high-unified-memory machines or multi-GPU workstations, with partial CPU offload as the slow-but-cheap fallback.

VRAM and memory options compared

Hardware	Memory	Bandwidth	Role for V4-class self-hosting
Mac Studio (M3 Ultra)	up to 512 GB unified	819 GB/s	Simplest single-box option
RTX 6000 Ada	48 GB GDDR6	960 GB/s	Multi-GPU workstation build
RTX A6000	48 GB GDDR6	768 GB/s	Prior-generation workstation
H100	80 GB HBM3	3350 GB/s	Datacenter inference workhorse
H200	141 GB HBM3e	4800 GB/s	Highest-capacity tracked option

The table mirrors the hardware records Compare AI Hardware tracks for the V4 family. A high-memory desktop without a discrete GPU can run heavily quantized builds on the processor alone — the small active-parameter count keeps that viable — but token speed will test your patience.

Frequently asked questions

Is DeepSeek V4 Pro 0813 free to use?

The weights are open, so downloading and self-hosting costs nothing in licensing fees. Your real costs are hardware, power, and setup time; hosted API access is priced separately by providers.

How much VRAM does DeepSeek V4 Pro need?

Think combined memory, not just VRAM: roughly 100 GB for heavily quantized builds and up to about 170 GB for lossless weights, per the hardware breakdown linked above.

Can a single RTX 4090 run it?

No. A 24 GB card cannot hold the stored experts even at heavy quantization; it can only participate in multi-GPU or CPU-offload setups.

What is DSpark?

DSpark is the speculative-decoding technique demonstrated in the episode linked below — it pairs a small draft model with the large one to raise token throughput.

V4 Pro or V4 Flash — which should I run?

Flash is the lighter, faster sibling and far easier to host; Pro trades convenience for capability. Match the tier to your memory budget before deciding.

Sources and further reading

• DeepSeek V4 Pro 0813 (official): https://ift.tt/LYU6jXT
• Episode sponsor — Lambda GPU Cloud: https://ift.tt/Z8SFAru
• DSpark full episode: https://www.youtube.com/watch?v=1yBU41auQhw
• Original source links: https://ift.tt/iP6QqZG, https://ift.tt/uBXoN5Y, https://ift.tt/lxnZRcN, https://ift.tt/WhpIRbk, https://ift.tt/pgaH8Yq, https://ift.tt/B9jHd2X, https://ift.tt/XvLqtRm

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

Disclosure: machine-learning.null.pictures and compareaihardware.com are operated by the same team. Links to compareaihardware.com are editorial recommendations, not paid placements.

Claude AI Failed 650 Times…Then Beat The Human Record

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers 📝 The paper is available here: https://ift.tt/VYqLmQd Source: https://ift.tt/7m1sGkc 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

OpenAI’s AI Agents Just Crossed A Line

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/R4Xo3FP 📝 More reports are available here: https://ift.tt/g2LXSYM https://ift.tt/3cxwGeq https://ift.tt/MoNJGOw 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Adam Bridges, B Shang, Carlos Galarza, Christian Ahlin, Eric Tyson, Juan Benet, Lukas Biewald, Michael Tedder, Owen Skarpness, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi

DeepMind's AI Trick Everyone Should Copy

❤️ Check out Lambda here and sign up for their GPU Cloud: https://ift.tt/maZYt4K 📝 The Gemma4 paper and some more is available here: https://ift.tt/WHuC3tz https://ift.tt/DYHyr82 https://ift.tt/7Ecy4fn 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Adam Bridges, Benji Rabhan, B Shang, Cameron Navor, Charles Ian Norman Venn, Christian Ahlin, Eric T, Fred R, Gordon Child, Juan Benet, Michael Tedder, Owen Skarpness, Richard Sundvall, Ryan Stankye, Shawn Becker, Steef, Taras Bobrovytsky, Tazaur Sagenclaw, Tybie Fitzhugh, Ueli Gallizzi