Buying a mini PC for local AI? Start with the application and model you want to run. An “AI PC” label—or a higher TOPS number—does not tell you whether your chatbot, transcription tool or image generator will use the hardware inside.

For occasional local chat, a capable CPU may be sufficient, while a supported iGPU can provide acceleration. An NPU is useful for applications designed to use it. For demanding generative AI, a suitable discrete GPU is often the stronger option.

This guide focuses on Windows mini PCs. Software support and how memory is shared can differ on Linux and Apple silicon systems.

NPU, iGPU and Discrete GPU: The Key Differences

An integrated GPU, or iGPU, is a type of GPU built into the processor or system-on-chip. A discrete GPU is a separate graphics processor, typically with its own dedicated video memory, known as VRAM.

These are general tendencies, not performance rankings. Integrated systems vary widely, and some premium platforms offer more usable model memory than discrete GPUs with limited VRAM. Results from those systems should not be assumed to apply to budget mini PCs.

The CPU also remains relevant. It handles everyday computing and can run local models through compatible software, even without GPU or NPU acceleration.

Do You Need an NPU for Local AI?

No. Local AI means the model runs on your computer; it does not require an NPU.

An NPU is designed to execute supported neural-network workloads efficiently. It can be useful for camera effects, audio processing and compact language models when the application provides a compatible implementation.

Having an NPU does not mean your chatbot will use it. The application needs software that can send the model’s operations to that processor. This software layer is often called a backend.

Microsoft’s Windows ML framework, for example, lets developers use execution providers to run inference on supported CPUs, GPUs and NPUs. Applications must integrate the relevant functionality; Windows ML does not automatically accelerate every installed AI application. 

An NPU is therefore worth prioritising when you can identify a feature or application that will benefit from it. Otherwise, give more weight to CPU performance, memory and GPU capabilities.

Why TOPS Cannot Predict Chatbot Speed

TOPS measures trillions of operations per second under specified conditions. It does not directly tell you:

• How quickly a chatbot starts answering.
• How many tokens it generates per second.
• Whether your model is supported.
• Whether the model and its context fit in memory.

Numerical precision, memory movement and software implementation all affect performance. Also check whether an advertised TOPS figure refers to the NPU alone or combines multiple processors.

A benchmark using your intended application and model is more useful than an isolated TOPS figure.

When Is an iGPU Enough?

An iGPU is worth considering when you want a compact everyday computer that also runs lighter AI workloads.

For example, you might primarily use office applications, browse the web and play media, with occasional local AI for drafting text or summarising short documents. A capable CPU, sufficient RAM and a supported iGPU can be a practical combination.

GPU compatibility needs to be specific. “Supports AMD graphics” or “supports Intel graphics” is not enough. Ollama, for example, documents different GPU support paths for different operating systems and provides additional hardware support through Vulkan. Its hardware documentation is a useful starting point for checking an exact device. 

Because an iGPU typically shares RAM with the CPU and other applications, memory capacity and memory bandwidth both matter. More RAM can help a model fit, but it does not automatically make generation faster.

When Is a Discrete GPU Worth It?

A discrete GPU becomes more attractive when waiting for AI output regularly interrupts your work.

Examples include generating images repeatedly, processing batches of files or using a local language model throughout the day. GPU acceleration can substantially reduce processing time in applications designed to use it.

VRAM capacity deserves as much attention as processing power. If a workload exceeds available VRAM, some applications can move part of it to system memory, often at a performance cost. Others may require a smaller model or lower settings, or fail to load the workload.

For mini PC buyers, the trade-offs include enclosure size, cooling, power consumption and price. An external GPU adds connection-bandwidth and compatibility considerations, so confirm that the mini PC supports the intended connection and enclosure before buying.

If demanding generative AI is your primary workload, compare the complete mini PC setup with a larger desktop that can accommodate the GPU you need.

How Much Memory Does a Local AI Mini PC Need?

Memory use depends on model size, quantisation, context length and the application.

Quantisation stores model weights at lower precision to reduce memory use. Context length describes how much text a language model can process at once, measured in tokens. Both affect the memory required for a particular task.

These are buying guidelines, not model-specific requirements. LM Studio, for example, currently recommends at least 16GB RAM for Windows and at least 4GB dedicated VRAM. 

As a simplified calculation, 8 billion parameters stored at exactly 4 bits each would require 4GB for the weights alone.

Real quantised models also include metadata and may retain some parameters at higher precision. Running the model requires additional memory for caches, buffers and the application. Longer contexts can increase cache memory, so short chats and long-document tasks may have different requirements. 

Leave room for Windows and other applications too. On an iGPU system, check how much shared memory the runtime can actually use. With a discrete GPU, evaluate VRAM separately from system RAM.

Check Your Software Before Choosing Hardware

Before comparing mini PC specifications, identify the setup you intend to use:

After installation, confirm which processor the app actually uses and whether response times meet your needs. Check its device information, logs or GPU offload settings. Hardware activity monitors can provide supporting evidence, but they may not explain how work is divided between processors.

If you already own a compatible computer, try your intended model there first. That can reveal whether you need more speed, more memory or different software settings.

Look for Relevant Performance Tests

A useful local AI benchmark should identify the full configuration and workload.

Look for:

• The processor, RAM configuration and power settings.
• The application version and acceleration backend.
• The exact model, quantisation and context length.
• Time to first response and generation speed.
• Memory use during the test.

For document assistants, look for tests with representative documents. For transcription, compare processing time with the recording’s duration. For image generation, check the model, resolution and generation settings.

Sustained performance matters on a mini PC. Power limits and cooling can affect speed during repeated or long-running inference. Extended tests and noise measurements are more informative for regular use than a single short demonstration.

Which Mini PC Configuration Should You Choose?

For occasional local chat and everyday productivity: a capable CPU and sufficient RAM may be enough. A supported iGPU can provide useful acceleration.

For a specific NPU-enabled feature: choose a system that meets that feature’s published hardware requirements.

For frequent image generation or demanding LLM use: prioritise GPU performance, usable memory and sustained cooling. Consider a larger desktop if it better accommodates your workload.