Local AI

I cheered Qwen 3.8 27B before the weights existed. Then I loaded NVFP4 on the ASUS Ascent GX10 and timed it. Threads, comments, and the numbers from that week.

I posted Qwen 3.8 27B three times and meant it. Why a 27B open-weight model still matters on a 128GB Spark-class box, and how I would actually load it.

Hermes agent, DeepSeek V4 Flash 0731, ASUS Ascent GX10 — local only. How I shipped the Moon Trail browser game without cheating on a cloud model.

I unboxed an ASUS Ascent GX10 — NVIDIA DGX Spark in a 150mm cube — and started running local models on 128GB of unified memory. No brochure. First notes from the bench.

Cornerstone WikiWayne guide: Best GPU for Local AI (2026). Open-weight, practitioner-tested local AI.

Shopping the secondary market without overspending on VRAM you cannot use.

Load checkpoints, wire KSampler, export PNGs locally.

Cornerstone WikiWayne guide: ComfyUI Local Stable Diffusion Guide. Open-weight, practitioner-tested local AI.

Slower tokens, stronger air-gap story.

What `-ngl` / GPU layer sliders actually do.

Compose services for local chat without cloud relay.

Quant-specific VRAM bands for Meta Llama 3 8B class models.

Step-by-step Ollama install paths for the three major desktop OS families.

Import GGUF and tune narrative sampling locally.

Cornerstone WikiWayne guide: KoboldCpp Local LLM Guide. Open-weight, practitioner-tested local AI.

Compile with GPU backends for NVIDIA cards.

Cornerstone WikiWayne guide: llama.cpp Complete Guide. Open-weight, practitioner-tested local AI.

Leave the managed service when you need custom builds or flags.

Use the LM Studio catalog without guessing quant labels.

Cornerstone WikiWayne guide: LM Studio vs Ollama vs llama.cpp: Which Local AI Tool?. Open-weight, practitioner-tested local AI.

Cornerstone WikiWayne guide: Local AI Model Tracker (2026). Open-weight, practitioner-tested local AI.

Network egress, logging, and backup habits for homelabs.

Cornerstone WikiWayne guide: MLX on Apple Silicon for Local AI. Open-weight, practitioner-tested local AI.

Python venv, mlx-lm, and a tiny model smoke test.

CUDA maturity vs ROCm tradeoffs for GGUF stacks.

Point agents and UIs at `http://localhost:11434/v1`.

Cornerstone WikiWayne guide: Open WebUI for Local AI. Open-weight, practitioner-tested local AI.

Docker and bare-metal pairing for household chat.

From zero to a working chat with a small Ollama tag.

When to spend extra gigabytes on higher precision.

Cornerstone WikiWayne guide: Quantization Explained for Local AI. Open-weight, practitioner-tested local AI.

What runs at usable speed on 8 GB Pi hardware.

Cornerstone WikiWayne guide: Raspberry Pi Local AI: Limits and Use Cases. Open-weight, practitioner-tested local AI.

Cornerstone WikiWayne guide: Run Open-Weight Models Locally (2026). Open-weight, practitioner-tested local AI.

Cornerstone WikiWayne guide: VRAM Requirements for Local LLMs. Open-weight, practitioner-tested local AI.

GGUF packs tensors and metadata for llama.cpp-compatible runners.

Ollama favors CLI and API automation; LM Studio favors GUI model browsing. Compare setup, VRAM use, and GGUF workflows on real hardware.

A complete guide to self-hosting your own services in 2026 — from hardware and Docker to Nextcloud, Vaultwarden, and OpenClaw on a Raspberry Pi or VPS.