TinyCoder AI — Ultra-fast local AI coding agent for VS Code

Key features

Built for speed. Built for your hardware.

No cloud, no API keys, no telemetry — a native C++ inference engine that squeezes every last token out of the machine you already own.

⚡

SIMD CPU Acceleration

Runtime dispatch auto-detects the fastest instruction set available — Intel AMX, AVX-512 or AVX2 — with OpenMP thread parallelism for every matrix operation.

AMXAVX-512AVX2OpenMP

🎮

CUDA GPU Offload

Full-model GPU offload with cuBLAS tensor-core prefill, on-the-fly dequant GEMV decode and flash attention — with automatic CPU fallback when CUDA is unavailable.

cuBLASFP16 GEMMGEMV

🧩

Mixed Quantization

Up to 12 GGML quantization formats inside a single model file. Weights stay in their native quantized format and dequantize on-the-fly — keeping memory near file size.

IQ3_XXSIQ2_SQ4_K_XLQ6_K

🧠

Modern Model Families

Qwen2.5-Coder (0.5B–7B), Gemma 4 (26B MoE) and Qwen3.6 (35B MoE) — dense and MoE architectures with RoPE, GQA, SwiGLU and RMSNorm.

DenseMoEGQA

💻

VS Code Native Integration

A chat panel for interactive assistance, plus Explain, Complete and Generate commands — select code and get AI help right where you work.

Chat panelExplainCompleteStreaming

🤖

Autonomous ReAct Agent

An agent harness that reads and writes files, runs terminal commands and searches your codebase through structured JSON tool calls — nothing leaves your machine.

ReActToolsPrivate

Architecture

From keystroke to tokens — on-device

A thin TypeScript layer drives a native N-API addon over a purpose-built C++ inference engine with runtime SIMD dispatch and quantized kernels.

VS CodeUI & commands

TypeScriptpanel · bridge · agent loop

C++ N-APIload · generate · status

GGUF Enginequantized weights · fused kernels

SIMD / CUDAAMX · AVX-512 · AVX2 · cuBLAS

Supported models

Big models, tiny footprint

Every model is load-verified, benchmarked and ready to run on a consumer desktop with 11 GB of VRAM.

Qwen2.5-Coder Dense

0.5B · Q4_K_M~350 MB
1.5B · IQ3_XXS~637 MB
7B · IQ3_XXS~2.9 GB

Gemma 4 MoE · 26B-A4B

Coding · Q2_K~8.5 GB
Coding · Q4_K_M~13 GB
Instruct · Q4_K_XL~14 GB

Qwen3.6 MoE · 35B-A3B

A3B · IQ2_M (UD)~9.4 GB
A3B · IQ3_XS~12 GB
A3B · Q4_K_M~20.6 GB

Mixed quantization

12 GGML formats, one lean model

Each tensor is stored in its most efficient native format and dequantized on the fly during matrix-vector multiplication.

IQ3_XXS

3.06 bits/w

IQ3_S

3.44 bits/w

IQ3_XS

3.31 bits/w

IQ2_S

2.56 bits/w

IQ2_M

2.69 bits/w

Q2_K

2.56 bits/w

Q4_K

4.06 bits/w

Q4_K_M

4.50 bits/w

Q4_K_XL

4.50 bits/w

Q5_K

5.06 bits/w

Q5_1