Compatibility testing in progress

Compatibility

Bring a GGUF file. Every family runs on the GPU, and most on the CPU as well. Loculus Core reads what it needs from the model itself, so new releases of a supported family work as they arrive. Every model is validated against an exact reference before it is listed.

Model families

• Llama

  Llama 2, Llama 3.x, Mistral

  GPUCPU

• Qwen

  Qwen2, Qwen2-VL (text), Qwen3, Qwen3 MoE

  GPUCPU

• Qwen3.8

  Hybrid recurrent + attention

  GPU

• DeepSeek V4.1

  Latent attention, 748B MoE streamed from SSD

  GPU

• GLM

  GLM-4 (GPU and CPU); GLM-5.3 (GPU)

  GPUCPU

• Nemotron 3.5 Lightning

  Hybrid Mamba-2 MoE

  GPU

• gpt-oss

  MXFP4 mixture of experts

  GPUCPU

• Gemma

  Gemma 2, Gemma 3

  GPUCPU

• Phi

  Phi-3, Phi-4

  GPUCPU

• Granite

  IBM Granite

  GPUCPU

• SmolLM3

  Long-context small model

  GPUCPU

• OLMo 2

  Fully open model family

  GPUCPU

Quantization formats

• Q8_0
• Q4_0
• Q5_0
• Q5_1
• Q4_K
• Q5_K
• Q6_K
• F32
• F16
• BF16
• IQ1_S
• IQ1_M
• IQ2_XXS
• IQ2_XS
• IQ2_S
• IQ3_XXS
• IQ3_S
• IQ4_NL
• IQ4_XS
• MXFP4

Platform

GPU

Vulkan compute, with or without matrix cores. Broader GPU testing is under way.

CPU

x86-64, vector-accelerated. Runs without a GPU.

Models

GGUF files, single or split. Experts stream from the SSD when a model is larger than memory.

APIs

Anthropic Messages, OpenAI Responses and OpenAI Chat Completions, streamed or not, with tool calls and structured outputs.

Operating systems

Linux first, then Windows, Android, macOS and iOS.

Your devices

One purchase, linked to your account: sign in with Google or Apple and download Loculus Core on any device you use.