Skip to main content

​
⚙️ Prerequisites

  • If you haven’t already, install the nexa-SDK.
  • Below are the GGUF-compatible model types you can experiment with right away.

​
LLM - Language Models

📝 Language models in GGUF format. Try out this quick example: Try it out:
bash
nexa infer NexaAI/Qwen3-0.6B
⌨️ This will spawn an interactive REPL conversation session with the model.

​
LMM - Multimodal Models

🖼️ Multimodal models that accept vision and/or audio inputs. Try out this quick example:
bash
nexa infer NexaAI/Qwen2.5-Omni-3B-GGUF
⌨️ Drag images and audio files to the conversation input to chat with images / audio.

​
Supported Model List

We curated a list of top, high quality models in GGUF format.

LLMs for GGUF

Multimodal for GGUF

To try other GGUF models, visit Hugging Face, copy the path of any compatible GGUF model (e.g., unsloth/Qwen2.5-VL-3B-Instruct-GGUF), and replace the model path in the command above.
For more advanced models, you may visit the Nexa Model Hub. Also, access token is required to download and use these models. To get access token:
  • Create an account at sdk.nexa.ai
  • Generate a token: Go to Deployment → Create Token
  • Activate your SDK: Run the following command on the terminal to set your license:
bash
nexa config set license '<your_token_here>'

​
🙋 Request New Models

Want a specific model? Submit an issue on the nexa-sdk GitHub or request in our Discord/Slack community!
⌘I