Running Your First AI Model

Load and run AI models on your Thox.ai device for intelligent code completion.

Back to Articles

Your Thox.ai device comes with thox-coder pre-installed and ready to use. This guide shows you how to verify it's working, download additional models, and start using AI-powered code completion.

The Pre-installed Model

Every Thox.ai device ships with thox-coder ready to go. This is our recommended model for most use cases - it provides an excellent balance of speed and quality for code completion tasks.

To verify it's loaded and ready:

thox models status

You should see "thox-coder" listed as the active model with status "loaded".

Available Models

thox-coder

Recommended

Default model. Balanced speed and quality for code completion.

7B parameters
4.1 GB disk

thox-coder-fast

Faster responses, slightly lower quality. Good for quick completions.

3B parameters
1.9 GB disk

codestral

Mistral's code model. Excellent for complex generation and review.

22B parameters
12.4 GB disk

llama3-8b

General-purpose. Good for documentation and explanations.

8B parameters
4.7 GB disk

View all available models: thox models list --available

Downloading Additional Models

Download a Model

thox models pull codestral

Download time depends on model size and connection speed. A 7B model takes ~10-20 minutes on typical broadband.

Switch Active Model

thox models switch codestral

The new model loads into memory and becomes active for all requests.

Remove a Model

thox models delete codestral

Free up disk space by removing models you no longer need.

Testing Your Model

Quick Test via CLI

thox complete "def fibonacci(n):"

The model will complete the function. First request may take a few seconds as the model loads into memory.

Web Interface Chat

Navigate to http://thox.local:8080/chat for an interactive chat interface. Try asking it to explain code or generate functions.

API Test

curl http://thox.local:8080/v1/completions \ -H "Content-Type: application/json" \ -d '{"prompt": "def hello():", "max_tokens": 50}'

Connect Your IDE

For the best experience, connect your development environment to Thox.ai for real-time code completions as you type.

VS Code SetupAll IDE Guides

Tips for Best Results

Start with thox-coder

It's optimized for the device and provides excellent results.

Give it context

More code context leads to better completions. Open related files in your IDE.

Use descriptive names

Good function and variable names help the model understand intent.

Iterate and refine

Accept partial completions and continue typing for better results.

Next Steps

Explore Features

Learn about all device capabilities

Optimize Performance

Get the best speed and quality