Advanced Configuration

Fine-tune your Thox.ai device for optimal performance and security.

Model Management

Listing Models

# List all available models

thox models list --all

# List installed models

thox models list --installed

Installing Models

# Install a model

thox models pull thox-coder-large

# Install with specific quantization

thox models pull thox-coder-large:q4_k_m

Model Priority

Configure model loading priority and memory allocation:

# /etc/thox/models.yaml
models:
  thox-coder:
    priority: high
    memory_limit: 8GB
    auto_load: true

  thox-chat:
    priority: medium
    memory_limit: 4GB
    auto_load: false

Custom Models

Import GGUF-compatible models:

# Import from local file

thox models import ./my-model.gguf --name custom-model

# Import from URL

thox models import https://example.com/model.gguf --name custom-model

Performance Tuning

Inference Settings

# /etc/thox/inference.yaml
inference:
  # Number of threads for CPU operations
  threads: 4

  # Batch size for inference
  batch_size: 512

  # Context window size
  context_length: 8192

  # GPU memory fraction to use
  gpu_memory_fraction: 0.9

  # Enable flash attention
  flash_attention: true

  # KV cache quantization
  kv_cache_type: q8_0

Memory Optimization

Low Memory Mode

Use smaller context, aggressive offloading

thox config set memory_mode low

High Performance Mode

Maximize speed, use full memory

thox config set memory_mode high

Benchmarking

# Run performance benchmark

thox benchmark --model thox-coder

# Expected output:

Model: thox-coder

Prompt eval: 125 tokens/s

Generation: 45 tokens/s

Memory usage: 6.2GB

Security Settings

API Authentication

Enable API key authentication for remote access:

# Generate API key

thox auth generate-key --name "my-app"

# Enable authentication

thox config set auth.enabled true

# Use in requests

curl -H "Authorization: Bearer sk-xxx" http://thox.local:8080/v1/models

Network Access Control

# /etc/thox/security.yaml
network:
  # Bind to specific interface
  bind_address: "0.0.0.0"

  # Allowed IP ranges
  allowed_ips:
    - "192.168.1.0/24"
    - "10.0.0.0/8"

  # Rate limiting
  rate_limit:
    requests_per_minute: 60
    tokens_per_minute: 100000

TLS/HTTPS

Enable HTTPS for secure connections:

# Generate self-signed certificate

thox tls generate --hostname thox.local

# Or use existing certificate

thox tls import --cert /path/to/cert.pem --key /path/to/key.pem

# Enable TLS

thox config set tls.enabled true

Security Note: When exposing your device to the internet, always enable authentication, use HTTPS, and configure firewall rules to restrict access.

Backup & Restore

Creating Backups

# Full backup (config + models)

thox backup create --output /path/to/backup.tar.gz

# Config only backup

thox backup create --config-only --output /path/to/config-backup.tar.gz

# Automatic scheduled backup

thox backup schedule --daily --keep 7 --output /backups/

Restoring from Backup

# Full restore

thox backup restore /path/to/backup.tar.gz

# Restore config only

thox backup restore --config-only /path/to/backup.tar.gz

What's Included

Configuration

  • • Device settings
  • • Model configurations
  • • Network settings
  • • API keys

Data

  • • Installed models
  • • Custom prompts
  • • Chat history (optional)
  • • Usage statistics

Factory Reset

# Reset to factory defaults (keeps models)

thox system reset --keep-models

# Full factory reset

thox system reset --full

Warning: Factory reset is irreversible. Always create a backup before performing a reset.