Local AI, models and integrations

Where does my request run?

ThoxOS provides the runtime and device layer. MeshStack connects enrolled devices for configured network and compute workflows. ThoxWork is the workspace product in preview. Check the selected application, runtime and route before sharing a request.

• On this device: an installed model runs on the machine you are using. Model downloads, updates and connected tools can still require a network.
• On a peer: another enrolled device handles the request. A machine on your local network is still a separate recipient of that request.
• Hosted processing: the request is sent to a server or external model service. Review the connection, consent and data terms first.

The live website demos use hosted processing. Edge Lab also offers a separate deterministic browser sample. Neither the sample nor a hosted response proves that a model ran on your own hardware. Read the Edge Lab guide and ThoxEmployee guide before using them.

Choose a model that fits the task

1. Check the exact model identifier, license, supported inputs and runtime compatibility. A catalog entry is not proof that your device can load it.
2. Allow memory for weights, the runtime, context and working data. Model download size alone is not the total RAM or accelerator-memory requirement.
3. Start with a short, non-sensitive example. Measure load time, response time and output quality on your actual hardware before increasing context or concurrency.
4. Review generated answers. A small model, successful request or confident response does not establish correctness or permission to act.

What does quantization change?

Quantization stores model weights at lower precision to reduce their memory footprint. The effects on speed and answer quality depend on the model, runtime and hardware. It does not guarantee a particular context length or tokens-per-second rate. Use the compatibility guide and your installed runtime's model information.

Connect to the runtime you actually installed

Use the base URL, authentication method and model ID shown by your runtime or deployment administrator. An API-compatible request format does not make every endpoint, streaming mode, tool call or model interchangeable.

For a standard local Ollama installation, the native API uses http://localhost:11434/api, while the compatibility API uses http://localhost:11434/v1. These are Ollama defaults, not universal THOX device addresses. Ollama can also use cloud models through a local server; a localhost URL alone does not prove local inference. See the runtime API reference and authentication guidance.

Keep credentials out of browser code and shared logs. Do not expose an unauthenticated local runtime directly to the public internet. Follow your deployment's access controls when connecting an editor or another device. See inference API compatibility.

What does MCP add?

Model Context Protocol (MCP) connects an AI application to servers that expose tools, resources and prompts. A server may run locally or remotely; MCP itself does not determine where a model runs. Client and server versions must support the same capabilities. The MCP architecture reference describes those boundaries.

Inspect a connection's publisher, data access and available tools before enabling it. Reading context and performing an action are different permissions. Use the controls provided by the installed client, and review consequential actions before they run. A website demo output or saved employee profile does not activate an MCP connection.

See MCP integration guidance for setup checks and security support for data-sharing questions.