Let's Infer documentation

Let's Infer turns local AI hardware into one reliable, OpenAI-compatible inference service. Start with a model name; Let's Infer handles the target, runtime, model artifacts, engine, lifecycle, safety, and telemetry.

Features at a glance

• one-command, hardware-aware model installation;
• automatic start, reboot persistence, and ordinary crash recovery;
• one stable OpenAI-compatible API across engines;
• target-aware replication across compatible main and child nodes;
• dynamic request admission and memory-aware queueing;
• live throughput, request, cache, temperature, power, and system telemetry;
• independent Watchdog protection with explicit safety recovery;
• signed, digest-pinned runtimes with independent community verification;
• independent core/runtime updates with runtime rollback; and
• secure mDNS discovery, scoped API keys, controller pairing, and audit.

Explore every feature

Start here

• Installation
• Features
• CLI reference
• Runtime format
• Runtime candidates
• Engine protocol and distributions
• Nodes, replication, and trust

Operations

• Core and runtime updates
• Watchdog
• Core release process
• macOS app release process

Contributing

• Testing and pull-request gates
• Developing and publishing runtimes

The public contract is runtime schema 6, runtime artifact schema 6, catalog schema 7, revocation-ledger schema 1, and Engine protocol 2. Unsupported schemas fail closed.