Panel Builds and Engines

Install upstream releases, keep side-by-side engine builds, configure an instance to use a supported runtime, and manage gateway quotas from the Panel. A build selected for next start is not necessarily the one already serving requests. Only an observed process and a successful request confirm live serving.

Where to start

Open Builds and engines overview at /rust/builds-and-engines. Its links lead to Builds (/rust/builds), Media engines (/rust/builds-and-engines/media), Engine profile (/rust/lexi-engine), Advanced engine features (/rust/lexi-engine/features), Pinned llama.cpp build (/rust/engines/llama), and Inference gateway quotas (/rust/builds-and-engines/quota). The Builds page and its release list now use native, server-rendered controls; there is no separate classic Builds workflow. Each operation displays the real owner response. A small ? beside a control explains its effect.

Install the newest supported llama.cpp release and serve a model

Open Builds and select Find upstream releases (or follow Inspect and install the newest supported llama.cpp release from the overview). The Published archives table lists exact tags and compatible CPU, Vulkan, CUDA and ROCm archive choices. The latest entry is not necessarily compatible with the host.
Check the archive size, backend and verification text. Choose Download CPU, Download Vulkan, Download CUDA, or Download ROCm as offered for that release, and tick I confirm this change. The command stages an archive side by side; it does not switch the active build. Offline or unavailable release metadata is not a successful check.
In Transfers, choose Refresh transfer status to inspect job phase, byte progress and verification. Wait for ready and inspect the recorded evidence. Where the upstream project supplies no SHA-256, a size-only check is not the same as cryptographic verification.
When a compatible build is ready, tick I confirm this change. and select Use on next start. The selected build takes effect only when an instance starts or restarts. Keep the old build for rollback; avoid deleting an active build.
On Instances, create a distinct instance with llama.cpp, its local model path, supported device/backend and a free serving port. Start it with the instance controls. Check its running status and test an inference response. Gateway connection only lists locally serving targets; merely listing a model does not prove that generated output works.
To reverse an unfinished download, use Cancel transfer. To reverse a selected build, select the previous installed build; do not restart until you intend to change the live process.

For the separate, pinned source release controls, open /rust/engines/llama. Start side-by-side build, Activate for next start, Rollback build pointer, and Request build cancellation act on a pinned build lifecycle. Exact binary capabilities, not release claims, control usable flags. Source-build planning on the Builds tab is for developer hosts with the required toolchain; it is not the normal LexiOS release path.

On LexiOS, the runtime integrates Rust-provided interfaces corresponding to libgomp, libssl, and libcrypto for compatible upstream binaries. A successfully staged archive still needs verification and a compatible backend before use; do not equate a queued transfer with a running server.

Build inventory, source jobs and update policy

The Installed builds panel lists the selected build and its verification evidence. An administrator can use Use for next start, Qualify with active model, and, for an unselected build only, Delete build after typing DELETE. The target build is checked by the existing owner; qualification reads the selected instance model from the real parameter route. Selection does not restart a running process.

Under Automatic updates, an administrator can enable Check daily, enter UTC check time (HH:MM), choose Activation policy and How many older builds to retain, then select Save update policy and confirm. Check now launches an immediate check. Held updates expose Activate verified update after confirmation. The controls call the existing update owner, and viewer sessions have no mutation form.

Under Build from source, inspect available toolchains and patches before Preview source build. A compatible developer toolchain is necessary. Start source build requires confirmation and stages without activation. Inspect source build job queries the real job ID; Cancel source build job requires confirmation. The returned plan or job result appears above the refreshed page. If the toolchain probe fails, the Panel shows a reason rather than proposing an unusable build.

In Other engine builds, use Check published engine releases to request a release catalogue for the engine, Select media engine build to change a verified pointer or Delete media engine build after typing DELETE. The Transfers section includes both upstream and other engine downloads, and shows progress snapshots on page load.

Add a media engine to an instance

Open Install a media engine or create a media instance. Review Reported engine inventory and the installed builds in Builds before making changes.
In Stage a supported media engine, select Engine, Published release tag and Build flavour. Supported install choices are sd.cpp (CPU, Vulkan, ROCm), audio.cpp (CPU, Vulkan), and camelid (Linux release flavour). Tick the confirmation and select Install media engine. The Panel's engine owner validates published releases and records the transfer; a returned job ID is not completion.
After the build has completed, inspect Installed engine capabilities, Check flag changes and Validate an instance profile on the Builds tab. A missing or unsupported binary is a blocker, not an invitation to try another backend silently.
Use Create a media instance to choose New instance ID, Instance engine, Execution backend, Exact device ID, Model path and Listener port. Tick confirmation, then select Create media instance. This creates a separate instance without stopping or converting an existing one. For a GPU, use an exact identifier reported by the Hardware page; cpu must be paired with the CPU backend.
Configure and start the new instance from Instances. Inspect the matching image/audio serving page and test the requested operation. Creation alone neither starts inference nor establishes hardware support.

vLLM appears as a separate instance engine for batching with an accelerator; its dedicated runtime and driver environment must already be installed and can be inspected using /api/vllm/status. This form does not offer to install that runtime. ComfyUI is listed as a remote workflow kind on the media page and configured under Network; it and video/media workflows are separate integrations in Panel Media Generation; they are not falsely listed as available packages in this native installer. Audio tasks such as speech generation, transcription and music require the corresponding compatible runtime and model.

For per-instance native execution, use Engine profile to choose CPU, Vulkan, CUDA or supported Metal and save without restarting. Advanced engine features exposes requested expert placement, memory tiers and speculation settings but does not treat requests as observed acceleration. The Feature matrix distinguishes reported support from measured operation. The CLI interfaces shown by the installed runtime include lexi-engine serve --help and lexi-engine serve -m MODEL.gguf --backend cpu; use its current --help for exact flags.

Inference gateway: status and quotas

At /rust/builds-and-engines/quota, Live gateway status shows models, admission, holds and batch information. Gateway connection readiness explains whether multi-user API-key access is configured. Viewers receive no other users' quota or usage data; only an authenticated administrator sees the Save a user's gateway quota form.

Enter User ID, Requests per minute, Tokens per day, and Concurrent requests. Each limit must be an unsigned integer; 0 means no limit for that dimension.
Optionally enter Allowed model IDs, separated by commas or new lines. Blank permits every model the gateway otherwise exposes; at most 64 distinct IDs are accepted.
Tick Apply the new limits to this user for the current Panel process., then select Save gateway quota. The form calls POST /api/gateway/quota, displays the owner result and then loads current status. Errors do not claim success.

Gateway quotas and counters are process-local and do not survive a Panel restart. A quota applies to the authenticated API-key user, not to a model file or an instance's configuration. API-key setup and user permissions are covered in Panel Network and Access.

Controls and help text

The following labels and sentences come from the live UX feature registry.

Control / capability	What it does (tooltip)
Builds and engines overview	Shows installed engine inventory and links to release, instance and gateway tasks.
Automatic updates	Schedules release checks and decides whether verified updates wait for activation.
Check now	Checks the selected engine for published builds without restarting any instance.
Installed llama.cpp builds	Shows installed builds, verification status and the next-start backend selection.
Use for backend	Selects an installed build for the next start; running instances are not restarted.
Qualify	Tests build compatibility against an active build without replacing the running process.
Delete build	Removes a build only after confirmation and when it is not selected for use.
Build from source	Stages a source build beside installed releases and leaves activation to the operator.
Cancel source build	Stops a queued or running source build job after explicit approval.
Source build status	Shows the source build job and its recorded progress or failure reason.
Check for new releases	Lists upstream release assets that match the host and their reported verification evidence.
Download release	Stages an explicitly selected upstream archive; installation does not activate the build.
Transfers	Displays live upstream download verification and installation progress.
Activate downloaded build	Changes the next-start build after an explicit selection; the current server keeps running.
Cancel transfer	Requests cooperative cancellation and cleanup of an unfinished release download.
Engine capabilities	Reads supported flags from installed binaries rather than assuming a feature is available.
Flag changes	Compares installed binary flags and identifies profiles affected by a changed build.
Validate profile	Checks selected instance parameters against the exact installed engine binary.
Published media engine releases	Looks up published release assets for the selected installed engine type.
Install other engine	Stages a supported media engine build for later use by an instance.
Select engine build	Selects a verified media engine build without restarting a running instance.
Delete engine build	Removes an unneeded engine build after explicit confirmation.
ComfyUI remote integration	Lists externally served image workflows without claiming that ComfyUI is a locally installable engine.
Create media instance	Creates a separate instance with the selected supported image, audio or batching engine.
Engine profile	Saves per-instance execution settings without restarting the server.
Advanced engine features	Configures requested expert, memory and speculation options; unverified lanes remain disabled.
Feature matrix	Shows reported support separately from live measured engine performance.
Pinned llama.cpp build	Installs and probes a pinned source build beside existing releases.
Llama build selection	Selects or rolls back the pinned build pointer without restarting a process.
Batching engine status	Shows detected accelerator runtimes and installation guidance for the batching engine.
Gateway status	Shows local serving targets, queue activity and role-appropriate quota information.
Gateway connection	Explains whether multi-user API-key access is ready and which models can be served.
Save gateway quota	Sets request, token, concurrency and model limits for a user; zero means unlimited.

Relevant API routes

Read-only: GET /api/builds, GET /api/builds/upstream/releases, GET /api/builds/upstream/status, GET /api/builds/source/job, GET /api/builds/source/patches, GET /api/builds/source/toolchains, GET /api/downloads, GET /api/engines, GET /api/engines/releases, GET /api/engines/catalog, GET /api/engines/catalog/validate, GET /api/engines/catalog/diff, GET /api/gateway, GET /api/gateway/connect, GET /v1/models, GET /api/vllm/status, GET /api/remotes/kinds, GET /api/remotes.
Changing builds: POST /api/builds/upstream/install, POST /api/builds/upstream/cancel, POST /api/builds/upstream/activate, POST /api/builds/source/plan, POST /api/builds/source/start, POST /api/builds/source/cancel, POST /api/builds/activate, POST /api/builds/qualify, POST /api/builds/delete, POST /api/engines/policy, POST /api/engines/check, POST /api/engines/install, POST /api/engines/activate, POST /api/engines/delete.
Instances and quotas: POST /api/instance/create, POST /api/params, POST /api/gateway/quota.

Requests that modify engine installation, instance state or user quota require administrator privileges at the Panel boundary. A native form's confirmation does not remove the endpoint's own validation. See Panel Instances and Panel API for related operations.

When something fails

A release marked offline or missing a compatible asset cannot be staged from this catalogue. Check the exact backend and architecture, refresh only when network access is available, and never substitute an unverified archive. If the installer reports a failure, inspect Transfers and its error; it has not silently activated the build. If a new instance refuses launch, read the installed-binary capability report and its launch plan, then verify the model file, port and device assignment. If a viewer cannot edit quotas, ask an administrator rather than trying to change roles from a URL.