LexiChat — first-party conversations
Chat with a running model using the Panel's own conversation store and engine adapters. Open Chat to check model readiness, select a model, and use the composer. Camelid is not required.
Open LexiChatModel availability is checked on the Chat page; this link does not mean an engine is running.
Running models
llama-server on this box, each
asked on its own port. Start / Restart / Stop below act only on the managed one
(main's system unit); everything else on this tab describes that one too.Live, every engine the same numbers for llama.cpp, vLLM and SGLang, on this box and registered remotes, read from each server's /metrics every 5 s while this view is open
Remote endpoints model servers on other machines (llama.cpp, vLLM, Ollama, LM Studio, any OpenAI-compatible API), listed above by address
Instance
Instance profiles
Server
Hermes Agent
config.yaml for this instance
llama.cpp web UI
Generate
negative prompt
Run
model options for this run
Recent outputs
Throughput
Decode vs context depth
Measure again
Safe restarts
Live
VRAM drops to near zero when idle — amdgpu evicts to GTT and pages back on the next request. Judge placement by the peak while serving, or by t/s.
Backend
Model preset
Camelid models
Models on this instance
Model catalog
model_specs. Packages download from Hugging Face into
~/audiocpp/models, shared by all audio.cpp instances. Q8_0
packages are smaller and usually as good; check sizes before installing music models.Memory calculator
Host RAM budget
ps RSS nor the cgroup. GTT ceiling + prompt cache + process
overhead has to fit in physical RAM. It did not between Sep 4 and Sep 8,
and nothing in this panel was modelling it.
Launch parameters
All other options from this build
--help of the build this instance launches, so a new daily build's new
options show up here without a panel update. Only options the form above does not already set are
listed. Ticking one writes it into the extra-arguments field above; press Save to keep it.Model profiles
Defaults & fallback tiers
Optimize
Autopilot · optimize my workload
Optimize
Validate running instance
Results
Compare endpoints
Compare two models · perplexity and task suites
llama-perplexity, without touching a running server, or a task suite
(multiple choice and exact answers, JSON file) sent to two running instances you name. The difference comes with a
95 % interval; an interval that includes zero is called inconclusive. Each run is kept with a qualification
record. A KL-divergence run writes the full baseline logits under panel/evals/work while it runs:
tens of GB for a large model and corpus.Bench
Bench · measurements that say how sure they are
Your traffic
Calibration · the platform's own error rates
Run
Profile the running server as it is (no restart)
Calibrate A/A: the saved settings against themselves, then the self-test
Compare A/B: saved settings (A) against saved + your changes (B); restarts the server
From another tool's command line (an optimizer's recommendation, a forum post): paste it, and measure the claim here
Goodput concurrency: 1..N simultaneous agent turns
Runs
Qualification records
Workload
What your requests look like
Context depth of requests
Busy and idle hours
Findings
Auto-fit
Proposals
History
Gateway: queue and batch jobs
Connect a chat app (Open WebUI or any OpenAI-compatible client)
Fleet
Boxes
Link reliability and drift
Access
Users
Result vault
Model verification
API keys
Audit log
Panel layout
Fit: requants made for your cards
1 · Plan
Recipes · read how a GGUF was made, make it again
2 · Candidates (report card)
~/models/fit.
Verify runs the optimizer in models mode on the chosen instance: today's model first,
then each selected candidate, with the same coding and agent tasks, speed probes, a decode curve
by depth, and the VRAM each one really occupies. It restarts the server once per model; your
saved settings are snapshotted and restored. Progress shows in the Optimize tab.Refusals · uncensoring, phase 0
mlabonne/harmful_behaviors) and ordinary ones (mlabonne/harmless_alpaca)
to catch over-refusal, each reply's opening checked for refusal phrases. Downloaded on first
use, not shipped. Nothing restarts: it sends one chat request at a time to the running
server, so other clients wait a few seconds between them. Refused counts only a
first-person refusal at the start of the reply ("I can't…", "I won't…", "Sorry…").
Keyword is Heretic's own score, for comparison: it also counts an answer that opens
with "this is illegal" and then complies, so it runs high. Check the listed verdicts.3 · Your cards: measured formats
Measure formats on an instance's cards (phase B)
What each vendor's cards are good at
4 · Importance matrix
Import an importance matrix made elsewhere
5 · Parts and other engines
--mmproj separately, so Fit builds the
projector as its own file (Q8_0 is ~2/3 the size of F16). stable-diffusion.cpp keeps the VAE at
f16 by default: it is small and quantizing it shows as banding. audio.cpp takes per-prefix keep rules.6 · Upstream watch
Fetch and convert a source (phase A)
Disk
Other engines · the same source for ONNX Runtime, vLLM / SGLang and AutoRound GGUF
Built
What was measured
Available models
Delete is permanent and these files are re-downloadable only over the link that took hours the first time. A model held by any backend profile or fallback tier cannot be deleted — hover the reason to see which file still points at it, and repoint that first.
Fetch by URL
Saves into ~/models. Resumable — re-submitting the same URL continues a partial file.
Use the repo's /resolve/main/<file> link, not the repo page — a page URL
returns 200 with HTML and is rejected here rather than saved.
Vision projectors (mmproj-*.gguf) download here too, then get
selected with Use on the Models tab. Each model repo ships its own and they are
not interchangeable: a projector from another repo has the same shape, so it loads
with no error and only the image answers are wrong.
Hugging Face access token
meta-llama/Llama-3.2-1B-Instruct)
refuse anonymous downloads. Accept the repo's terms on huggingface.co with your account,
create a read token, and paste it here. It is stored in
~/.cache/huggingface/token (mode 600) — outside panel/, so
backups never include it — and handed to curl through a header file, never on its
command line. Only sent to huggingface.co.Publish measured model to Hugging Face
Preview a generated card and file list before any upload. Requires a Hugging Face write token above and sealed panel qualification IDs for the model. All files are relative to the panel's model folder. New repositories are private by default.
Transfers
Installed templates
Jinja chat templates in ~/models. The launch script SHA-pins
the selected template and aborts on a mismatch; the panel writes the matching pin
on every save. So editing a selected template on disk will stop the server starting —
re-save on the Parameters tab to re-pin it.
Use a model’s own template
Every GGUF here embeds the template it was built with. This lifts it out
into ~/models as a file you can select, instead of hunting for it on
Hugging Face.
Paste a template
Checked with Jinja2 before it is written. llama.cpp parses with minja, a subset of Jinja, so a clean parse here is necessary but not sufficient — Diagnostics reads the authoritative result out of the server log after a restart.
Download a template
Use the Download tab with a direct link to a .jinja
file (the /resolve/main/… form). It saves into ~/models
and appears in the table above.
GPUs
The cards below show this instance's primary device in detail.
Power cap
A cap limits average board power (PPT). It lowers sustained draw and heat, but sub-second spikes still go through, so it is not a fix for transient trips. Applied caps are saved and re-applied by the panel at start and every minute, so they survive reboots and GPU resets. Clear forgets the saved cap; the live value stays until the next reboot.
Memory
Per-process residency
From DRM fdinfo. This is the trustworthy placement signal — unlike the whole-GPU counter, it isn't distorted by idle eviction.
Thermals & power
Clocks & link
VRAM vs GTT over time
amdgpu_top
GPU tuning
Card
Clocks, voltage & power
Profiles
Benchmark
Board power during the benchmark
Results
Advice
VBIOS
Usage
Throughput
Prefill excludes prompts under 200 tokens — on short prompts the figure is almost entirely fixed overhead and reads misleadingly low.
Decode rate, recent requests
Prefill rate, recent requests
Plotted separately from decode on purpose. Prefill runs hundreds of t/s against decode's tens; sharing one axis would flatten decode into a straight line.
Draft acceptance
Workload dependent — around 0.82 on long documents, 0.48–0.61 on short chat turns. No fixed baseline: acceptance depends on what is being written, not on the card.
Recent requests
Checks
For a model
Bundle carries pre-analysed findings with evidence and fixes, the exact command line and env, the VRAM/GTT series, usage stats, known baselines and the trap list — so a model can diagnose without re-deriving what we already know.
Configuration backup
Packages every hand-tuned config on this box: the launch script and its revisions, this panel and its params history, the chat template, systemd units, the Caddyfile, ttyd config, netplan staging, and the archived engine logs — plus a live-state snapshot to diff against later. Excludes the 17 GB of .gguf weights and the llama.cpp binaries, which are re-downloadable. Roughly 700 KB. Built fresh on each click. Contains the Caddy basicauth hash, so treat the file as a secret.
Live command line
Power options
Host restart & shutdown
This controls the entire machine, including a LexiOS live session, not the selected inference instance. Active instances and jobs may be interrupted. Requires an administrator and the current root power helper.
Equipment
Settings
Profiles
Stability by profile
Power budget & UPS
Change log
Crash analysis
Findings
Boot history
Parameter changes
Automatic updates
Installed llama.cpp builds
~/llama; each backend records which one
it launches from in panel/builds.env. Nothing is overwritten in place,
so rolling back is the same dropdown as upgrading. A build is classified by the
ggml library it ships, not by its directory name.
Changing the active build needs a restart to take effect.
Engine capabilities
--help once per exact build (by SHA-256)
and kept. Saving or starting an instance checks its settings against this: a flag the build does not know blocks,
anything else is a warning. After an upgrade, Flag changes shows what the new build added or dropped.Build from source
Available upstream
ggml-org/llama.cpp carrying an x64 Ubuntu asset
this box can run. Downloads unpack to a staging directory and are only promoted
once they yield a working llama-server of the expected flavour, so a
dropped connection cannot leave a half-build selectable.
Transfers
Log window
Refreshes every 5s while this tab is open. The view sticks to the bottom only if you are already at the bottom, so scrolling back to read something will not be yanked away by the next refresh.
Engine log
Launch log
Files ~
Click to select (Ctrl / Shift for more), double-click a
folder to open it. Drag rows onto a folder, the .. row or a path above to move them; hold Ctrl to copy.
Keys: Ctrl+A, Ctrl+C, Ctrl+X, Ctrl+V, Delete, F2 rename, Backspace up, Esc clear.
Folders and multi-selections download as a zip.
Rooted at your home folder. Hidden system folders (.ssh, .config…)
are not reachable from here, panel/ is read-only, and a file a server is using or a
setting points at (models, projectors, templates, a log being written) cannot be deleted, moved or renamed.
Disk usage tree
Apparent file sizes grouped by directory, compared with the selected drive's free space. The scan has strict entry/time/depth limits; a partial tree is an undercount, not the complete drive usage. Symlinks and other mounted drives are never followed. A full-system scan is available to administrators on request.