Panel TokenGate

TokenGate lets administrators import a gate checkpoint, train a local gate, and compare served inference with and without a token keep budget. Open TokenGate at /rust/tokengate to choose the workflow. All submissions require an authenticated administrator. A job is considered successful only when its recorded process exit status says it succeeded.

Import checkpoint

1. Open Import checkpoint (/rust/tokengate/import).
2. Enter absolute paths for Gate checkpoint .pt, LoRA checkpoint .pt, Base GGUF and a new Output GGUF. You may also provide Parameters JSON and Mapping TSV together.
3. Select Import checkpoint. The Panel invokes lexi-engine tokengate-import GATE.pt LORA.pt params.json mapping.tsv base.gguf output.gguf as a tracked job.
4. Open Jobs and status. Confirm the import succeeded before using the output adapter.
5. Enter Instance identifier and Existing adapter GGUF. Tick Confirm adapter binding, then select Use in instance. Before writing the native instance profile, the Panel checks that the adapter exists, has a readable GGUF header and includes TokenGate layer metadata. It saves TOKENGATE_ADAPTER only after confirmation. Restart the instance separately to activate the adapter; the save response is not proof of an active gate. An incompatible profile or unsupported runtime is rejected.

When both Parameters JSON and Mapping TSV are empty, the importer derives the fields and exact mapping from recognizable checkpoint tensor names, metadata and base geometry. Unsupported inventories are rejected, not guessed. If either is supplied, both must be supplied. When alpha is absent from checkpoint metadata, the generated LoRA scale defaults to the derived rank. Files must already exist on the Panel host; the native form does not upload file bytes.

Train a gate

1. Open Train a gate (/rust/tokengate/train).
2. Supply Base GGUF, Training rows JSONL and a new Output GGUF.
3. Optionally set Steps (default 1), Gate layer (default 0), Seed (default 42) and Resume checkpoint (optional).
4. Select Train a gate, then inspect the job record on Jobs and status.

The Panel launches the existing lexi-train tokengate --base BASE.gguf --data rows.jsonl --out ADAPTER.gguf command, adding supplied numeric options. Successful process completion does not independently certify model quality.

Evaluate gate

1. Start the instance with a validated gate adapter.
2. Open Evaluate gate (/rust/tokengate/evaluate).
3. Enter Loopback completion URL and Served model identifier. The URL must use an explicit loopback host and a numeric nonzero port; user information, fragments and remote hosts are refused.
4. Submit Evaluate gate and inspect Jobs and status (/rust/tokengate/status).

The evaluation uses lexi-engine tokengate-eval --mode served --url URL --model NAME. It compares the ungated baseline against gated keep budgets. It refuses to claim latency, pruning or memory savings unless the runtime returns verifiable per-request evidence. Some server versions cannot produce that evidence and their evaluations fail rather than presenting made-up measurements.

Controls and API

Field help

The work area uses the same Panel navigation, accessibility controls and security boundary as other native pages. Read-only roles never receive a TokenGate job-submission or binding form. Administrator permissions and token route scopes are checked again by the owner when submitting a job or changing a binding. Form submissions are size-limited; malformed fields produce an error rather than a success page.

These APIs accept JSON input for the POST actions. The job submission response includes a job identifier. The status API returns stored job details and, after completion, a result object with the exit status. Output and log paths are local to the Panel host. The status page also displays adapter bindings from existing native instance profiles; unverified gate layers and savings remain unset. The status page displays the most recent records available in the local job store, not a guarantee that an active process survived a Panel restart.