← Back to the list
P

pi-llama-slot-isolation

Prevents concurrent pi sessions from clobbering each other's llama.cpp KV cache — each session is pinned to the best available slot (empty > cold > low tokens > low id) via the server's /slots endpoint, coordinated across windows through a state file

Weekly downloads
24
Monthly downloads
413
Stars
0
Users (7 days)
0
Likes
0

Install

Pi
pi install npm:pi-llama-slot-isolation
Version
0.1.0
Publisher
driderr
License
MIT