Spaces:

Duplicated from Leon4gr45/gemma-inference

Leon4gr45
/

agent-inference

Running

App Files Files Community

agent-inference

48.2 kB

Ctrl+K

Ctrl+K

1 contributor

History: 39 commits

Leon4gr45's picture

Document client contract: thinking budget, think-stripping, routing policy

3675bfd verified 6 days ago

__pycache__
v2.0: single-instance CPU proxy, native OpenAI tools+streaming, flash-attn+q4_0 KV, all-cores 7 days ago
.gitattributes

1.52 kB
initial commit 3 months ago
.gitignore

45 Bytes
Upload gemma-inference space 3 months ago
.hfignore

45 Bytes
Upload gemma-inference space 3 months ago
Dockerfile

2.11 kB
llama-swap dual-specialist setup: Dockerfile 6 days ago
README.md

2.38 kB
Document client contract: thinking budget, think-stripping, routing policy 6 days ago
app.py

18.7 kB
Split K/V cache quant (K=q8_0,V=q4_0), 32K ctx, cache-reuse, tool-call smoke test 6 days ago
config.yaml

1.6 kB
Thinking fixes: strip client max_tokens (enforce n-predict 2048), deepseek reasoning-format 6 days ago
entrypoint.sh

735 Bytes
llama-swap dual-specialist setup: entrypoint.sh 6 days ago
requirements.txt

38 Bytes
v2.0: single-instance CPU proxy, native OpenAI tools+streaming, flash-attn+q4_0 KV, all-cores 7 days ago
test_chat.py

1.1 kB
v2.0: single-instance CPU proxy, native OpenAI tools+streaming, flash-attn+q4_0 KV, all-cores 7 days ago