alvarobartt/Magicoder-Vicuna-1.0
Viewer
ā¢
Updated
ā¢
75.2k
ā¢
31
Alvaro Bartolome
alvarobartt
AI & ML interests
machine learning @huggingface (inference + cloud)
Recent Activity
updated
a dataset
about 17 hours ago
huggingface/DEH-image-scan-data
liked
a model
3 days ago
nvidia/personaplex-7b-v1
replied to
their
post
4 days ago
š„ `hf-mem` v0.4.1 now also estimates KV cache memory requirements for any context length and batch size with the `--experimental` flag!
`uvx hf-mem --model-id ... --experimental` will automatically pull the required information from the Hugging Face Hub to include the KV cache estimation, when applicable.
š” Alternatively, you can also set the `--max-model-len`, `--batch-size` and `--kv-cache-dtype` arguments (Ć la vLLM) manually if preferred.
Organizations
![[INTERNAL] Hugging Face + Dell's profile picture](https://cdn-avatars.huggingface.co/v1/production/uploads/60f0608166e5701b80ed3f02/otGPyK6OMlrQrkngKw4tJ.png)
