Robert Shaw's picture

17 6 4

Robert Shaw

robertgshaw2

·

rsnm2

AI & ML interests

None yet

Organizations

New activity in RedHatAI/DeepSeek-R1-Distill-Llama-70B-FP8-dynamic 10 months ago

Update tokenizer_config.json

#3 opened 10 months ago by

New activity in nm-testing/pixtral-12b-w4a16-actorder-group 12 months ago

What is an actorder group and what are the advantages of running this in vLLM?

#1 opened 12 months ago by

New activity in RedHatAI/Sparse-Llama-3.1-8B-2of4 about 1 year ago

Can I apply a LoRA?

#1 opened about 1 year ago by

New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic about 1 year ago

Nice model, any info on scripts used to quantize?

#1 opened about 1 year ago by

New activity in RedHatAI/Meta-Llama-3-8B-Instruct-FP8 about 1 year ago

How to download the model with transformer library

#6 opened about 1 year ago by

New activity in mistralai/Pixtral-12B-2409 about 1 year ago

Update README.md

#25 opened about 1 year ago by

New activity in RedHatAI/Meta-Llama-3.1-70B-Instruct-FP8 about 1 year ago

Issue running on vLLM using FP8

#3 opened about 1 year ago by

New activity in RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 over 1 year ago

vllm says the requested model does not exist

#1 opened over 1 year ago by

New activity in RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 over 1 year ago

Storage format differs from other w4a16 models

#2 opened over 1 year ago by

New activity in RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 over 1 year ago

Model weights are not loaded

#3 opened over 1 year ago by

New activity in RedHatAI/Mistral-Nemo-Instruct-2407-FP8 over 1 year ago

Can not be inferenced with vllm openai server

#1 opened over 1 year ago by

New activity in RedHatAI/Meta-Llama-3-70B-Instruct-quantized.w4a16 over 1 year ago

Code example request with vllm

#1 opened over 1 year ago by

New activity in RedHatAI/Mistral-7B-Instruct-v0.3-GPTQ-4bit over 1 year ago

4bit quantisation does not reduce vram usage.

#2 opened over 1 year ago by

New activity in RedHatAI/Meta-Llama-3-70B-Instruct-FP8 over 1 year ago

How to run Meta-Llama-3-70B-Instruct-FP8 using several devices?

#3 opened over 1 year ago by

New activity in open-llm-leaderboard/open_llm_leaderboard over 1 year ago

Reproduction

#792 opened over 1 year ago by

New activity in RedHatAI/Meta-Llama-3-8B-Instruct-FP8 over 1 year ago

Fails to run with nm-vllm

#1 opened over 1 year ago by

New activity in RedHatAI/Llama-2-7b-ultrachat200k-pruned_50 over 1 year ago

Update chart template

#2 opened over 1 year ago by