kaitchup

/

Olmo-3.1-32B-Instruct-fp8-dynamic

like 0

Follow

The Kaitchup 84

Hi, I got this error running the model. Any ideas?

(APIServer pid=21459) pydantic_core._pydantic_core.ValidationError: 2 validation errors for VllmConfig
(APIServer pid=21459) scale_dtype
(APIServer pid=21459) Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
(APIServer pid=21459) For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden
(APIServer pid=21459) zp_dtype
(APIServer pid=21459) Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
(APIServer pid=21459) For further information visit https://errors.pydantic.dev/2.12/v/extra_forbidden

Command:
VLLM_USE_V1=1 CUDA_VISIBLE_DEVICES=0,1 CUDA_DEVICE_ORDER=PCI_BUS_ID vllm serve kaitchup/Olmo-3.1-32B-Instruct-fp8-dynamic --host 0.0.0.0 --port 8000 --max-model-len 32000 --max-num-batched-tokens 2048 --enable-chunked-prefill --max-num-seqs 16 --gpu-memory-utilization 0.95 --dtype auto --tensor-parallel-size 2 --tool-call-parser olmo3 --enable-auto-tool-choice --quantization compressed-tensors