pcuenq/coco-2017-mirror
Preview
• Updated
• 141 • 1
Pedro Cuenca PRO
pcuenq
·
AI & ML interests
None yet
Recent Activity
updated
a dataset 41 minutes ago
huggingface-projects/drlc-leaderboard-data upvoted an article 1 day ago
Bringing Autonomous Driving RL to OpenEnv and TRL reacted
to
sergiopaniego's
post with 🚀 1 day ago
What happens when you make an LLM drive a car where physics are real and actions can't be undone?
I ported CARLA, the autonomous driving simulator, to OpenEnv and added training support via TRL + Hugging Face Spaces.
The model interacts with the simulator through tool calls (observe, brake, change lane) and learns from a reward signal.
In 50 training steps, Qwen 0.6B learns to swerve and brake to avoid pedestrians in emergency situations.
The project supports text and vision (VLMs can see through a camera sensor), open-world driving with traffic, and multiple driving scenarios.
This builds on the carla-env project by sinatras, which originally placed LLMs inside CARLA for evaluation. We extended it with vision, new scenarios, rubric-based rewards, and made it trainable end-to-end.
Blog: https://huggingface.co/blog/sergiopaniego/bringing-carla-to-openenv-trl/
CARLA env in OpenEnv: https://github.com/meta-pytorch/OpenEnv/tree/main/envs/carla_env
Training script: https://github.com/huggingface/trl/blob/main/examples/scripts/openenv/carla.py Organizations
