Blog

Our previous post introduced the research vehicle we will use for real-world testing, helping us understand and account for hardware-related challenges such as sensor noise and calibration. However, systematic progress also requires experiments that other researchers can repeat. This is the motivation behind the common task framework: shared development data, a defined evaluation protocol, held-out test data, and a public leaderboard. Researchers evaluate their methods under a set of common rules on examples withheld from development and openly share their findings. This approach has helped organize and accelerate research in vision, speech, and language since the 1980s. We now aim to bring that same discipline to the development of frontier models for real-world end-to-end autonomous driving.

Full self-driving is still unsolved. We have been working on this problem for more than fifteen years, and for the last decade most of that work has happened in simulation. Today that changes: our research vehicle arrived, and we can start testing our ideas where they actually have to hold, in the physical world. Our new test platform is a VW ID.7 carrying ten cameras, five LiDARs and six radars, deliberately overequipped so that we can develop world models and policies that generalize across embodiments and measure what each sensor actually contributes.

For years the target point bias was an open secret in the CARLA end-to-end driving community: the best driving policies scored well in part by steering toward the next GPS waypoint rather than by understanding the traffic and road layout around them. CARLA, the most established simulator for autonomous driving, has long served as the initial proving ground where new self-driving ideas are validated before the industry takes them up. However, the target point, which is intended as a proxy for a coarse GPS-based navigation system in the simulator, is in practice implemented as a precise coordinate along the lane center and provided as an input signal to end-to-end driving policies, enabling shortcut learning.

Kyutai and ELLIS Tübingen announce the official launch of KE:SAI – Kyutai ELLIS Scalable Autonomous Intelligence. KE:SAI is a non-profit open science research laboratory dedicated to the next frontier of artificial intelligence: systems that can understand and act in the real world. KE:SAI brings together two of Europe’s most prominent AI brands to address the fundamental challenges of world models and physical AI. Founded in Paris in 2023, Kyutai has emerged as one of Europe’s leading AI laboratories, with a focus on breakthrough research with immediate and lasting impact, and a team driven by a shared commitment to openness and excellence.