We just have to hope that the real world behaves exactly like the simulation.
But the magic isn't in the graphics; it's in the scenarios . google driving simulator
In the simulation, the rain is perfect. It follows a mathematical curve. In the real world, rain splashes, dirt smears on the lens, and humidity fogs the sensor glass. In the simulation, pedestrians move with biomechanical logic. In the real world, a pedestrian might be carrying a mirror that reflects the lidar back into the sensor, creating a "ghost" object. We just have to hope that the real