This release of openpilot includes a major architecture change to the driving models (#30392). We changed the architecture from using EfficientNets to using FastViTs. These models use a Hybrid Vision Transformer architecture and perform much better in all our tests. This is the biggest improvement we’ve shipped since the 0.9.0 model redesign.
The Non-Inflatable (#29003) model inputs navigation instructions in addition to the navigation features into the driving model. Navigation instructions are encoded as a ternary (left, straight, right) vector of length 50 (from -500 to +500 meters at a resolution of 20 meters). Adding this input improves Navigate on openpilot performance, especially when the map image is not enough to determine the maneuver type and its distance to ego.
Before this release, openpilot lateral and longitudinal planning relied on MPCs and some hand-coded logic to transform the model-predicted plan into an executable and smooth trajectory. With the New Lemon Pie model (#30209), we moved the lateral MPC and planner completely to our ground truth stack and inside the model, which means that the model directly outputs an executable lateral trajectory. In the next releases, we will do the same for longitudinal planning. Doing the planning as part of the model is an important step towards making the driving models more end-to-end.