Uber uses machine learning (ML) models to power critical business decisions. An ML model goes through many experiment iterations before making it to p

Tuning Model Performance | Uber Engineering Blog

submited by
Style Pass
2021-07-09 17:30:08

Uber uses machine learning (ML) models to power critical business decisions. An ML model goes through many experiment iterations before making it to production. During the experimentation phase, data scientists or machine learning engineers explore adding features, tuning parameters, and running offline analysis or backtesting. We enhanced the platform to reduce the human toil and time in this stage, while ensuring high model quality in production.

Searching through a large hyperparameter search space is expensive due to the curse of dimensionality , which holds true even for Bayesian optimization techniques that are able to balance explore vs. exploit and incorporate belief updates via a posterior. Hyperparameter importance and selection can be important to help reduce the complexity of the search space as certain hyperparameters have a larger impact on model performance. We have found that for tree-based models such as XGBoost , tuning the maximum tree depth, maximum tree count, learning rate, and minimum child weight etc. tend to yield more performance than others. 

The goal is to proactively surface the appropriate parameters for known classes of problems, saving time overall. In addition to optimizing AUC for binary classification problems or mean-square-error (MSE) for regression problems, we expose additional objective functions so that data scientists and machine learning engineers can choose appropriate optimization goals for their problems. To prevent overfitting , we introduced a “penalty” term into the optimization function, which captures the difference between training and test performance. To speed up hyperparameter search for extremely large datasets, we allow early stopping the Hyperparameter Optimization study, if we do not see significant improvements with new trials.

Leave a Comment