Search code, repositories, users, issues, pull requests...
The nnScaler is a system that takes a DNN model that designed for single device, e.g., GPU, and automatically convert it into the program that can execute concurrently on multiple devices.
Under the hood, nnScaler analyzes the given DNN models, plans for appropriate parallelization strategies, and generates corresponding execution code. With nnScaler, users can focus on single device DNN model design, and offload the complex parallelization work to nnScaler, and easily achieve high performance parallel DNN execution.
Due to high compatibility and extensibility, nnScaler can be used for innovation of a wide range of new DNN models and DNN systems, including new model structure, training patterns, as well as new parallelization techniques that are beyond existing data-parallelism, tensor-parallelism or pipeline parallelism.
For execution performance, nnScaler can support new parallelisms that outperform existing parallel execution approaches: 1) fitting larger DNN model given the same hardware; 2) providing faster execution for the same model on the same hardware. (included in our OSDI’24 paper) For compatibility, nnScaler can support paralleling new DNN models by providing user-defined functions (a few lines of code) for the new operators unrecognized by the nnScaler.
