ProjectsThe Deep Learning Model Serving (DELOS) System
The development and training of deep learning models are perceived as challenging tasks requiring abundant hard-to-have talent and expertise. However, the greater challenge is the model serving phase, where the model is actually deployed and performing inference in production. This is where the best talent is needed to prevent many aspects from going astray, causing negative consequences that may have a much deeper business impact.
Northwestern University’s Center for Deep Learning is developing a serving system addressing the needs of deep learning models. The DEep Learning mOdel Serving (DELOS) system is based on Kubeflow and thus Kubernetes, an open-source system for automating deployment, scaling, and management of containerized applications.
The objectives of DELOS are adding the following modules to Kubeflow:
- A module for monitoring KPIs together with algorithms to trigger alerts, and, more importantly, to automatically start retraining of the underlying deep learning model
- A component to assess and monitor confidence of predictions during model serving
- A TensorFlow and PyTorch modules to efficiently retrain a deep learning model
- A module to automate the serving-to-training data transfer and processing including possible changes to feature engineering
- A module checking data quality changes, feature importance adjustments, and data covariate shifts in the deep learning context
- A module to transfer the data from serving to training
- A module enabling integration of new data sources to serving.
Become a Member
If you are interested in participating, influencing, and benefiting from DELOS by means of becoming a member of CDL, please email us at cdl@northwestern.edu or