We see different numbers for what percentage of machine learning models never make it into production. I do not want to give an exact number but let’s just say most machine learning models fail to be productionalized.
There might be several different reasons for not being able to take the final step and deploy.
For instance, the operation might be too expensive in terms of the computation and storage compared to the value the model provides. Or, the results and performance dramatically worsen when the model is scaled.
We might also fail to deploy the model into production properly. This is an operational issue, which can be addressed and solved by implementing a robust MLOps system.
MLOps refers to the process of building a system with many different types of operations that are synched and work in tandem.
Just like DevOps is a common practice for creating large-scale software systems, MLOps is required for efficient machine learning systems.
A data scientist can create and evaluate a machine learning model using a sample dataset in a Jupyter notebook.
However, when it comes to converting this model in the Jupyter notebook to a ML system that is continuously operated in production, there are other roles that need to be involved.
In this article, we will talk about what I think is the most critical role for creating a robust, lucrative, and beneficial ML system: Domain expert.
Domain experts
These are the people who know the business. They are not necessarily data-oriented but provide highly valuable feedback as to how data can make a difference in that particular domain.
Data-oriented people in an ML system (e.g. data scientist, data analyst, data engineer) cannot be expected to have a comprehensive understanding of a business, which typically takes years of experience in a specific domain.
Domain experts serve as a bridge between the data-oriented people and the business. Information flows through both ways on this bridge in the form of insight, feedback, or action.
Consider a data science team trying to create a forecasting system for a grocery retailer with many stores.
How to start
ML systems should start with domain experts. They help define the business goals and key performance indicators (KPIs), which are of crucial importance to measure the performance of an ML system.
Domain experts know what the business needs. They help address the problems that need to be solved and the processes that have the potential to be improved.
Data scientists or machine learning engineers dig into the data to extract insights, find patterns, and relationships. However, they need to be directed to the problems with a larger impact or business value. This is something domain experts can help with.
How to end
Let’s say data-oriented professionals and domain experts sit together and have long sessions of brainstorming. They frame the problem and ways to tackle it.
The ML system is designed and implemented. Then, we start seeing the results. The metrics seem to be fine, at least based on the pre-defined KPIs.
However, we are still away from making a business impact. For instance, the problem might be churn prediction for a bank. From a data scientist’s perspective, the solution is just fine as long as the evaluation metrics are aligned with predefined values.
The main goal of an ML system is not meeting the metrics though. What matters is the business value created. Therefore, domain experts should get involved in such scenarios.
They make the information flow between the team responsible for the ML system and the business-oriented departments such as customer service and marketing. This information flow mainly consists of iterative feedback with a goal of making the ML system in production create value.
Final words
An ML system consists of many building blocks, which is why there is an entire discipline created to make it work: MLOps.
The professions involved in the MLOps might change depending on the organization or business. The typical roles are data scientist, data engineer, software engineer, machine learning engineer/architect, and domain expert.
The job of each role is, of course, crucial but I feel like the importance of domain expertise is sometimes undervalued. An ML system without domain experts or people that understand the business very well is likely to fail.
You can become a Medium member to unlock full access to my writing, plus the rest of Medium. If you already are, don’t forget to subscribe if you’d like to get an email whenever I publish a new article.
Thank you for reading. Please let me know if you have any feedback.
