ML Ops tools for NLP

The typical and most traditional way to do Machine Learning is basically a manual approach where the data scientist will conduct cycles of experimentation (EDA, pre-processing, training, validation…) to get a model that has a good predictive performance based on relevant and consistent hypotheses and validation metrics. Once we have such a model, the role of the data scientist ends, and it’s the engineers who will take it from here to deploy that model as a prediction microservice with a REST API, for example, so that we can make predictions in a production environment.

Once we deploy this model in a production environment, and like any environment, there is a tendency for it to change dynamically, and therefore the data regarding that environment will change, and here there will be a misalignment between the hypothesis and metrics used by the data scientists and the new data relative to the environment, as a result, the model will break when it fails to adapt to the changes in the real world.

We can note here that this way of doing ML is adapted to environments with limited and slow changes where they either did not require the model to be adapted to new data or require extremely infrequent fine-tuning (for example, once a year) and it will be the same process where the data scientist will do the work and then the engineer will entirely deploy the new model.

To solve this problem, as well as other challenges related to this approach, such as the tedious manual tasks of data acquisition, model training, testing, building, and deployment… the MLOps is increasingly adopted by companies with a fast-changing environment and large-scale applications.

MLOps is derived from DevOps, hence its name, however it incorporates some unique properties related to the nature of a data scientist’s work.

First, let’s start with the simplest kind of MLOps. In the traditional approach, we deploy a trained model as a prediction service in production. But now, we will deploy a full training pipeline, which runs automatically and in a triggered manner to provide a trained model as a prediction service responsive to emerging changes in the data (automatically gathered data).
This method is called Continuous Training, where the deployed pipeline is defined by the data scientist who will work with the data engineers to convert it into reproducible scripts.
Thus, this type of basic MLOps guarantees that the model is automatically trained in production using new data, that there is Model Continuous Delivery (automatically trained model deployment steps are automated), that experiment results are tracked, and that the training code and models are well versioned.

Secondly, as we automated the model training and delivery to fit the new data by deploying the entire training pipeline, you might now be wondering:
What if we needed to update the pipeline to fit the new business needs?
What if using a new algorithm and a new set of parameters would result in a pipeline that delivers better models?
So, the data scientist here will iteratively try new ML algorithms and techniques, which will result in the source code for the ML pipeline steps, and then we need to build that source code and run various tests related to the new pipeline updates, which is known as Pipeline Continuous Integration, resulting in a pipeline to be deployed in the target environment, and this is the Pipeline Continuous Delivery, the result of this step is a deployed pipeline with the new implementation.

Check out this Microsoft article to learn more about MLOps maturity levels.

Note: we need to collect real-time statistics on the model’s performance to decide if we should run the training pipeline or perform a new experimental cycle to update the pipeline (Monitoring).