Model Drift or Concept Drift has been a popular topic for the past few years. One key driven factor is the COVID-19. However, it’s a envolving world, the environment is changing all the time. Most companies who has a ML or AI product cares about if it’s producing degraded predictions.
Let’s first clarify some key concepts mentioned in the paper.
Real concept drift: The relationships between X (features) and y (target variables) have changed compared to training time.
Virtual drift: he relationships between X (features) and y (target variables) are still valid but the distribution of feature X is different than training time. Therefore, the predicted y hat is also different than training time.
Only the changes that affect the prediction decisions, in this case the real concept drift, require adaptation.
Adaptive learning algorithms
can be seens as advanced incremental learning algorithms that are able to adapt to evolution of the data generating process over time.
- Requirements for predictive models in changing environments
We can group applications into 4 cateogries based on how they handle concept drifts.
- Monitoring and control
- xx
- xx
- xx
Adaptive learning systems