Systems and Means of Informatics

2022, Volume 32, Issue 4, pp 14-20

DETECTION OF DISTRIBUTION DRIFT

  • A. A. Grusho
  • N. A. Grusho
  • M. I. Zabezhailo
  • D. V. Smirnov
  • E. E. Tmonina
  • S. Ya. Shorgin

Abstract

Changing the properties of the data being collected is often referred to as data drift (various options for shifting the characteristics of the data).
The existence of drift in artificial intelligence system training data often leads to a decrease in the efficiency of machine learning and erroneous solutions of artificial intelligence systems built on these data. In this regard, the problems of detecting drift in machine learning data, the moment of drift formation, and the consequences of changes in training data become relevant. The work proposes a method for detecting the drift of a probability distribution in an arbitrary metric space of large dimension. The method relies on the difference between unknown probability distributions in different regions of the original space in the event of drift. A drift model consisting of two different probability distributions is considered. Using the balls in metric space as the basis of the method allows one to create an efficient algorithm for calculating the ownership of data points to one of the balls associated with different distributions of the drift model.
This circumstance seems to be essential for revealing the drift of a distribution in a high-dimensional space.

[+] References (7)

[+] About this article