Modeling security violation processes in machine learning systems

• Main Authors: Maxim A. Chekmarev, Stanislav G. Klyuev, Viktor V. Shadskiy
• Format: Article
• Language: English
• Published: Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University) 2021-08-01
• Series: Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki
• Subjects: machine learning; security breach; integrity; confidentiality; functional modeling; logical probabilistic modeling
• Online Access: https://ntv.ifmo.ru/file/article/20592.pdf

The widespread use of machine learning, including at critical information infrastructure facilities, entails risks of security threats in the absence of reliable means of protection. The article views the processes in machine learning systems as the ones occurring in information systems susceptible to malicious influences. The results of modeling events leading to a security breach in machine learning systems operating at critical information infrastructure facilities are presented. For modeling, the technology of creating functional models SADT (Structured Analysis and Design Technique) and the IDEF0 (Integration definition for function modeling) methodology were used as a tool for transition from a verbal functional description of the process under study to a description in terms of mathematical representation. In order to study the scenarios of the transition of machine learning systems to a dangerous state and the numerical assessment of the probability of security violation, mathematical modeling of threats was carried out using the logical-probabilistic method. The authors obtained a visual functional model of system security violation in the form of a context diagram of the system and two levels of decomposition. The hazard function of the system is determined and the arithmetic polynomial of the probability function is derived. In further work the described models will allow researchers to develop methods and algorithms for protecting machine learning systems from malicious influences, as well as to apply them in assessing the level of security.