A Real-Time Hidden Anomaly Detection of Correlated Data in Wireless Networks

• Main Authors: Tengfei Sui, Xiaofeng Tao, Shida Xia, Hui Chen, Huici Wu, Xuefei Zhang, Kechen Chen
• Format: Article
• Language: English
• Published: IEEE 2020-01-01
• Series: IEEE Access
• Subjects: Anomaly detection; random matrix theory; data decomposition; network traffic prediction; big data; 5G and beyond
• Online Access: https://ieeexplore.ieee.org/document/9050719/

Wireless networks have been generating a plethora of unstructured and highly-correlated big data with hidden anomalies. The anomalies may bring inaccurate predictions of network behaviors, which further lead to inefficient system designs such as proactive caching placement. Current Random Matrix Theory (RMT) approaches are unable to detect hidden anomalies with a satisfying tolerance of data correlation. We present a novel data Decomposition aided Random Matrix Theory (DC-RMT) framework, which enables a real-time anomaly detection of large scale multi-dimensional and highly-correlated data. The detection results show that the proposed DC-RMT methodology can detect anomalies with an accuracy of 28 times better than RMT applied without data decomposition. The prediction results present a 6 times higher accuracy than data with anomaly, which will facilitate the identification of regions of interests, and contribute to the improvement of resource allocation efficiency and user QoE.