Incremental Adaptive Learning and Alert Grouping for False Alarm Reduction in Intrusion Detection

• Main Authors: Heng-Sheng Lin, 林恆生
• Other Authors: Hahn-Ming Lee
• Format: Others
• Language: en_US
• Published: 2007
• Online Access: http://ndltd.ncl.edu.tw/handle/u9s78g

碩士 === 國立臺灣科技大學 === 資訊工程系 === 95 === As applications relying on network become increasingly diverse in commerce, governments, organizations and social network communities, attempts to compromise those services or steal sensitive information have become increasingly sophisticated. Consequently, Intrusion Detection Systems (IDSs) have been adopted as an essential protection method. However, IDSs have many side effects, particularly the large number of false alarms, which cause irrelevant information covering relevant alarms. Hence, the analysts and network administrators waste considerable time discovering relevant alarms. This study presents a system for providing organized information, including the predicted class, which labels an alarm into relevant or irrelevant one; the group information, which represents a single event, grouping those redundant or similar alarms, and useless statistical information, a rank list of statistic of valueless signatures which helps analysts tuning the rules of their signature-based IDS. Additionally, two algorithms related to machine learning and data mining are proposed in our system. The first one is the Incremental Adaptive Concept Learning (IACL) algorithm, which is adopted to train the committee classifier that categorizes the incoming alarms as relevant and irrelevant. Capable of incrementally learning new knowledge and adapting to changing target concepts, the algorithm is a continuously learning method, meaning that the model is trained by recently collected data without considering entirely accumulative data; this approach is more practical in on-line operation than ideal case, re-training underlying model with entire accumulation of recorded data in each time of invoking learning. The second algorithm, On-line Alert Grouping (OAG) algorithm, is designed to reduce the amount of redundant alarm information by grouping the similar or repetitive alarms into a single alarm group referring to a single event. Moreover, experimental results demonstrate that our IACL algorithm performs better in terms of accuracy and resources than combining all of trained models and only keeping the last learned model after each invoked learning process. In particular, the proposed learning model has a better average accuracy than others tested, revealing that it has better stability. Finally, On-line operation requirements, such as limited resources, are also considered.