The Research of Detecting Shellcodes through Microsoft Windows API Hooking

碩士 === 國立清華大學 === 資訊工程學系 === 94 === Nowadays, there is a growing trend of attacks using vulnerabilities to compromise victim's computer system. Attackers often write shellcodes through these vulnerabilities to compromise the victim's system. These shellcodes are a kind of "harmful cod...

Full description

Bibliographic Details
Main Authors: Kang-Chiao Lin, 林港喬
Other Authors: Hung-Min Sun
Format: Others
Language:en_US
Published: 2006
Online Access:http://ndltd.ncl.edu.tw/handle/18272792227945168598
Description
Summary:碩士 === 國立清華大學 === 資訊工程學系 === 94 === Nowadays, there is a growing trend of attacks using vulnerabilities to compromise victim's computer system. Attackers often write shellcodes through these vulnerabilities to compromise the victim's system. These shellcodes are a kind of "harmful code", and use the specific Windows APIs to destroy the system or disclose sensitive data. However, the shellcodes should get the APIs’ addresses before they use it. In this thesis, we proposed an effective system using API hooking and finite state machine to detect shellcodes. Our system detects the shellcodes by monitoring the Windows APIs detect the shellcodes through the API hooks. We use a finite state machine (FSM) to evaluate the behavior of shellcodes. In Recoding Phase, the FSM stores the feature codes and compare in Monitor Phase. If the feature codes mismatch, our system detects shellcodes. Furthermore, our system is low cost and convenient since it does not need hardware support and source code of programs.