On the Scalability of Static Program Analysis to Detect Vulnerabilities in the Java Platform

Java has been a target for many zero-day exploits in the past years. We investigate one category of vulnerabilities used by many of these exploits. Attackers make use of so called unguarded caller-sensitive methods. While these methods provide features that can be dangerous if used in malicious ways...

Full description

Bibliographic Details
Main Author: Lerch, Johannes
Format: Others
Language:en
Published: 2016
Online Access:http://tuprints.ulb.tu-darmstadt.de/5580/1/document.pdf
Lerch, Johannes <http://tuprints.ulb.tu-darmstadt.de/view/person/Lerch=3AJohannes=3A=3A.html> : On the Scalability of Static Program Analysis to Detect Vulnerabilities in the Java Platform. Technische Universität Darmstadt, Darmstadt [Ph.D. Thesis], (2016)
Description
Summary:Java has been a target for many zero-day exploits in the past years. We investigate one category of vulnerabilities used by many of these exploits. Attackers make use of so called unguarded caller-sensitive methods. While these methods provide features that can be dangerous if used in malicious ways, they perform only limited permission checks to restrict access by untrusted code. We derive a taint-analysis problem expressing how vulnerabilities regarding these methods can be detected automatically in the Java Class Library before its code is being released to the public. Unfortunately, while describing the analysis problem is relatively simple, it is challenging to actually implement the analysis. The goal of analyzing a library of the size as the Java Class Library raises scalability problems. Moreover, analyzing a library while assuming attackers can write arbitrary untrusted code results in mostly all parts of the library being accessible. Most existing approaches target the analysis of an application, which is less of a problem, because usually only small parts of the library are used by applications. Besides the fact that existing algorithms run into scalability problems we found that many of them are also not sound when applied to the problem. For example, standard call-graph algorithms produce unsound call graphs when only applied to a library. While the algorithms provide correct results for applications, they are also used when only a library is analyzed---the incompleteness of the results is then usually ignored. The requirements for this work do not allow to ignore that, as otherwise security-critical vulnerabilities may remain undetected. In this work we propose novel algorithms addressing the soundness and scalability problems. We discuss and solve practical challenges: we show a software design for the analysis such that it is still maintainable with growing complexity, and extend an existing algorithm to enrich results with exact data-flow information enabling comprehensible reporting. In experiments we show that designing the analysis to work forward and backward from inner layers to outer layers of the program results in better scalability. We investigate the challenge to track fields in a flow-sensitive and context-sensitive analysis and discuss several threats to scalability arising with field-based and field-sensitive data-flow models. In experiments comparing these against each other and against a novel approach proposed in this work, we show that our new approach successfully solves most of the scalability problems.