Protecting Against Unexpected System Calls
C. M. Linn,
M. Rajagopalan,
S. Baker,
C. Collberg,
S. K. Debray,
J. H. Hartman
Department of Computer Science
University of Arizona
Tucson, AZ 85721, U.S.A.
Abstract
This paper proposes a comprehensive set of techniques which limit the scope of
remote code injection attacks. These techniques prevent any injected code from
making system calls and thus restrict the capabilities of an attacker. In
defending against the traditional ways of harming a system these techniques
significantly raise the bar for compromising the host system forcing the
attack code to take extraordinary steps that may be impractical in the
context of a remote code injection attack.
There are two main aspects to our approach.
The first is to embed semantic information
into executables identifying the locations of legitimate system call
instructions; system calls from other locations are treated as intrusions.
The modifications we propose are transparent to user level processes that do
not wish to use them (so that, for example, it is still
possible to run unmodified third-party software), and add more security at
minimal cost for those binaries that have the special information present.
The second is to back this up using a variety of techniques, including a novel
approach to encoding system call traps into the OS kernel, in order to deter
mimicry attacks. Experiments indicate that our approach is effective against
a wide variety of code injection attacks.