Using DynamoRIO
(c) Copyright 2002-2005 HEWLETT-PACKARD COMPANY
(c) Copyright 2002-2005 Massachusetts Institute of Technology
Contents
What's in the Release Distribution
- Three different builds of the DynamoRIO library:
- Release build
- Debug build
- Profile build (the debug build, plus support for the
-prof_counts runtime option).
- A GUI for viewing statistics of programs running
under DynamoRIO control. The Windows GUI also has features for setting
runtime options, systemwide injection, and launching individual
programs.
- This documentation.
- Header files for the DynamoRIO client API.
- Sample uses of the API.
What Does and Doesn't Work
Remember: THIS IS BETA SOFTWARE!
It's not bulletproof.
Here are the platforms we support:
- Linux (tested on RedHat 7.2, RedHat Enterprise Linux WS 3, Fedora Core
2, and Fedora Core 3)
- Windows NT (tested on Workstation 4.0 and Server 4.0)
However, systemwide injection is not supported on Windows NT.
- Windows 2000 (tested on Professional and Server)
- Windows XP (tested on Professional), including SP2, although we do not
support Data Execution Prevention (DEP)
- Windows 2003
We do not currently support these program features (but we plan to in
the future, unless noted):
- 16-bit code (we do not plan to ever support 16-bit code)
- Exceptions or signals that examine the machine context for
more than the faulting instruction address may fail in certain rare
situations
- Any other program using the AppInit_DLLs registry key
(some anti-virus software uses it)
- Address Windowing Extensions on Windows (AllocateUserPhysicalPages,
MapUserPhysicalPages, and FreeUserPhysicalPages)
- Systemwide injection on Windows does not work on programs that do
not map in user32.dll
- On Windows a process can unload a dll or free memory in another
process' address space. We do not currently synchronize the code
cache with such an occurrence.
- Attaching a debugger to a process running under DynamoRIO does not
always work properly, especially on Windows. We recommend using
windbg from the Debugging
Tools for Windows rather than something like Visual Studio's
debugger.
Windows
Many large programs work fine under DynamoRIO, including Microsoft Office,
Adobe Acrobat Reader, Adobe Illustrator, Microsoft Accessories, Winzip,
Mozilla, Microsoft Internet Explorer, etc.
There are some programs that do not work due to bugs in
DynamoRIO. We are working on fixing these. Here is a list of programs
that do not work properly:
- Java, including applets loaded by internet browsers
- .NET
- Explorer.exe not caught by injector
Linux
Threads and signals are fully supported except for some corner cases:
- POSIX thread group signal semantics are not yet supported
- Threads suspending each other are not completely handled safely
- The -prof_pcs runtime option may cause applications
with their own signal-based profiling to fail
- The LD_PRELOAD injection method doesn't take over setuid/setgid
binaries. Applications that depend on executing such binaries, e.g. for printing, will
also fail.
A program running under DynamoRIO with the -stats option
(see Runtime Options) that crashes may leave its
shared memory containing statistics sitting around. Too many of these
can cause problems. They can be cleaned up using the ipcrm
command:
#!/bin/csh
foreach id (`ipcs -m |grep 0x0| awk '{print $2}'`)
ipcrm shm $id > /dev/null
end
Bugs other than those listed here should be reported to us.
Run All warning
Booting your machine with Run All turned on has the potential to crash
the operating system. To recover you need to clear the Microsoft\Windows
NT\CurrentVersion\Windows\AppInit_Dlls registry key, or delete the
DynamoRIO shared libraries.
Using the Windows GUI
The Windows GUI is useful for launching applications under DynamoRIO,
viewing statistics of currently running applications, and setting
DynamoRIO's runtime options.
The main portion of the GUI window displays statistics for the
currently selected process that is running under DynamoRIO. All such
processes that were run with the -stats option (see Runtime Options) are listed in the listbox at the top of
the window below the menus. The log file (see Runtime Options) for the currently running program
is displayed at the bottom of the window. Pressing the Explore
button will launch the Windows explorer at the directory containing the
log file, making it easier to view it and any
dynamorio_traces.log files.
The Run menu has two options: Run a single application, and Run All
(note that Run All is disabled on Windows NT). Running a single
application launches that application under the control of DynamoRIO.
Setting Run All sets the registry key that DynamoRIO uses to inject
itself into every new process that is created. Run All is
dangerous! We haven't tested DynamoRIO on every program, and it
does cause some programs to crash. Please be careful of what you run
while you have Run All turned on. When you exit the GUI while you have
Run All turned on it asks you if you want to turn it off, so you won't
accidentally leave it on.
The most-recently run applications are listed at the
bottom of the Run menu.
The Options menu allows for copying the displayed statistics to the
clipboard. It also can launch two dialog boxes: one for setting
DynamoRIO's runtime options, and another for setting the list of
programs not to run while Run All is checked. Both of
these are stored in environment variables (DYNAMORIO_OPTIONS and
DYNAMORIO_IGNORE, respectively). Changing the variables
locally will affect all programs launched through the GUI, but will not
affect programs run outside of the GUI! Also, when the GUI is
exited, the changes to the environment variables are lost. This is why
there is a "Set Permanently" button on each of the dialogs. This does
the same things as setting the variable using the Control Panel.
However, this still does not propagate the new variable to
already-running programs, unless they watch for notifications that
environment variables have changed (as the Windows Explorer does).
The Library menu selects which library is used for injection into
target applications. Note that the release library does
not support the -stats option (see Runtime Options) and so any program run under the
release build of DynamoRIO will not show up in the GUI's list of
processes.
Using the Linux GUI
Unlike the Windows GUI, the Linux GUI is only a statistics viewer.
Unfortunately we are only able to provide a version that requires various
gnome libraries and may not run on all Linux distributions.
Running DynamoRIO from the Command Line
On Windows, the GUI shows the command line it is using when it
launches each application. This is useful for determining the path to
the target application as well as how to invoke the DynamoRIO injector.
The injector's first argument must point to the DynamoRIO library to use
(remember, there are three versions of it in this release). The rest of
the arguments form the command line that would be used to launch the
target application natively. Here is an example of launching notepad
from bash inside a cygwin shell.
c:/dynamorio/bin/drinject.exe c:\\dynamorio\\bin\\debug\\dynamorio.dll c:\\WINNT\\system32\\notepad.exe
On Linux, we use the LD_PRELOAD environment variable to inject the
DynamoRIO library into a target process. The dynamorio script
can be used. It relies on the DYNAMORIO_HOME environment variable to
find the initial injection library. Which DynamoRIO library proper is
used can be specified by pointing to its containing directory with the
DYNAMORIO_SYSTEMWIDE environment variable, or by passing
-debug, -profile, or -release to the
dynamorio script. Here's an example of usage:
/usr/local/dynamorio/bin/dynamorio -profile mybenchmark args
Environment Variables
DynamoRIO uses the following environment variables:
-
DYNAMORIO_HOME
-
Points to the base of the DynamoRIO directory tree. Used by the Windows
GUI to find the injector and libraries, by the Linux dynamorio script to
find the libraries, and by the uninstaller.
-
DYNAMORIO_OPTIONS
-
Contains the runtime options used by DynamoRIO.
-
DYNAMORIO_SYSTEMWIDE
-
On Windows, this points to the library to use when injecting
systemwide. On Linux, this points to the directory containing the
library to use when injecting systemwide.
-
DYNAMORIO_IGNORE
-
Only used on Windows. Contains a semicolon-separated list of program
names not to run under DynamoRIO.
Example:
csrss.exe;winlogon.exe;emacs.exe;regedit.exe
Runtime Options
The environment variable DYNAMORIO_OPTIONS contains flags that
control the runtime behavior of DynamoRIO. The following is a list of
the available DYNAMORIO_OPTIONS for all builds:
-
-instrlibname S
-
The string S specifies the full pathname of the shared library
containing user-defined instrumentation routines. See the DynamoRIO API documentation for more
details.
-
-hot_threshold N
-
The value of N corresponds to the number of times a trace head must
be executed before it is used as the head of a trace fragment. By
default, the threshold for building traces is set to 50. A threshold
of 0 means that no traces are ever built.
- -cache_regen N
-cache_replace N
-
Controls the adaptive working set cache sizing algorithm (see Derek Bruening's PhD
thesis, Section 6.3.3). If -cache_regen is 0, the algorithm is
disabled (the cache grows unhindered, unless -cache_{bb,trace}_max is
set). If -cache_replace is 0, the cache will not grow at all beyond its
initial size. Otherwise, the cache is resized when -cache_regen fragments
out of every -cache_replace fragments replaced are regenerated. The
default values are -cache_regen 10 and -cache_replace 50.
- -cache_bb_max N
-cache_trace_max N
-
The value of N is the maximum size, in KB, of the basic block (or
trace) cache. The default value is 0, which implies an infinite
cache. The maximum cannot be set to a value less than 128KB.
-
-tracedump_text
-tracedump_binary
-
These options cause DynamoRIO to dump trace fragments to the log file
``traces.TID'' whenever they are flushed from the cache (by default, only
at program termination). Traces that are deleted to make room in the cache
during execution are also written here, with a note that they were flushed
and by what fragment. The two options select either a text dump or a
binary dump. The text dump takes up considerable room and time to dump!
Binary dump format:
* top of file:
int linkcount_size # 0, 4 (32-bit counters), or 8 (64-bit counters)
* each trace:
int frag_id; # always 0 for release build
app_pc tag;
int entry_offs;
int num_exits;
int code_size;
uint num_bbs;
if num_bbs > 0 # tracedump_origins
foreach bb:
app_pc tag;
int bb_code_size;
byte code[bb_code_size];
endif
foreach exit:
int exit_cti_offs;
int exit_stub_offs; # for separate stub, this is NOT within trace!
app_pc target;
bool linked; # bool is 4 bytes
if linkcount_size > 0
linkcount_type count; # sizeof == linkcount_size
endif
byte code[code_size];
See the provided sample binary trace dump
reader for an example of how to disassemble a binary trace dump file.
Printing symbolic information for addresses is not yet supported, but will
be in a future release.
-
-tracedump_origins
-
When selected by itself with neither -tracedump_text nor -tracedump_binary,
dumps only a text list of the constituent basic block tags of each trace to the
``traces.TID'' log file. When combined with either of -tracedump_text or
-tracedump_binary, adds a full disassembly of the constituent basic blocks
to the dump.
-
-prof_pcs
-prof_counts
-
These options turn on different types of profiling. Please see Profiling below for more information about each
kind of profiling. Note that -prof_counts is only available with the
profile build, and that -prof_pcs is only available on Linux.
-
-nolink
-
Do not allow any linking of fragments in the code cache. That is,
to go from one code fragment in the code cache to another, control
always returns to DynamoRIO first. This significantly slows down the
execution. The default is to allow the linking of code fragment exits
to code fragment entries.
-
-noasynch
-
Do not attempt to capture any asynchronous events. This option is
only valid on Windows operating systems and will cause DynamoRIO to miss any
code executed in callbacks, exception handlers, and asynchronous procedure
calls.
-
-nullcalls
-
This option turns dynamorio_app_init, dynamorio_app_start, and
dynamorio_app_stop into truly empty routines. This option can be used
to measure the run-time overhead of DynamoRIO. It is also useful for
debugging your application without DynamoRIO.
Options that are only available with debug builds:
-
-stats
-
This option causes DynamoRIO to export statistics about the running program to
shared memory. These statistics can be viewed by the GUI provided for
both Windows and Linux.
-
-loglevel N
-
If N is greater than 0, DynamoRIO will print out a log of its actions. The
greater the value of N, the more information DynamoRIO prints. Verbosity
is set to 0 by default, i.e., no log written. All log files are kept in a
log directory. There is one directory per address space per run. The
directories are named dynamorio.NNN, where NNN is a number that is
incremented with each directory created. There is one main log file per
directory named mainlog.TID, where TID is the thread id of the initial
thread. There is also a log file per thread, named log.TID. The loglevel
may be changed during program execution, but if it began at 0 then it
cannot be raised later.
-
-logmask 0xN
-
Selects which Dynamo/RIO modules print out logging information, at the
loglevel level. The mask is a combination of these bitfields:
LOG_NONE 0x0000
LOG_STATS 0x0001
LOG_TOP 0x0002
LOG_THREADS 0x0004
LOG_SYSCALLS 0x0008
LOG_ASYNCH 0x0010
LOG_INTERP 0x0020
LOG_EMIT 0x0040
LOG_LINKS 0x0080
LOG_CACHE 0x0100
LOG_FRAGMENT 0x0200
LOG_DISPATCH 0x0400
LOG_MONITOR 0x0800
LOG_HEAP 0x1000
LOG_VMAREAS 0x2000
LOG_ALL 0x3fff
It is easiest to set the mask using the Windows GUI's dialog box for
setting it. You can copy and paste the resulting mask value to set the
mask outside of the GUI.
-
-notify
-
This option causes a small number of status messages to be printed to
stderr. These include where the log directory is, a notice each time a new
log file is created, and notices when a fork or an execve occurs.
Profiling
DynamoRIO currently comes with two profiling options. The first,
selected with the -prof_pcs runtime
option, is a simple statistical sampler. (This option is currently
available only under Linux.) This option sets a timer to periodically
interrupt DynamoRIO and query which part of DynamoRIO was running. When
the program ends, a distribution of the samples is printed to the file
``pcsamples.TID''. (This option is currently available only
under Linux.) You should expect to see the majority of the samples
occurring in the fragment cache. The value of the pc each time the
timer went off is also recorded, and these pc's (along with
corresponding fragment tags and offsets) are also printed to the file
``pcsamples.TID''.
The other profiling option requires the use of the profile library. To
use it, set the -prof_counts runtime
option. This measures the execution count of each exit from each
trace. Its overhead is noticeable. The exit counts are printed out
with the traces in the file ``traces.TID'', so you must also select either
-tracedump_text or -tracedump_binary.
Application Interface
An alternative to running an entire application under DynamoRIO is to
use the application interface to specify a portion of the application to
run. This interface consists of the following routines:
-
dynamorio_app_init()
dynamorio_app_exit()
-
These functions perform the per-process initialization and
cleanuprequired by the DynamoRIO system. It must be called before the
application creates any threads or makes any other DynamoRIO API calls.
This function takes no parameters and returns 0 if the initialization
was successful.
-
dynamorio_app_start()
-
Execution of this function causes the thread of control to be
transfered to DynamoRIO. The program will appear to return from this call
with no visible side effects except those associated with the
invocation of an empty function. However, the running of the program
now occurs within DynamoRIO's code cache. As we describe below, you are
now able to observe, capture, and modify the application's code using
the hooks described later in this manual. Execution continues under
DynamoRIO until it encounters a call to dynamorio_app_stop. This function takes
no parameters and returns no values.
-
dynamorio_app_stop()
-
If the program is running under DynamoRIO control, it will again run
directly on the machine upon invocation of this routine. If the
program was already running under its own control, this call has no
effect on the application's state. This function takes no parameters
and returns no values.
-
dynamorio_app_take_over()
-
Calling this routine is similar to dynamorio_app_start except that all
subsequent dynamorio_app_ calls are ignored. This is useful to override
existing dynamorio_app_start() and dynamorio_app_stop() calls.
Example
The following program prints ``Hello world!'' under DynamoRIO control:
#include
#include
#include "dynamorio.h"
main()
{
int rc = dynamorio_app_init();
assert(rc == 0);
dynamorio_app_start();
printf("Hello world!\n");
dynamorio_app_stop();
dynamorio_app_exit();
}
You simply compile this application and link it with the DynamoRIO
library corresponding to your current execution environment (Linux or
Win32). On Linux, you must also link with the libc and libdl
libraries. When you execute this application, you shouldn't notice
any difference, except that it might run a bit slower. Since there is next
to zero reuse in this simple application, running the application and
its library routines in DynamoRIO's code cache will slow down the
application. If you wrap dynamorio_app_start and dynamorio_app_stop around the
frequently executed portions of your application (those with lots of
reuse), you should observe very little overhead.
Documentation Home