Using DynamoRIO

(c) Copyright 2002-2005 HEWLETT-PACKARD COMPANY
(c) Copyright 2002-2005 Massachusetts Institute of Technology

Contents


What's in the Release Distribution


What Does and Doesn't Work

Remember: THIS IS BETA SOFTWARE! It's not bulletproof.

Here are the platforms we support:

We do not currently support these program features (but we plan to in the future, unless noted):

Windows

Many large programs work fine under DynamoRIO, including Microsoft Office, Adobe Acrobat Reader, Adobe Illustrator, Microsoft Accessories, Winzip, Mozilla, Microsoft Internet Explorer, etc.

There are some programs that do not work due to bugs in DynamoRIO. We are working on fixing these. Here is a list of programs that do not work properly:

Linux

Threads and signals are fully supported except for some corner cases:

A program running under DynamoRIO with the -stats option (see Runtime Options) that crashes may leave its shared memory containing statistics sitting around. Too many of these can cause problems. They can be cleaned up using the ipcrm command:

#!/bin/csh
foreach id (`ipcs -m |grep 0x0| awk '{print $2}'`)
  ipcrm shm $id > /dev/null
end

Bugs other than those listed here should be reported to us.

Run All warning

Booting your machine with Run All turned on has the potential to crash the operating system. To recover you need to clear the Microsoft\Windows NT\CurrentVersion\Windows\AppInit_Dlls registry key, or delete the DynamoRIO shared libraries.


Using the Windows GUI

The Windows GUI is useful for launching applications under DynamoRIO, viewing statistics of currently running applications, and setting DynamoRIO's runtime options.

The main portion of the GUI window displays statistics for the currently selected process that is running under DynamoRIO. All such processes that were run with the -stats option (see Runtime Options) are listed in the listbox at the top of the window below the menus. The log file (see Runtime Options) for the currently running program is displayed at the bottom of the window. Pressing the Explore button will launch the Windows explorer at the directory containing the log file, making it easier to view it and any dynamorio_traces.log files.

The Run menu has two options: Run a single application, and Run All (note that Run All is disabled on Windows NT). Running a single application launches that application under the control of DynamoRIO. Setting Run All sets the registry key that DynamoRIO uses to inject itself into every new process that is created. Run All is dangerous! We haven't tested DynamoRIO on every program, and it does cause some programs to crash. Please be careful of what you run while you have Run All turned on. When you exit the GUI while you have Run All turned on it asks you if you want to turn it off, so you won't accidentally leave it on.

The most-recently run applications are listed at the bottom of the Run menu.

The Options menu allows for copying the displayed statistics to the clipboard. It also can launch two dialog boxes: one for setting DynamoRIO's runtime options, and another for setting the list of programs not to run while Run All is checked. Both of these are stored in environment variables (DYNAMORIO_OPTIONS and DYNAMORIO_IGNORE, respectively). Changing the variables locally will affect all programs launched through the GUI, but will not affect programs run outside of the GUI! Also, when the GUI is exited, the changes to the environment variables are lost. This is why there is a "Set Permanently" button on each of the dialogs. This does the same things as setting the variable using the Control Panel. However, this still does not propagate the new variable to already-running programs, unless they watch for notifications that environment variables have changed (as the Windows Explorer does).

The Library menu selects which library is used for injection into target applications. Note that the release library does not support the -stats option (see Runtime Options) and so any program run under the release build of DynamoRIO will not show up in the GUI's list of processes.

Using the Linux GUI

Unlike the Windows GUI, the Linux GUI is only a statistics viewer. Unfortunately we are only able to provide a version that requires various gnome libraries and may not run on all Linux distributions.


Running DynamoRIO from the Command Line

On Windows, the GUI shows the command line it is using when it launches each application. This is useful for determining the path to the target application as well as how to invoke the DynamoRIO injector. The injector's first argument must point to the DynamoRIO library to use (remember, there are three versions of it in this release). The rest of the arguments form the command line that would be used to launch the target application natively. Here is an example of launching notepad from bash inside a cygwin shell.

c:/dynamorio/bin/drinject.exe c:\\dynamorio\\bin\\debug\\dynamorio.dll c:\\WINNT\\system32\\notepad.exe

On Linux, we use the LD_PRELOAD environment variable to inject the DynamoRIO library into a target process. The dynamorio script can be used. It relies on the DYNAMORIO_HOME environment variable to find the initial injection library. Which DynamoRIO library proper is used can be specified by pointing to its containing directory with the DYNAMORIO_SYSTEMWIDE environment variable, or by passing -debug, -profile, or -release to the dynamorio script. Here's an example of usage:

/usr/local/dynamorio/bin/dynamorio -profile mybenchmark args


Environment Variables

DynamoRIO uses the following environment variables:
DYNAMORIO_HOME
Points to the base of the DynamoRIO directory tree. Used by the Windows GUI to find the injector and libraries, by the Linux dynamorio script to find the libraries, and by the uninstaller.
DYNAMORIO_OPTIONS
Contains the runtime options used by DynamoRIO.
DYNAMORIO_SYSTEMWIDE
On Windows, this points to the library to use when injecting systemwide. On Linux, this points to the directory containing the library to use when injecting systemwide.
DYNAMORIO_IGNORE
Only used on Windows. Contains a semicolon-separated list of program names not to run under DynamoRIO. Example:
csrss.exe;winlogon.exe;emacs.exe;regedit.exe

Runtime Options

The environment variable DYNAMORIO_OPTIONS contains flags that control the runtime behavior of DynamoRIO. The following is a list of the available DYNAMORIO_OPTIONS for all builds:

-instrlibname S
The string S specifies the full pathname of the shared library containing user-defined instrumentation routines. See the DynamoRIO API documentation for more details.
-hot_threshold N
The value of N corresponds to the number of times a trace head must be executed before it is used as the head of a trace fragment. By default, the threshold for building traces is set to 50. A threshold of 0 means that no traces are ever built.
-cache_regen N
-cache_replace N
Controls the adaptive working set cache sizing algorithm (see Derek Bruening's PhD thesis, Section 6.3.3). If -cache_regen is 0, the algorithm is disabled (the cache grows unhindered, unless -cache_{bb,trace}_max is set). If -cache_replace is 0, the cache will not grow at all beyond its initial size. Otherwise, the cache is resized when -cache_regen fragments out of every -cache_replace fragments replaced are regenerated. The default values are -cache_regen 10 and -cache_replace 50.
-cache_bb_max N
-cache_trace_max N
The value of N is the maximum size, in KB, of the basic block (or trace) cache. The default value is 0, which implies an infinite cache. The maximum cannot be set to a value less than 128KB.
-tracedump_text
-tracedump_binary
These options cause DynamoRIO to dump trace fragments to the log file ``traces.TID'' whenever they are flushed from the cache (by default, only at program termination). Traces that are deleted to make room in the cache during execution are also written here, with a note that they were flushed and by what fragment. The two options select either a text dump or a binary dump. The text dump takes up considerable room and time to dump!

Binary dump format:

* top of file:
     int linkcount_size  # 0, 4 (32-bit counters), or 8 (64-bit counters)
* each trace:
     int frag_id;  # always 0 for release build
     app_pc tag;
     int entry_offs;
     int num_exits;
     int code_size;
     uint num_bbs;
     if num_bbs > 0 # tracedump_origins
       foreach bb:
         app_pc tag;
         int bb_code_size;
   	 byte code[bb_code_size];
     endif
     foreach exit:
       int exit_cti_offs;
       int exit_stub_offs; # for separate stub, this is NOT within trace!
       app_pc target;
       bool linked; # bool is 4 bytes
     if linkcount_size > 0
       linkcount_type count; # sizeof == linkcount_size
     endif
     byte code[code_size];
See the provided sample binary trace dump reader for an example of how to disassemble a binary trace dump file. Printing symbolic information for addresses is not yet supported, but will be in a future release.

-tracedump_origins
When selected by itself with neither -tracedump_text nor -tracedump_binary, dumps only a text list of the constituent basic block tags of each trace to the ``traces.TID'' log file. When combined with either of -tracedump_text or -tracedump_binary, adds a full disassembly of the constituent basic blocks to the dump.
-prof_pcs
-prof_counts
These options turn on different types of profiling. Please see Profiling below for more information about each kind of profiling. Note that -prof_counts is only available with the profile build, and that -prof_pcs is only available on Linux.
-nolink
Do not allow any linking of fragments in the code cache. That is, to go from one code fragment in the code cache to another, control always returns to DynamoRIO first. This significantly slows down the execution. The default is to allow the linking of code fragment exits to code fragment entries.
-noasynch
Do not attempt to capture any asynchronous events. This option is only valid on Windows operating systems and will cause DynamoRIO to miss any code executed in callbacks, exception handlers, and asynchronous procedure calls.
-nullcalls
This option turns dynamorio_app_init, dynamorio_app_start, and dynamorio_app_stop into truly empty routines. This option can be used to measure the run-time overhead of DynamoRIO. It is also useful for debugging your application without DynamoRIO.

Options that are only available with debug builds:

-stats
This option causes DynamoRIO to export statistics about the running program to shared memory. These statistics can be viewed by the GUI provided for both Windows and Linux.
-loglevel N
If N is greater than 0, DynamoRIO will print out a log of its actions. The greater the value of N, the more information DynamoRIO prints. Verbosity is set to 0 by default, i.e., no log written. All log files are kept in a log directory. There is one directory per address space per run. The directories are named dynamorio.NNN, where NNN is a number that is incremented with each directory created. There is one main log file per directory named mainlog.TID, where TID is the thread id of the initial thread. There is also a log file per thread, named log.TID. The loglevel may be changed during program execution, but if it began at 0 then it cannot be raised later.
-logmask 0xN
Selects which Dynamo/RIO modules print out logging information, at the loglevel level. The mask is a combination of these bitfields:
   LOG_NONE           0x0000 
   LOG_STATS          0x0001 
   LOG_TOP            0x0002 
   LOG_THREADS        0x0004 
   LOG_SYSCALLS       0x0008 
   LOG_ASYNCH         0x0010 
   LOG_INTERP         0x0020 
   LOG_EMIT           0x0040 
   LOG_LINKS          0x0080 
   LOG_CACHE          0x0100 
   LOG_FRAGMENT       0x0200 
   LOG_DISPATCH       0x0400 
   LOG_MONITOR        0x0800 
   LOG_HEAP           0x1000 
   LOG_VMAREAS        0x2000 
   LOG_ALL            0x3fff 
It is easiest to set the mask using the Windows GUI's dialog box for setting it. You can copy and paste the resulting mask value to set the mask outside of the GUI.
-notify
This option causes a small number of status messages to be printed to stderr. These include where the log directory is, a notice each time a new log file is created, and notices when a fork or an execve occurs.

Profiling

DynamoRIO currently comes with two profiling options. The first, selected with the -prof_pcs runtime option, is a simple statistical sampler. (This option is currently available only under Linux.) This option sets a timer to periodically interrupt DynamoRIO and query which part of DynamoRIO was running. When the program ends, a distribution of the samples is printed to the file ``pcsamples.TID''. (This option is currently available only under Linux.) You should expect to see the majority of the samples occurring in the fragment cache. The value of the pc each time the timer went off is also recorded, and these pc's (along with corresponding fragment tags and offsets) are also printed to the file ``pcsamples.TID''.

The other profiling option requires the use of the profile library. To use it, set the -prof_counts runtime option. This measures the execution count of each exit from each trace. Its overhead is noticeable. The exit counts are printed out with the traces in the file ``traces.TID'', so you must also select either -tracedump_text or -tracedump_binary.


Application Interface

An alternative to running an entire application under DynamoRIO is to use the application interface to specify a portion of the application to run. This interface consists of the following routines:

dynamorio_app_init()
dynamorio_app_exit()
These functions perform the per-process initialization and cleanuprequired by the DynamoRIO system. It must be called before the application creates any threads or makes any other DynamoRIO API calls. This function takes no parameters and returns 0 if the initialization was successful.
dynamorio_app_start()
Execution of this function causes the thread of control to be transfered to DynamoRIO. The program will appear to return from this call with no visible side effects except those associated with the invocation of an empty function. However, the running of the program now occurs within DynamoRIO's code cache. As we describe below, you are now able to observe, capture, and modify the application's code using the hooks described later in this manual. Execution continues under DynamoRIO until it encounters a call to dynamorio_app_stop. This function takes no parameters and returns no values.
dynamorio_app_stop()
If the program is running under DynamoRIO control, it will again run directly on the machine upon invocation of this routine. If the program was already running under its own control, this call has no effect on the application's state. This function takes no parameters and returns no values.
dynamorio_app_take_over()
Calling this routine is similar to dynamorio_app_start except that all subsequent dynamorio_app_ calls are ignored. This is useful to override existing dynamorio_app_start() and dynamorio_app_stop() calls.

Example

The following program prints ``Hello world!'' under DynamoRIO control:
  #include 
  #include 
  #include "dynamorio.h"

  main()
  {
      int rc = dynamorio_app_init();
      assert(rc == 0);

      dynamorio_app_start();
      printf("Hello world!\n");
      dynamorio_app_stop();
      dynamorio_app_exit();
  }

You simply compile this application and link it with the DynamoRIO library corresponding to your current execution environment (Linux or Win32). On Linux, you must also link with the libc and libdl libraries. When you execute this application, you shouldn't notice any difference, except that it might run a bit slower. Since there is next to zero reuse in this simple application, running the application and its library routines in DynamoRIO's code cache will slow down the application. If you wrap dynamorio_app_start and dynamorio_app_stop around the frequently executed portions of your application (those with lots of reuse), you should observe very little overhead.



Documentation Home