v14, i12: Security Forensics Using DTrace Page 1 of 5

SysAdminMag.com

Security Forensics Using DTrace

Boris Loza, PhD

Solaris 10 has introduced a new tool for Dynamic Tracing in the OS environment -- dtrace. This is a
very powerful tool that allows systems administrators to observe and debug the OS behavior or even
to dynamically modify the kernel. Although this tool has been designed primarily for developers and
administrators, in this article, I will explain how to use dtrace as a security forensics tool for
analyzing suspicious files and processes.

Using DTrace

Dtrace has its own C/C++-like programming language called "D language" and comes with many
different options. Dtrace is easy to use, once you familiarize yourself with D language and know a
little bit about Solaris internals. For example, the following construction using dtrace could serve as
a "primitive" strace command for the process ID 510:

# dtrace -n syscall:::'/pid == 510/ {}'

I will explain the syntax of such constructions in a moment.

To understand how to use dtrace as a security forensic tool, I will present a case study, as follows.
Let's assume that we are going to investigate the process ID 968 from the suspicious "srg"
application that we found running on our system.

Typing the following at the command line will list all files that this process opens at the time of
monitoring. We'll let this run for a while and terminate with Control-C:

# dtrace -n syscall::open:entry'/pid == 968/

{ printf("%s%s",execname,copyinstr(arg0)); }'

dtrace: description 'syscall::open*:entry' matched 2 probes

^C
CPU ID FUNCTION:NAME
0 14 open:entry srg /var/ld/ld.config
0 14 open:entry srg /lib/libdhcputil.so.1
0 14 open:entry srg /lib/libsocket.so.1
0 14 open:entry srg /lib/libnsl.so.1

D language comes with its own terminology, which I will address here briefly. The whole
'syscall::open:entry' construction is called a "probe" and defines a location or activity to which
dtrace binds a request to perform a set of "actions". The 'syscall' element of the probe is called a
"provider", and, in this case, permits probes on ' e n t r y ' (start) to any ' o p e n ' Solaris system call
(' o p e n ' syscall is sent to file when it is about to be opened). The so -called "predicate" ( /pid == 968/ )
uses the predefined dtrace variable ' p i d ' , which always evaluates to the process ID associated with

http://www.samag.com/print/ 04/08/2007
v14, i12: Security Forensics Using DTrace Page 2 of 5

the thread that fired the corresponding probe.

The 'execname' and 'copyinstr(arg0)' are called "actions" and define the name of the current
process executable file and convert the first integer argument of the system call (a r g 0 ) into a string
format, respectively. The p r i n t f ' s action uses the same syntax as in C language and serves the same
purpose -- to format the output.

Each D program consists of a series of clauses, with each clause describing one or more probes to
enable and an optional set of actions to perform when the probe fires. The actions are listed as a
series of statements enclosed in curly brackets { } following the probe name. Each statement ends
with a semicolon (;).

In general, each probe clause has the general form:

probe descriptions:
/ predicate /
{
action statements
}

You may want to read the dtrace(1M) man pages and the Introduction to the Solaris Dynamic
Tracing Guide at:

http://docs.sun.com/app/docs/doc/817-6223

for more options and further explanation of the syntax.

As the name suggests, the dtrace (Dynamic Trace) utility will show you the information about a
changing process -- dynamically. That is, if the process is idle (doesn't do any system calls or opens
new files), you won't be able to get any information. To analyze such a process, either restart it or
use "static" methods and utilities, such as mdb(1) , or process analyzing commands, such as pfiles(1) .

Next, we will use the following command-line construction to list all system calls for "srg". Again,
we'll let this run for a while and terminate it with Control-C:

# dtrace -n 'syscall:::entry /execname == "srg"/ \

{ @num[probefunc] = count(); }'
dtrace: description 'syscall:::entry ' matched 226 probes
^C
pollsys 1
getrlimit 1
connect 1
setsockopt 1
...

You may recognize some of the building elements of this small D program. Additionally, this clause
defines an array named ' n u m ' and assigns the appropriate member ' p r o b e f u n c ' (executed system
call's function) the number of times these particular functions have been called (count() ).

Using dtrace, we can easily emulate utilities traditionally being used to analyze suspicious binary
files and processes. However, dtrace is a much more powerful tool and may provide more
functionality; for example, you can dynamically monitor the stack of the process in question:

# dtrace -n 'syscall:::entry/execname == "srg"/{ustack()}'

0 286 lwp_sigmask:entry
libc.so.1'__systemcall6+0x20
libc.so.1'pthread_sigmask+0x1b4

http://www.samag.com/print/ 04/08/2007
v14, i12: Security Forensics Using DTrace Page 3 of 5

libc.so.1'sigprocmask+0x20
srg'srg_alarm+0x134
srg'scan+0x400
srg'net_read+0xc4
srg'main+0xabc
srg'_start+0x108

Based on all our investigation (see the list of opened files, syscalls, and the stack examination
above), we may positively conclude that srg is a network -based application. Does it write to the
network? Let's check by constructing the following clause:

# dtrace -n 'mib:ip::/execname == "srg"/{@[execname]=count()}'

dtrace: description 'mib:ip::' matched 412 probes
dtrace: aggregation size lowered to 2m
^C
srg 520

It does. We used a ' m i b ' provider to find out whether this application transmits to the network.

Could it be just a sniffer or a netcat -like application that is bounded to a specific port? Let's run
dtrace in the truss(1) fashion to answer this question (inspired by Brendan Gregg's d t r u s s utility
from users.tpg.com.au/adsln4yb/DTrace/shellsnoop):

#!/usr/bin/sh
#
dtrace='

inline string cmd_name = "'$1'";

/*
** Save syscall entry info
*/
syscall:::entry
/execname == cmd_name/
{
/* set start details */
self->start = timestamp;
self->arg0 = arg0;
self->arg1 = arg1;
self->arg2 = arg2;
}

/* Print data */
syscall::write:return,
syscall::pwrite:return,
syscall::*read*:return
/self->start/
{
printf("%s(0x%X, \"%S\", 0x%X)\t\t = %d\n",probefunc,self->arg0,
stringof(copyin(self->arg1,self->arg2)),self->arg2,(int)arg0);

self->arg0 = arg0;
self->arg1 = arg1;
self->arg2 = arg2;

}
'
# Run dtrace
/usr/sbin/dtrace -x evaltime=exec -n "$dtrace" >&2

Save it as truss.d , change the permissions to executable, and run:

http://www.samag.com/print/ 04/08/2007
v14, i12: Security Forensics Using DTrace Page 4 of 5

# ./truss.d srg
0 13 write:return write(0x1, " sol10 -> 192.168.2.119
TCP D=3138 S=22 Ack=713701289 Seq=3755926338 Len=0
Win=49640\n8741 Len=52 Win=16792\n\0", 0x5B) = 91
0 13 0 13 write:return write(0x1,
"192.168.2.111 -> 192.168.2.1 UDP D=1900 S=21405 LEN=140\n\0",
0x39) = 57
^C

Looks like a sniffer to me, with probably some remote logging (remember the network transmission
by ./srg discovered by the ' m i b ' provider above). You can write pretty sophisticated programs for
dtrace using D language. Take a look at /usr/demo/dtrace for some examples.

You may also use dtrace for other forensic activities. The following is an example of a more
complex script that allows monitoring of who fires the suspicious application and records all the files
opened in the process:

#!/usr/bin/sh

command=$1

/usr/sbin/dtrace -n '

inline string COMMAND = "'$command'";

#pragma D option quiet

/*
** Print header
*/
dtrace:::BEGIN
{
/* print headers */
printf("%-20s %5s %5s %5s %s\n","START_TIME","UID","PID", \
"PPID","ARGS");
}

/*
** Print exec event
*/
syscall::exec:return, syscall::exece:return
/(COMMAND == execname)/
{
/* print data */
printf("%-20Y %5d %5d %5d %s\n",walltimestamp,uid,pid,ppid,
stringof(curpsinfo->pr_psargs));
s_pid = pid;
}
/*
** Print open files
*/
syscall::open*:entry
/pid == s_pid/
{
printf("%s\n",copyinstr(arg0));
}
'

Save this script as a wait.d, change the permissions to executable 'chmod +x wait.d' and run:

# ./wait.d srg
START_TIME UID PID PPID ARGS

http://www.samag.com/print/ 04/08/2007
v14, i12: Security Forensics Using DTrace Page 5 of 5

2005 May 16 19:51:20 0 1582 1458 ./srg

/var/ld/ld.config
/lib/libnsl.so.1
/lib/libsocket.so.1
/lib/libresolv.so.2
...
^C

Once the srg is started, you will see the output.

The real power of dtrace comes from the fact that you can do things with it that otherwise aren't
possible without writing a comprehensive C program. For example, the shellsnoop application
written by Brendan Gregg allows you to use dtrace at the capacity of ttywatcher ; see:

http://users.tpg.com.au/adsln4yb/DTrace/shellsnoop

It is not possible to show all the capabilities of dtrace in this article. Dtrace is a very powerful as well
complex tool with virtually endless capabilities. Although Sun insists that you don't need a "deep
understanding of the kernel for DTrace to be useful", knowledge of Solaris internals is a real asset.
Looking at the include files in /usr/include/sys/ directory may help you write complex D scripts and
give you more of an understanding of how Solaris 10 is implemented.

Conclusion

When monitoring your systems, be creative and observant. Apply all your knowledge and experience
for analyzing suspicious binary files and processes. Also, be patient, have a sense of humor, and
learn how to use dtrace.

Boris Loza, PhD, CISSP is an author of UNIX, Solaris and Linux: A Practical Security Cookbook,
which deals with securing UNIX OS without any third-party applications. Boris is also a contributor
to several industry magazines and has been quoted in many information security books and Web
sites. He loves nature, reading books, watching movies, and enjoys scuba diving and entomology.

http://www.samag.com/print/ 04/08/2007