Tracing Oracle background processes with Dtrace for Linux

I’ve been googling for articles on using Dtrace with Oracle databases. There isn’t all that much. I’ve found an article from Andrey Nikolaev on using Dtrace to investigate exclusive latches on Solaris. He uses this dtrace command to investigate the MMON process (in his case PID 16444):

$/usr/sbin/dtrace  -n 'pid16444:oracle:kslgetl:entry'

But unfortunately this doesn’t work on Linux. Reason: on OEL Dtrace doesn’t have an oracle provider (which is used in Andey’s Dtrace command). So neither can you use Dtrace to trace only kslgetl executions that way.

If you look at the dtrace syntax, you find that you can trace a process with dtrace -p. The usage info tells this about the -p option:

    -p  grab specified process-ID and cache its symbol tables

So that is worth a try. In my case the PID of MMON was 3573. So here is my trace one-liner:

dtrace -p 3573 -n 'syscall:::entry'

And now the results will fly over your screen:

dtrace: description 'syscall:::entry' matched 300 probes
CPU     ID                    FUNCTION:NAME
0     36                      ioctl:entry
0     36                      ioctl:entry
0     30               rt_sigaction:entry
0     30               rt_sigaction:entry
0     30               rt_sigaction:entry
0     30               rt_sigaction:entry
0      6                      write:entry
0    388                      futex:entry
0    388                      futex:entry
0      4                       read:entry
0    206                     ptrace:entry
0    126                      wait4:entry
0    388                      futex:entry
0    200                  getrusage:entry
[..]
0    200                  getrusage:entry
0    204                      times:entry
0    200                  getrusage:entry
0    204                      times:entry
0    416                 semtimedop:entry
0    388                      futex:entry
0    388                      futex:entry

The thing is, these are not just the syscalls for process 3573, but all of them. So I’m missing some clue here.

Luckily there are Dtrace one-liners and scripts everywhere and usually there is one that does what you need. Like the script I found on Oracle’s Dtrace on Linux page. This is the script (call it syscalls_pid.d):

#!/usr/sbin/dtrace -qs
syscall:::entry
/pid == $1/
{
  @num[probefunc] = count();
}

And we can run it with the PID as an argument (after adding execute privileges on the syscalls_pid.d script):

./syscalls_pid.d 3573

And after a minute or so, this was the output:

[root@sequoia ~]# ./syscalls_pid.d 3573
^C

  semtimedop                                                       12
  times                                                            52
  getrusage                                                       478

I have to say, this is quite disappointing. Where are the kslgetl’s and everything? Or any Oracle database related system call?

It seems we really need to wait on an Oracle provider. There isn’t one yet for Dtrace for Linux. But will there ever be one? The more and more I dig around about Oracle Dtrace on Linux I find out that Dtrace on Solaris is actually the cool utility as advertised and Oracle on Linux is … dead?

I decided to ask Oracle (at Rabobank we have our contacts). I’ve asked them if I’ve missed anything? Is Oracle’s Dtrace for Linux still in Tech Preview (like an OTN Forum on Dtrace)? Turns out there are two sites with a lot of Dtrace information: the Oracle Linux Dtrace Tutorial and the Oracle Linux Dynamic Tracing Guide. I’ll have to look into those.

First I’ll go to Harmony 2015 in Tallinn, Estonia, then I’m a couple of days off and I’ll see after that.

 

Advertisements

About Marcel-Jan Krijgsman

Ever since I started working with Oracle, I had an interest in Oracle database performance tuning. This led, eventually, to a four day training I made and gave for customers of Transfer Solutions. Since 2012 I work for Rabobank Nederland. A few years ago I also became interested in Oracle database security. All technology aside, it is my experience that security usually plays out on a political level. I'm a Oracle certified professional for the 8i, 9i, 10g and 11g databases and Oracle Database 11g Performance Tuning Certified Expert.
This entry was posted in Linux, Oracle database administration, Oracle performance tuning and tagged , , , . Bookmark the permalink.

One Response to Tracing Oracle background processes with Dtrace for Linux

  1. Pingback: Tracing Oracle background processes with Dtrace for Linux | Dinesh Ram Kali.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s