ptrace is a clone of strace (or rather, strace is a clone of truss,
and ptrace is a clone of truss - a very old and early tool of mine).
ptrace (like strace) let you monitor a process or tree of processes
and see the system calls. ptrace tries to combine the best
bits of truss and strace, along with new functionality - very
important functionality.
A particular target of interest is process-termination records.
When a process terminates, there is a lot of valuable information
available - but normally discarded. This includes:
- Process size - current and peak
- CPU time - user + system, along with child user + system
- Open files - file descriptors in use, and peak open files
- Parent process id
- Context switches
There is a lot of information available; existing Unix tools, like the
shell "time" or /bin/time command make much of this avaliable, but
usually focussed on the single command you run. Getting the tree
of information from a command script (e.g. make), is not doable with
out a lot of work.
I have used LD_PRELOAD in the past to collect this information, but
this is fairly intrusive, and using a LD_PRELOAD sits inside the process.
With ptrace, you can attach to a running process or launch the process, and
with many options to control what to monitor and the amount of output,
it is an invaluable tool in the toolkit. strace is a very good
alternative, but ptrace purports to be better.
I hadnt touched my ptrace implementation in over 6 years - the code was
very stale and lacking many system calls, missing the newer features
of strace (do you know what the "-y" switch does? Neither did I till I look
at strace in depth).
ptrace comes with good documentation describing the features, but
the primary switch is "-exit". When this is used, the process
termination records are written and you can use the Perl script (psummary.pl)
to print out the most interesting processes, or see a timeline to
show the order or process fork/exit's, in a tree like fashion, or
modify the script to suit yourself.
I am putting this out as the first public release - I may release
to github at a later date, but for now, this is a binary only release.
(Delaying publication of the source simply because ptrace is not
free of the CRiSP utility routines I use to wrap malloc and a few other
functiosn); the source code makefile is ugly - it used to work on Solaris,
but I have given that up for now, and concentrated on getting ptrace
to work - to fulfil the goal of getting useful output.
Here is the link to ptrace. (It is available on my crisp.demon.co.uk
site as well).
Download here
And heres some examples of the psummary.pl script:
/home/fox/src/trace@dixxy: psummary.pl
Number of processes : 245
#1 Longest running: 3.322815 31025 /usr/bin/make
#2 Longest running: 3.283731 31029 /bin/dash
#3 Longest running: 3.271198 31033 /usr/bin/make
#1 Peak RSS: 28184 31209 /home/fox/crisp/bin.linux-x86_64/cr
#2 Peak RSS: 8988 31260 /home/fox/crisp/bin.linux-x86_64/hc
#3 Peak RSS: 8988 31261 /home/fox/crisp/bin.linux-x86_64/hc
Maximum file descriptor : 14 31209 /home/fox/crisp/bin.linux-x86_64/cr
Maximum RSS : 4692 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max VmStk : 140 31033 /usr/bin/make
Max VmData : 0 31025 /usr/bin/make
Max voluntarty ctx : 6301 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max non-voluntarty ctx : 68 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max CPU time : 0.98 31025 /usr/bin/make
Max cumulative CPU time : 1.1 31025 /usr/bin/make
/home/fox/src/trace@dixxy: psummary.pl -timeline | head -15
21:44:24.987056 start 31025 /usr/bin/make
21:44:25.009420 start 31026 /bin/rm
21:44:25.013167 end 31026 /bin/rm
21:44:25.014206 start 31027 /bin/rm
21:44:25.018092 end 31027 /bin/rm
21:44:25.019040 start 31028 /bin/rm
21:44:25.023004 end 31028 /bin/rm
21:44:25.025169 start 31029 /bin/dash
21:44:25.028427 start 31030 /bin/dash
21:44:25.028740 end 31030 /bin/dash
21:44:25.030161 start 31031 /bin/dash
21:44:25.030594 end 31031 /bin/dash
21:44:25.031529 start 31032 /usr/bin/basename
21:44:25.035048 end 31032 /usr/bin/basename
21:44:25.036802 start 31033 /usr/bin/make
/home/fox/src/trace@dixxy: psummary.pl -top 5 -l
Number of processes : 245
#1 Longest running: 3.322815 31025 /usr/bin/make
#2 Longest running: 3.283731 31029 /bin/dash
#3 Longest running: 3.271198 31033 /usr/bin/make
#4 Longest running: 2.871467 31077 /usr/bin/make
#5 Longest running: 0.808828 31104 /bin/dash
#1 Peak RSS: 28184 31209 /home/fox/crisp/bin.linux-x86_64/cr -batch -mregress
#2 Peak RSS: 8988 31260 /home/fox/crisp/bin.linux-x86_64/hc -compress -m prim/prim.hpj
#3 Peak RSS: 8988 31261 /home/fox/crisp/bin.linux-x86_64/hc -compress -m prog/prog.hpj
#4 Peak RSS: 8988 31262 /home/fox/crisp/bin.linux-x86_64/hc -compress -m relnotes/relnotes.hpj
#5 Peak RSS: 8988 31263 /home/fox/crisp/bin.linux-x86_64/hc -compress -m user/user.hpj
Maximum file descriptor : 14 31209 /home/fox/crisp/bin.linux-x86_64/cr
Maximum RSS : 4692 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max VmStk : 140 31033 /usr/bin/make
Max VmData : 0 31025 /usr/bin/make
Max voluntarty ctx : 6301 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max non-voluntarty ctx : 68 31209 /home/fox/crisp/bin.linux-x86_64/cr
Max CPU time : 0.98 31025 /usr/bin/make
Max cumulative CPU time : 1.1 31025 /usr/bin/make
The final piece of ptrace is to print out the peak cpu/memory information,
to help guide on how much resources are needed to run a script (typically
needed for a compile server or other shared compute resource).
If you are interested in the tool or the source, let me know (email)
(CrispEditor -a.t- gmail.com). If there is enough interest, I will
take it to the next level.
Future ideas for this could be to add dtrace like functionality, but I
am trying not to mix the two systems, for now (e.g. the ability to run
a script on entry/exit to system calls). ptrace contains facilities
for emulating ltrace - but these arent debugged as yet (allowing
tracing of arbitrary shared library entry points - you will see many
libXXX.so files in the bin directory, but that is not a goal at present).
If people have other "I wish ..." ideas, please forward them to me.
I may have the guts of such tools, or have a way to link ideas together.
Happy Xmas!
Post created by CRiSP v11.0.22a-b6663
No comments:
Post a Comment