Measuring Performance Data Using Collectl Under Linux

Using the tool collectl it is possible to measure a number of performance indicators under Linux. Particularly interesting are the methods to measure properties like CPU and memory consumption per taks / thread.

Example Code

// Filename: example.c
#include <stdio.h>
#include <pthread.h>
#include <math.h>
#include <stdlib.h>
void *f1(void *arg) {
while (1) {
return NULL;
void *f2(void *arg) {
while (1) {
float j = sqrt(cos(sin(1.2)));
printf(“stuff: %f\n”,j);
return NULL;

int main(void) {
pthread_t p1,p2;

while (1) {
float i = sqrt(1.2);
printf(“main stuff: %f\n”,i);
pthread_join(p1, NULL);
pthread_join(p2, NULL);
return 0;

Compile the code

gcc -lm -lpthreads example.c

Running Collectl

sh-4.1$ collectl -sZ -i5:10 –procopts t

waiting for 10 second sample…

### RECORD 1 >>> machine <<< (1355908840.001) (Wed Dec 19 10:20:40 2012) ###

# PROCESS SUMMARY (counters are /sec)
# PID User PR PPID THRD S VSZ RSS CP SysT UsrT Pct AccuTime RKB WKB MajF MinF Command
25989 5053 20 17675 2 S 28M 636K 3 1.20 0.41 16 0:02.14 0 0 0 0 ./a.out
25990+ 5053 20 25989 2 S 28M 636K 3 0.43 0.05 4 0:00.63 0 0 0 0 a.out
25991+ 5053 20 25989 2 S 28M 636K 3 0.33 0.13 4 0:00.60 0 0 0 0 a.out

GPS Track Profile Analysis using Perl

In this post I show a small example on how one can easily evaluate GPX track files using Perl and Gnuplot. The idea in brief: Read in GPX track infile.gpx and extract the velocity distribution. In a first approximation I have fitted gaussian distributions…

The calculation of distances between two points in WGS 84 coordinates is done using the approach given by Vincenty. There are methods with an accuracy of up to 10nm (geographiclib, interesting paper on this topic).

Velocity Distribution

Code Snipplets

The code pieces listed below are extracts of what I have done. Do not expect the code to work – it shall only serve as an idea for you…

Perl Script

use Geo::Gpx;
use Data::Dumper;
use DateTime;
use GIS::Distance;
use Statistics::Lite qw(:all);
use POSIX;
my $gis = GIS::Distance->new();
$gis->formula (“Vincenty”);
my $fh = “infile.gpx”;
# Extract vlat from GPX file
my $gpx = Geo::Gpx->new( input => $fh );
my $waypoints = $gpx->waypoints();
my $tracks = $gpx->tracks();
my $iter = $gpx->iterate_trackpoints(); #points();
my (@time, @lat, @lon, @ele);
while (my $pt = $iter->())
push (@time, $pt->{time}); # Linux Epoch time
push (@lat,$pt->{lat}); # WGS 84
push (@lon, $pt->{lon}); # WGS 84
push (@ele, $pt->{ele}); # meters
my %s_time = statshash (@time);
my %s_lat = statshash (@lat);
my %s_lon = statshash (@lon);
my %s_ele = statshash (@ele);
my $n = 1+$#time;
# Velocities
my (@vlat, @dlateral_next, @vhor);
my $ms_kmh = 3.6;
for my $i (0..$n-2)
my $d = $gis->distance ($lat[$i],$lon[$i] => $lat[$i+1],$lon[$i+1]); 
my $dlateral = $d->meters(); # meters
my $dhorizontal = $ele[$i+1]-$ele[$i]; # meters
my $dt = $time[$i+1]-$time[$i]; # seconds
$dt > 0 or warn;
$dt > 0 or next;
my $vv = sqrt($dlateral**2)/$dt * $ms_kmh; 
my $hh = sqrt($dhorizontal**2)/$dt * $ms_kmh;
push (@vlat, $vv); # km/h
push (@vhor, $hh); # km/h
push (@dlateral_next, $dlateral); # meters
my %s_vlat = statshash (@vlat);
my %s_vhor = statshash (@vhor);
my %s_dlateral_next = statshash (@dlateral_next);
# Distribution
sub distribution
my $nbins = floor(sqrt($n));
my $nsigma = 2;
my ($min, $max) = ($s_vlat{mean}-$nsigma*$s_vlat{stddev}, $s_vlat{mean}+$nsigma*$s_vlat{stddev});
my $d = ($max – $min) / $nbins;
my %hist;
for my $i (0..$nbins-1)
$hist{$i}{x} = $min+($i+.5)*$d;
for my $i (0..$n-2)
my $bin = floor(($vlat[$i]-$min)/$d);
$bin < 0 and warn;
$bin < 0 and $bin = 0;
$bin >= $nbins and warn;
$bin >= $nbins and $bin = $nbins-1;
$hist{$bin}{y} = $hist{$bin}{y}+1;
my $f_dist = “dist.txt”;
open (F_DIST, “>”.$f_dist);
for my $j (0..$nbins-1)
printf F_DIST “%d %f %f\n”, $j, $hist{$j}{x}, $hist{$j}{y};

GnuPlot Fitting

f1(x) = p1*exp(-(x-m1)**2/(2*s1**2))
f2(x) = p2*exp(-(x-m2)**2/(2*s2**2))
f3(x) = p3*exp(-(x-m3)**2/(2*s3**2))
f(x) = f1(x)+f2(x)+f3(x)
fit f(x) “dist.txt” u 2:3 via p1,p2,p3,m1,m2,m3,s1,s2,s3
plot  “dist.txt” u 2:3 w l,f(x)

Network Properties Simulation Using qdisc

If you ever want to simulate bad transmission conditions (packet loss, latency, bandwidth, …) in a network: there is a module built-in to the linux kernel named qdisc. More information can be found on



Add delay 80ms

tc qdisc add dev eth0 root netem delay 80ms

Add delay with +-10ms jitter

tc qdisc add dev eth0 root netem delay 80ms 10ms
NB: For very large jitter, package order will be changed by netemIn order to disable this:
tc qdisc add dev eth0 root handle 1: netem delay 10ms 100ms

tc qdisc add dev eth0 parent 1:1 pfifo limit 1000

Add loss etc

tc qdisc add dev eth0 root netem delay 10ms 1ms distribution normal loss 2% duplicate 0.1%

Show current parameters

tc qdisc show dev eth0tc filter show dev eth0 

More complex example:

Consider you have a machine connected on eth1 and eth2 in your network, which will be used to simulate the network conditions:

tc qdisc add dev $d1 root netem delay 50ms 2ms distribution normal loss 0.5% duplicate 0.1%
tc qdisc add dev $d2 root handle 1:0 htb default 15
tc class add dev $d2 parent 1:0 classid 1:1 htb rate 400mbit ceil 400mbit



Once you are done you may want to remove the filters again:
tc qdisc del dev eth0 root
tc qdisc show

Creating an Envelope from Commandline Using LaTeX

This small piece of code will create an envelope from commandline…


MYADDRESS = “Mr X // Street 1 // ZIP”

usage() {
   echo “$0 – print envelopes using LaTeX”
   echo ”   address:   Mr. A // Street 123 // ZIP City, State // Country”
   echo ”   -f <a4,c4,c5,c6,dl> default: $fomat”
   echo ”         a4: a4              “
   echo ”         c4: a4              “
   echo ”         c5: a4 folded in 1/2″
   echo ”         c6: a4 folded in 1/4″
   echo ”         dl: a4 folded in 1/3″
   exit 0

while getopts “f:h” opt; do
   case $opt in
      f) format=$OPTARG; ;;
      h) usage; ;;
shift $(($OPTIND -1))

if [[ $format == c6 ]]; then
elif [[ $format == c5 ]]; then
elif [[ $format == a4 ]]; then
elif [[ $format == c4 ]]; then
elif [[ $format == dl ]]; then
   exit 1


[[ -z $ADDRESS ]] && usage

TMP=$(mktemp -d /tmp/envelope-XXXXXX)
cd $TMP

cat << eof > envelope.tex




pdflatex envelope.tex
lpr -o Duplex=None -o Manualfeed=On -o InputSlot=Envelope -o PageSize=Custom.${lx/mm/}x${ly} -o PageRegion=Custom.${lx/mm/}x${ly} -Penvelope envelope.pdf
rm -rf $TMP

Make an Image with Email Address

This is a small piece of code to create an image from your Email address:

echo $0 Name Emailaddress
[[ -z $2 ]] && exit 2
convert -size 250×30 xc:transparent -font Verdana -fill “#003399” -pointsize 16 -draw “text 5,15 ‘Email: $@'” $NAME-mail.png

Scan PDF from Commandline

In the following I show a small piece of code to scan PDF files from commandline:

cat << EOF 
— $0 — Scan Images -> PNG-Archive & PDF
   (1) Name
   -> Scanning: counter_start counter_increment
   -> Postprocessing: 
      -rotate (by 90 deg)
      -gray (keep it grayscale)

function DIE { echo “$@”>&2; exit -1; }
function is_installed {
   while [[ -n $1 ]]; do
      [[ -z $( type -p $1  ) ]] && DIE “missing: $1”
#is_installed convert pdftk scanimage

[[ -z $1 ]] && DIE “missing name”

if [[ -n $2 ]]; then
   shift 2

   echo “Scanning images”
   scanimage –batch=$NAME”_%d”.tif –batch-increment $CNT_INC –batch-start $CNT –source ADF –mode $MODE –resolution $RES -x 210 -y 297 -v

#unpaper –layout double –output-pages 2

GRAY=”-threshold 50%”
while [[ -n $1 ]]; do
   case $1 in 
      -rotate) ROTATE=”-rotate 90″; shift; ;;
      -gray) GRAY=””; shift; ;;
      *) shift; ;;
echo “Processing images:”
for FILE in $NAME*tif; do
   echo “Processing “$FILE”:”
   PREFIX=`echo $FILE | sed ‘s/.tif//g’`
   LIMITS=”-limit memory 512 -limit map 512″
   PAGE=”$ROTATE -page a4 +repage”
   COMPRESSION=”-compress Group4″
   #[[ ! -e $PREFIX.png ]] && convert $LIMITS $GRAY -monitor $FILE $PREFIX.png
   [[ ! -e $PREFIX.pdf ]] && convert $LIMITS $PAGE $GRAY $COMPRESSION -monitor $FILE $PREFIX.pdf
echo “Creating PDF”
CNT_MAX=`find -name “$NAME*pdf” | tr -cd 0-9″\n” | awk ‘$1>max{max=$1}END{print max}’`
for I in `seq 1 $CNT_MAX`; do FILES=$FILES” ${NAME}_$I.pdf”; done;
pdftk $FILES cat output $NAME.pdf
echo “Creating archive”
#tar cvjf ${NAME}_png.tar.bz2 ${NAME}*png
tar cvjf ${NAME}_tif.tar.bz2 ${NAME}*tif
echo “Cleanup”
rm ${NAME}_*.pdf *tif #*png

Using SSH as VPN

One of the not so commonly known features of SSH is: you may use it for a poor man’s VPN.

My Network 1   —   Client   <—SSH—>   Server   — My Network 2


USER = username
HOST = ip of the server host
IDENTITY = ssh key file

Preparation of the Server Side
Edit the daemon configuration file and restart the sshd afterwards
PermitTunnel yes

Open the SSH session from the client side:
sudo ssh -i $IDENTITY -F /home/$USER/.ssh/config -vvvw any:any root@$HOST true &

Prepare the local tunnel and configure the routes on the client:
sudo modprobe tun
sudo tunctl -t tun0 -u $USER
sudo ifconfig tun0 pointopoint
sudo route add -net netmask dev tun0
Prepare the remote tunnel and configure iptables on the server:
cat << eof | ssh root@$HOST
modprobe tun
tunctl -t tun0 -u $USER
ifconfig tun0 pointopoint
echo 1 > /proc/sys/net/ipv4/ip_forward
iptables -t nat -A POSTROUTING -s -j SNAT –to-source $SERVER_IP
iptables -A FORWARD -d -j ACCEPT

The whole script:

sudo ssh -i $IDENTITY -F /home/$USER/.ssh/config -vvvw any:any root@$HOST true &
sleep 5
tid=$(ps aux | awk ‘/ssh -i/{print$2;exit}’)
echo “Prepare local tunnel”
sudo modprobe tun
sudo tunctl -t tun0 -u $USER
#sudo ifconfig tun0 up netmask
sudo ifconfig tun0 pointopoint
sudo route add -net netmask dev tun0
echo “Prepare remote tunnel”
cat << eof | ssh root@$HOST
modprobe tun
tunctl -t tun0 -u $USER
#ifconfig tun0 up netmask
ifconfig tun0 pointopoint
echo 1 > /proc/sys/net/ipv4/ip_forward
#iptables -t nat -A POSTROUTING -s -j SNAT –to-source $SERVER_IP
#iptables -A FORWARD -d -j ACCEPT
iptables -t nat -A POSTROUTING -s -j SNAT –to-source $SERVER_IP
iptables -A FORWARD -d -j ACCEPT
echo “Press any key to kill (tid:$tid)”
sudo kill -KILL $tid
sudo rmmod tun
cat << eof | ssh root@$HOST 
rmmod tun
iptables -F
echo 0 > /proc/sys/net/ipv4/ip_forward