Monday, 21 July 2014

Problems with RSS

RSS Feed

Not sure what did it - maybe I ran as root, accidentally when I didnt mean
to, but nothing would update, despite changing permissions and resetting
some internal state. In the end decided just to clean it all out and
start again. If you are reading the feed, you likely wont notice much
and in a few hours, the differing RSS feeds will race ahead with their
usual updates.

Despite the simplicity of the code, theres always a surprise lurking
in the code - but then, thats the fun of the fair !

Post created by CRiSP v11.0.33a-b6757

Sunday, 20 July 2014

A real good read... (Xplain)


I do like a good read, and the above link is a nicely written way to
understand X Windows and X11 and related. I remember the first time I heard
of X, and had some consultants explain it, a long long time ago. And it
took me a while to get used to it - the day I moved from a character based
terminal to a Sun/3 workstation with an infinite set of display options,
and a whoopingly large 1600x1200 display. It took nearly 20y to be able to
afford one of these at home, and today, we finally have 1920x1080 (alas,
no more 1920x1200), 2560x1440, 3200x1800 and real 4k and 8k screens
coming, with prices dropping on a weekly basis.

Have fun

Post created by CRiSP v11.0.33a-b6757

Saturday, 12 July 2014

A sad day...A glorious day

It is a sad day. I am going to decommission the CRiSP FTP server.
This is a macmini - PPC vintage (400MHz) I think. It has performed
absolutely admirably. 9y of service - the oldest file seems to
be March 2005.

At the time the PPC mac mini came out - it was a brilliant device.
It still is. It has not murmured in the last 9y, despite 24x7 operation.
A *huge* 80G hard drive, 256MB of RAM. This is a *big* machine compared
to my earliest machine. (My earliest machine - had *1K* of RAM; my
first real PC had 4MB of RAM, 20MHz i386 and 20MB of disk space). My
Galaxy Gear watch beats the pants out of all of these.

The PPC (I cally it "minny") hasnt misbehaved - but it was
murmuring a little - but I attribute this to either some DNS lookup
delays, or my news aggregator Perl script. The news aggregator
has a bug/memory leak, so after a few weeks of operation its used about
200min of CPU and a whopping 40MB of RAM. One of my weekend jobs is
to fix the memory leak.

Now the good news. Intel NUC. I keep seeing these devices, and they
are cute. Of course, one never buys a machine because it is cute, but,
err...ok, I did. The Intel NUC (go see Amazon or Intel for details) is
a device slightly taller than the old style Mac Mini, but smaller - about
the size of a CD case. I kept trying to decide what to do; previously
I had purchased a Raspberry Pi, as a replacement for minny, but I was not
happy with the Pi - too low performance, horrible dangly wires and
protusions and unreliable power supply. The NUC is good - I opted
for the outdated Intel Celeron model - the cheapest. I thought I would
use USB for hard drive or an SSD disk. But why pay for SSD. A 1TB
disk was about 50 pounds. Add 26 pounds for 4GB RAM, and we are looking
at a new machine for about 170 pounds. The machine screams.

I installed Ubuntu 14.04 - the same as my other machines, and I can now
divorce myself from the PPC vs x86 binary compatibility problems.

I never filled the 80GB drive in the mac mini and I definitely wont
fill the 1TB drive in the new machine. Of course, the new machine needs
a name - "nucky" - after Nucky Thompson, from "Boardwalk".

I am presently copying the old crisp releases on to the machine so it can
transparently take over from minny. And thats about it.

In all the years of minny - bots have been continuously attempting
automated dictionary attacks to break into the machine. Maybe, because
its a PPC, that no-one ever succeeded. I like to think they wont
on nucky, but, there is nothing to steal from there. It would be
darned annoying.

This new "mini" machine is great; I might get a higher end one - an i3 or i5,
but the price differential seems obsurd. The Celeron chip is a dual
core chip - with a decent size cache. Its a great CPU for this purpose. It
might not be for intensive video (but should be sufficient for watching
videos), or gaming.

So, say hello to "nucky" the next time you grab some news from the
aggregator, or a copy of CRiSP, or a copy of dtrace, etc.

Post created by CRiSP v11.0.33a-b6757

sort -V : Thanks. That was not helpful

Was just looking at the code in CRiSP which checks whether a new
software update is available. It works by downloading an index
file with version information, and comparing the version
you are running against the master index. Very
simply mechanism.

The current latest release of CRiSP (for linux 32/64 bit systems), is
11.0.32. I am staring at the generated file, and it says "11.0.9". So,
anyone using the "Check for updates" menu in CRiSP will have no idea
that new versions are available.

But why?

Looking at the script to generate the updates - the last time I touched
this was in 2003. I know the script worked at some point in the last
few years. I rarely need to see if I am running the latest version of
CRiSP - so it would not show up as an issue.

The root of the issue lies in how the script gets an index
of all the indexed files:

get_latest ()
find . -name crisp-$1-b\*.* -print |
sed -e 's;^./;;' | sort -n | tail -1

The "find" command is getting a list of filenames - files are stored
in version directories (eg 11.0.31/). And because the build (b) number
is at the end of the filename, we want to sort all the available versions
of the files, but we want to sort numerically on the version numbers.
This means that something like:


would come out in a natural order. The "sort -n" does that in the
shell script above.

Turns out the "sort" people changed the behavior of a numeric sort
in the presence of text: they ignore the attempt to sort! So
instead of generating a list of the most recent versions of CRiSP,
we have a list of the oldest versions available instead. Not very
helpful. A quick scan of "sort"'s switches shows that I needed
to use "-V". I had never seen it before, but am glad it is there and
now generates the correct version information.

This illustrates a problem with all software, that by depending on
other tools and libraries, you will one day be silently broken
and may not realise it.

Post created by CRiSP v11.0.33a-b6754

Thursday, 26 June 2014

Musing on Android Wear

I have a Galaxy Gear 1 watch. I consider it to be a great gadget -
I became comfortable flashing the "null_" ROM on it, and the fact
that I can "adb" to talk to the device of USB is great. I dont like
Android - simply because its a bastardised Linux kernel. I somewhat
refuse to use Eclipse to create apps or edit files, when CRiSP serves
all my needs. The lack of a proper shell is a real nuisance. The
built in shell, and lack of root/su is really a two finger exercise
to the people who pay for these products. (This includes all Android
phones and tablets).

OTOH, I love the freedom of Android compared to the Apple products;
I love Samsungs multi-window mode - I just wish Apple and Google would
wake up to adding real features I want, rather than rejiggling the
Setup menus on each release, and preventing me from either deleting or
removing the built in apps. There are more and more Google apps, and
I have a hard time telling what they do and try to avoid them. (Not all
of them, but the "we know what you want - HERE YOU ARE!" is annoying;
Apple and Microsoft are no different).

But I am talking about watches. Not long after the Galaxy Gear 1
came out was the Tizen based Galaxy Gear 2 (or whatever Samsung
want to call it). A really good way to destroy customer loyalty by
confusing and diluting the market. Fortunately, you can put Tizen
on to the Gear 1 (see xda-forums - brilliant site). I dont know *why*
I would want to do that; and a few months later (i.e. now), we have
Android Wear. I feel I have walked into a department store and
am trying to choose from the 300 brands of soap!

The Galaxy Gear 1 is great - fairly robust; let down by amateurish
software and user interface (why do I get to choose from "white" or
"orange" text? What happened to the other 2^24-2 colors?)

But heres the kicker. Because it has to be charged daily, I have
to take it off at night. And, in the morning, I forget to put it on.
(I have my old watch for night time - it has a luminous dial; having a watch
blind you with a searchlight brightness everytime you move your arm, meant
it lasted about 2mins in bed). So - I am a "loyal" or eager person for
these devices.

But the silly design means I forget to take it with me. That is worse
than forgetting your phone; if you leave your phone at home, you know
it very quickly into your work day. So you just dont do it. But,
leaving the watch behind - well, sorry, but that is flawed marketing.

All because it cannot last 24h of use.

The new Android range from LG, Samsung and Motorola look great - options
at reasonable prices with high spec hardware (although varying screen
resolutions). I see many complain of the price (which is lower than
the silly extortionate price Samsung wanted when the Gear 1 originally
came out).

Oh well...

I see Apple are finally going to come out with a phone with 128GB of
storage! Yippee! I waited 5+y for that to happen, and now I can have
144GB or 160GB of flash at less than half the price Apple will charge for
their device. I think Apple missed the boat.

It is very relaxing to use Android and use a variety of mechanisms to
copy files to the devices, vs the sole use of iTunes. And even in iOS 7,
the video player looks to be written by a child - totally unusable;
totally ignoring playlists; a horrible interface for deleting videos.
Unrecognizable icons.

I fear that computing has now peaked. The new generation of software
is rapidly worse than the prior releases of software, simply because
"change is good".

Post created by CRiSP v11.0.32a-b6748

Wednesday, 4 June 2014


I wrote the other day that I had solved the silly "Cannot run gmail in firefox
cookie problem". Alas, I was wrong. I have to give credit to google here.
There is no way that they would allow this problem to be fixed or fixable
or provide a useful or informative message about the true problem.

I believe I know the true problem, and I am really impressed how
such a technically gifted company can do this to their users. Well done

On another correction or update note. I asked about
how many function calls vs function exits exist in any program, and
how they are not even close to the same. I mentioned that inlining
can confuse the scenario - this is true. But equally, tail-function
calls which get optimised into JMP instructions account for a potentially
large bulk of the deficit. It would be possible to unoptimise the tail
calls to get more accurate accounting, but I am not so bothered at present.
It is a problem that needs solving to create accurate and bounded
stack traces of an application.

Post created by CRiSP v11.0.32a-b6738

Saturday, 31 May 2014

Metrics, Statistics and Code Quality

I've been toying with writing a code coverage tool - similar
to cov/gcov, but different. Its based on a piece of work I did a long
time ago - at the time, this was a debugger, but in playing with
the concept, and being surrounded by many good tools and scripting
languages, something "new" starts to appear.

The discussion below is about C/C++ - the results may not
be applicable for Java/.NET, Python/Perl.

Consider the following question: For your favorite application,
how many functions are called to start the application?

You can guess at the answer if you dont know the answer.

Follow up question: How many function *returns* do you estimate
happen during the program startup?

You are wrong! The number of function returns may not equal
(to a close approximation) the number of function calls.

I was surprised by this observation. I tested the 'cr' console
mode CRiSP startup. It runs about at about 550,000 function calls
to start it up. But, surprisingly, only about 447,000 function returns.

The reason is: inlining. An inlined function may show up
as a function entry, but may not show as a function returning.

Once we have a trace of the function executions, the possibilities
open up to all sorts of metrics generation.

We can look at the frequency - which functions are called the
most. Or we can look at duration. Duration can be problematic
for those functions which are called once, but never return (eg.
"main" will get called once, but wont return until the application
exits - or, maybe never, because exit() may be called).

The tracing tool I have logs every function call, line executed, and
function return, for offline analysis. Execution of CRiSP generates
a 500MB log file - quite hefty for a 'small' application.

Theres some other things that can easily be done here - such
as instrumenting certain instructions or functions to gain an insight
into other things. For example, it would be possible to trace
all mutex locks, or file I/O, or log the stack of specific scenarios.

Much of this may sound familiar, because gcov, strace and dtrace can do
variants of these. The point being that each tool excels at a specific
domain of monitoring, but almost none give you programmatic access
to detailed working of an app. dtrace comes close, with the D scripting
language, but its not really very good for user space introspection
(other than trapping function calls and stack traces).

If theres interest, I will publish the tool - its a simple Perl
script for the annotation recording, and a small C library. The tool
modifies the assembler code of your application (so you really need a
special area to build in - you dont want to test or distribute these
binaries, since there is a size and speed penalty to this optimisation;
I havent finished optimising - the execution penalty when the tracing
is disabled is small, but not small enough).

Post created by CRiSP v11.0.32a-b6738