First electronic computers during WWII, when my parents were children.
Within one life-time we have ubiquitous computing.
A first-person view of the second half of that revolution.
Predominantly in the software-as-such industry, but eclectic.
Not representative of the many who work with computers to support other
industries, or have stayed in one job for most of their career.
Heavilly skewed towards maintenance.
Electronic computers came on the scene during the
war against Hitler, while my parents were children. My father wrote a book on
analogue computers at just about the point when digital computers displaced
them. Throughout his career, they have played a pivotal rôle in his
work-place. Among other things, that's why, as an 18-year-old seeking a job, I
was living within two miles of a place which actually had some use for a bright
young lad who could program computers and wanted to earn some money between
leaving school and becoming a student.
In this page, I endeavour to relate my first-hand experience of the world of
computers; which is heavilly coloured by my career
history, mostly working for businesses whose only (or principal) product is
computer programs. Many folk who work with computers do not work for such
businesses – they support the process of a business producing some other
product, by deploying computers and software to provide the infrastructure that
facilitates the primary business and by helping their colleagues to understand
and make use of that infrastructure. I have a little experience of that, also;
if nothing else, well-run software houses employ folk in such rôles, quite
separate from the folk who produce the primary product; and many of these have
proven good friends. A hint to all: they appreciate being appreciated, and
prefer it if you at least try to make sense of your problem before
either trying to fix it yourself or calling them in.
As soon as we started programming, we found – to our
surprise – that it wasn't as easy to get programs right as we had
thought. Debugging had to be discovered. I can remember the exact instant when
I realized that a large part of my life from then on was going to be spent in
finding mistakes in my own programs.
Maurice Wilkes,
designer of EDSAC, on programming, 1949
Wilkes was lucky. I have spent a large part of my life
finding and fixing mistakes in other people's programs.
This page is also a presentation: if you can persuade your user agent
(a.k.a. web browser) to render it for media type projection (e.g.,
in Opera, by entering full-screen
mode with F11) you won't see this introduction, you'll just see some
slides. Because this just deploys some standard web technoligies, one document
can serve as both the presentation – essentially comprising cues to remind
me of what the salient points are in each section – and the full
text.
URL:
CSS media type projection (Opera Show): making a presentation
that's a web page at the same time.
Pupil (1976ish–1981)
The ancient main-frame:
The machine the coal board threw away – with good reason.
Paper tape, mechanical tele-types, optical reader on computer –
editing by selective copying while inserting corrections.
Batch processing (typical up-time: 20 mins) in Basic.
Boot panel: a row of switches and a button to load one instruction at a
time; a piece of paper inside the door listing the boot sequence.
Computing the digits of e; transcribing the octal core-dump by hand.
Research Machines Ltd.:
In 1945, the old staff scoffed at the plumbers – new science
teachers at the school that had frowned on young Darwin's hobbies;
In 1981 Mr. Roscoe applied (without much hope of success) for a budget
for 6 modern personal computers @ k£ each;
Nearly fell off his perch when they gave him all he'd asked for !
Audio tape casettes as media; VDU and on-screen editing; more Basic.
Assembler: a school-mate implemented pong and space invaders; sold them
back to RML !
Computational Fluid Dynamics (1982, 1983)
The next village round Leicester from home – Whetstone:
Home to CEGB, NNC and a major computing centre; heroes of the Algol era.
NNC hired me between school and university; and a summer job as a student.
They taught me ForTran; compilation and speed; 2500 nicely formatted
digits of e in six seconds.
VAXen running VMS – an Operating System where
typing Helpactually gets you help !
Weekly pay-packets, daily cycle-rides and the auto-pilot that saved my life.
Modelling the flow of boiling water under very high pressure
that's been released:
SATAN – a vast Monstrous ForTran simulation, taking 1hr per run.
Running remotely on a Cray in Harwell, @ k£/hr, via a dedicated
line; the phone that toggled between DATA and VOICE.
The perils of expensive testing: why spaghetti doesn't get un-tangled
(and the buglet wan't fixed).
The tortuous pump simulation and the bug:
Verifying (in order to be able to document) that it did what its
documentation claimed.
Every colour the secretary could find me in pencil, biro, felt tip; reams
of fan-fold, wiggly lines everywhere; only one minor buglet.
448.831169 – the importance of naming your constants.
Expressed concern about the use of quadratic interpolation, documented
why it was used despite its natural perils.
One byte error (5↔6 in an exponent) in data for the quadratic
interpolation: the bug that took 11 runs to track down.
Student (1980ish–1986)
School and University – primitive desk-top publishing:
The care and feeding of photocopiers.
The Old Gestetner – typing on special flimsy paper and cranking out
copies – manual labour.
Preparing Eureka for publication on primitive word-processors; writing
unavailable symbols and drawing illustrations by hand.
Summer job 1985 – Finite Element Modelling:
More ForTran, and some Basic, on VAXen running VMS.
Researcher with no time delegates pet project to student for the summer;
a scripting language and a beautiful idea – now called
Object-Orientation.
Command-line parser: recursive descent without recursion.
Self-referential structures and subtle meta-type system.
ForTran lacks function types, making vtables impossible.
University – Big Iron from Big Blue:
Phoenix: Cambridge's idiosyncratic cousin of Unix.
Resource shares; run-time cost less when demand was least; late-night geeks.
Acorn's BBC Micros as dumb terminals; Archimedes arriving at a party.
Batch processing and job control in JCL (all forgotten).
ForTran course work: but the only thing I was taught at university, that
has been of use to my work, was homotopy theory (called topology by
programmers).
Parametric Solid Modelling (1988–1991)
Decent tools (the toy system): making developers more productive.
AGA – C without the warts: seducing developers into writing sensibly.
Developers on VMS via vt100s (but, later, DECStations); and my first
introduction to Unix.
A team of two supporting delivery on VMS and many versions of Unix.
The KIU: clients see tags, internal code sees pointers to structures.
The perils of rectangles: nrows and ncols
vs tall and wide; the importance of testing non-squares.
A career-limiting step – learning to love poachers, and not
laugh too loud.
The squirrel: traversing trees, and the joy of in-line folded documentation.
The Dilbert Universe: US defence contractors at the end of The Cold War.
The joy of regression testing.
New version of VMS: mysterious memory tramplage and the importance of
following calling conventions.
Image Segmentation (1991–1993)
Synthetic Aperture Radar (SAR) and coherent speckle:
When each pixel is probably wrong by more than a factor
of two.
Convoluted software finally doing better than optics did in real-time.
Cleaning up the images: identifying contiguous regions of sameness.
Algorithmic complexity and the value of flexible data structures.
Plan to throw one away: general topology and the 2-D special case.
Broken free() makes clean code priceless – a
custom malloc() with zero over-head.
Beating Moore: from batch-processing to real-time in two and a half years.
Work does not happen in a vacuum:
The two Primi-Donni: developers with conflicting egos as sys-admins; the
need for people skills and the value of patience.
Incomprehensible academics and failures of peer review: how to politely
persuade the emperor to put on some clothes, when even the little boy knows how
unwise it would be to embarrass him.
Tools: the joy of version-control and the arrival of the
GNU: gcc and addiction to emacs.
The joy of TeX and the horrors of machine-generated code.
Owning a copy of the ANSI C standard – and studying its bugs.
Military intelligence and ill-conceived notions of security.
A 'phone call, an amusing interview and my return from exile.
× 16 for half the price; a factor of 32 in (under) 5 years.
Memory: emacs @ work
1991: when process grew to 8 MB (binary + heap), machine crawled
– Collecting garbabe …
2006: 356 kB of stack, 24.5 MB of heap; easy.
6.54 MB of .so-ery + 4.25 MB of primary executable = 10.8 MB of
loaded binary (5.78 MB r-xp, 3.15 MB rw-p, 1.79
MB r--p).
An O/S which makes that information easy to dig out.
Disk
1991: 100 MB was a nice big disk.
late 1990s: hassling colleagues to be disciplined about disk use.
2006/Feb/27: unzip then tar cjf reduced 10 MB by 19%.
On a 136 GB (i.e. > TB/8) disk, this is a waste of time.
1002 MB of memory and 1906 MB of swap.
Informatics (1993–1995)
The problem – coercing data between world-views:
Human genome project anticipated a problem that already
bedevilled Drosophila melanogasta researchers, so funded the latter to
trail-blaze a solution.
Managing Big Piles of Data is well-understood in relational databases.
Geneticists need a UI they can understand and interact with comfortably.
A French theoretical physicist (married to a fly geneticist) and a
Cambridge philosopher had written ACeDB: an idiosyncratic early object-oriented
database with a geneticist-friendly UI.
SQL and O-O are mutually alien – a bridge between them was needed.
Each end changes schema from time to time, so the bridge must be flexible.
A scripting language to solve the problem: which went on to unexpected
things before it was, in its original rôle, made redundant.
Recursive descent is so much easier in C than in ForTran !
Going on holiday, leaving the first working version; Aubrey de-bugging it
– the virtues of cleanliness.
Work-shop in Montpellier: alcohol-fueled late-night brainstorming and
days in the lake – a peculiarly productive fortnight.
Making the most of networks:
Sun work-stations – being my own sysadmin and learning (the hard
way) what /usr/bin/fsck is for – the need for LSB and FHS.
Trans-atlantic co-operative development,
late-night ftp; emacs running in Cambridge but displaying in
Berkeley; ange-ftp making all file-systems appear local.
Building the GNU tool-chain; fielding gccbug reports;
contributing to ange-ftp.
Net-news: signal-to-noise and why the web was obviously better.
The plan was gopher but the web intervened; a turning point for me.
Web-mastery (1995–1996)
Back from Bergen with no plan; the party, the question and the offer.
Competent techies in partnership with graphic-design yuppies (and their
poor apprentice).
The difference between mark-up and layout: teaching the newly-educated to
put aside the out-dated assumptions of his teachers (and masters).
Learning perl and writing it maintainably; the 'phone call many
months later.
Spaghetti machines – data in, web-pages out – fixing the gap
between spec and reality by munging with perl.
Suits and the Financial Times.
Cider at the beer festival: the joys of flexible employment.
Geographical Information Systems (1996–2000, part I)
GIS and other uses for lasers:
Founded by the inventors of lasers, to make them useful.
Scanning maps and etching money printers – scaling many thousands
to one or one to several dozen.
National mapping agencies, huge data-sets and copyright issues.
A natural opportunity for open-source data – now taken
up by OSM.
Technical sophistication.
Support for many GIS data formats, arcane plotters and SQL back-ends.
A core product round which clients can build richer applications by
writing code in a custom scripting language (which should be abandoned in favour
of python hooks).
Object-oriented queries supported by a query language (modelled on
Zermelo-Fränkel set theory); my first project at LSL.
When colleagues aren't angels:
du | xdu and the subtle art of getting Doom installations off
the source disk.
The perils of C function types and cheap programmers – smack the
bogus XCallbackProc: when fixing an error isn't enough.
Never try to shout down someone who can talk as loud as you can shout,
especially when he's right and the whole department knows it.
On the other hand, there are times when a good shouting match is a great
way to establish mutual respect – even if it does scare the living
daylights out of some on-lookers ;^>
continued…
ISO 9001 done properly (1996–2000, part II)
Know Thy Self – that thou mays't improve.
Developer-led, rather than by management fiat (or – the worst
– outside consultants).
Allow deviations from process as long as they are justified, explained
and documented, so that: we can work out whether to allow them or how to avoid
them in future; and we can track the fall-out from them, if any.
Annual audit by outsiders: kept us honest; helped see what exceptions we
should formalise and what to do to avoid the other deviations; helped us
identify best practice.
The Law was made for man, and not man for the law.
Even the lowliest can refuse the demands of the mighty, insisting that
they abide by the rules.
Even the lowliest can initiate the things they need done, by feeding them
into the system in ways that the rules insist on honouring.
The audit-trail protects the innocent when there's blame trying to find
someone to land on.
Helping the guilty to reform works better than punishing them.
make re-implemented – an incomplete solution in a new
language: python seducing maintainability.
A desk with long enough legs; and the only practical use for an Ultrix pizza-box.
Making tools support the process.
Arranged for the tools to seduce the developer into Doing Right Things,
by making it easier to use the tool than to by-pass it.
Non-judgemental support earns trust and elicits honesty; learning the
user's bad habits makes it possible to adjust the tools to encourage better
habits.
Web forms make a better UI than command-lines when the user has lots of
questions to answer.
A good bug-tracking system can ease work-flow management and aid
information sharing.
Incompatibilities and Surprises
Cross-platform development on Unix (Sun, HP-UX, AIX, …) and Windows NT.
Graceless failure and the perils of auto-rejecting everything you don't
recognize, instead of only the things you affirmatively do recognize as worthy
of it; when a major re-write gets top priority.
The while-pipe gotcha, differences among Unices and why the
abominable csh is mis-named.
Hand-hacking RCS files – recovery from disk failure and changing the past.
The (1<<32 milli-seconds ≅ ) 49.7 day bug; shoddy
software and the advantages of feeble up-time.
The Web Itself
Zeus (2000–2001):
The fastest web server. Linux for a living (at last) and C++.
The joys of rigorous selection in the recruitment process – an old
dog learning new tricks among brilliant youths.
The perils of reliability: daylight-saving time handled gracefully.
DEC (Compaq) once again break my code (scanning a dlopen-ed
library's memory segments to check the symbol found in the library really
came from that library) by not following their own specs in a new
release of their operating system (Ultrix, this time): the perils of more than
one way to do it.
The internet economy boom and bust: innocents in the cross-fire.
Opera (2002–…):
The best user experience on any device – being a user
agent rather than merely a browser.
As close as I can get, and still be paid for it, to the front line in the
war against The Evil Empire.
I'll leave saying more until I can look back on it.
Conclusions
Ignore maintenance at your peril.
Straight-forward is better than Clever – the other KISS.
Good tools make developers more productive and their code better.
Experienced software engineers know that perhaps 30% of the cost
of a software product goes into specifying it, 10% into coding, and the
remaining 60% on maintenance.
Written by Eddy.