Organising photographs

I have a Sony Ericsson W810i mobile 'phone, which comes with a built-in camera. I've been merrily snapping away at things since I bought it, about a week before my sister Lucy's wedding at the end of July 2007. Later that year, I decided it was time to actually transfer some imags off the 'phone.

At this point I discovered that the 'phone came (oh so helpfully) with a CD containing the software I need to access the 'phone from my MS-Windows PC. As a GNU/Linux user, I'm not much helped by that. Still, I have colleagues on MS-Window machines, so one of them kindly transfered the images to her computer and put them into a zip file I could fetch round the network. A quick unzip and GNU tar cjf later I had a smaller file in a more GNU-friendly format to transfer home (via another computer out in the big wide world). I merrilly unpacked all those images and renamed them to things that indicate what they are, in a directory hierarchy that makes it easier for me to find things by topic – and didn't do much more with them.

This (2007) Autumn I again took my camera to work: my colleague Markus, this time on an Ubuntu GNU/Linux machine, just had to plug my 'phone in via a USB port, since Ubuntu have taken the trouble to include all the drivers oen is likely to need, so that things Simply Work (without needing the unhelpful CD from the 'phone supplier). Score one to Ubunto (I don't know the right trickery to work out what my Debian box at work needs to do the job; and my machine at home is too old to have USB ports). Markus provided me with a tar-ball to transfer judiciously as before. This (September's last) weekend, I've unpacked that and renamed everything to incidate what it is, adding to the hierarchy I created last time.

So I've got about 70 MiB (or mibibyte; that's what used to be called a megabyte, MB, until IUPAC recognised the need for separate quantifiers to distinguish powers of 1024 = 210 = kibi from powers of a thousand = ten3 = kilo) of images on my computer at home, some of them on their side, and I'd like to put a selection them on my web-site; but they're typically a quarter to a third of a mibibyte in size and I don't really want to put such huge images up on chaos, even if it has got the disk space – if only because it'd use up a lot of bandwidth. Actual sizes (of the raw images) range from 0.10 MiB to 0.53 MiB; the median and mean are both 0.29 MiB. The images use up about 1.226 bits per pixel – which, given that they encode 8 bits per pixel of colour/intensity data, shows that the JPEG format is quite good at compressing images. When I tried saving the same images in other formats than JPEG, PNG was about a factor of ten bigger, GIF was about a factor of five bigger.

Shrinking image files

So, what can I do to make the image files smaller ? For one simple thing, I can simply make the images smaller: at 1632×1224 pixels (or the other way round if on their sides) they're rather big, so simply shrinking them should go some way towards reducing the file size. (The dimensions are 4 and 3 times 408 = 8×3×17, variously.) I also discovered, as soon as I asked The GIMP to save an image that I'd needed to rotate into the correct orientation, that the JPEG image format has a quality parameter, which controlls how lossy the compression is – I'm guessing that 100% preserves all the data in the original, while lower values sacrifice some of the fine detail in favour of better compression.

Rotated test image

I was mildly surprised to discover that halving the dimensions (hence quartering the number of pixels) in a test image (of Akerselva, in full spate, looking upstream to a sluice, with a willow in mid-stream growing out of where a pillar supports a small platform that sticks out over part of the river) only reduced the file size by about a factor of three, not four: presumably, the compression can't achieve as much on the smaller image. Likewise, shrinking image size by a factor of three only shrank the file by a factor of six, not nine. Still, smaller images are better, and I don't need them at full size (although I'll keep the originals locally).

Next, let's take a look at this quality parameter, whatever it may actually measure. A preliminary investigation revealed that saving rotated images at 80% quality produced files of roughly the same size as the originals (81% was closer to the mark, on average), hinting that perhaps that's the quality value my camera's using. At 100%, the image file is almost three times as big as the original. I cranked the quality down to 10% (on the same test image as above) and saw quite obvious colour artefacts – orange and green splodges in what should have been foam on dirty water. Those were also discernible, 'though fainter, at 20%. At 30%, the colour is OK, I can't see any other artefacts unless I zoom in; and the file size is down to two fifths of the original (i.e. the 80% quality rotated image). So, time to look for a point somewhere between 30 and 80 that achieves a good ballance of perceived quality and file size.

To guage perceived quality I need a mechanism for comparing images, that'll let me notice fine differences in details. Fortunately, there's a technique called blink comparison for doing this (first developed by astronomers, for detecting planets and asteroids, by comparing photographic plates of a selected piece of sky on different nights): take the two images and arrange to be able to switch between them. My web user agent (a.k.a. browser), Opera, supports tabbed browsing, so I can load images into two tabs (each maximised in the Opera window; and scrolled to display the same portion of the image) and flip between tabs. This makes it easy to see the differences between two images.

Using my rotated test image, at full scale, I saved to various quality values and compared the images. The following table records the file size, the result of dividing this by the quality (understood as a fraction, so 100% is 1 and 65% is 0.65) and the lowest magnification at which the image had discernibly poorer quality than the 100% image:

QualitySize/KiBSize/Quality/KiBMagnification
30%1254162.1
40%1503732.2
55%2254082.8
60%2353913.5
65%253388
70%267381
75%282375
80%308384
90%373414
100%855855

(Note that Size/Quality is just something I was curious about; I have no meaningful interpretation to put on it.) Beyond 60% quality (and actually the same was more or less true at 60%) the lower quality images were as good as the 100% image, albeit different (crisper, but this applied as much to the image artefacts – present even in the 100% quality version – as to the true image). At higher magnification, I could see differences, but could not call either image better. So there appears to be little benefit in setting the quality parameter above 60%, at least for full-scale images. That's still a quarter megabyte file, though, so it's time to look at scaled versions.

When I tried comparing scaled images to the full-scale 100% (or, indeed, > 60%) quality images, I found that they were perceptibly less good at any magnification (e.g. comparing a full-scale image, with quality > 60, to even the 100% quality scaled image, with the former at half the zoom factor of the latter, the natural size for the scaled image was just fine but any enlargement was visibly corrupted). So scaling incurrs a perceived quality penalty.

Comparing half-scale images of various qualities against the half-scale image at 100% quality, I find the following:

QualitySize/KiBSize/Quality/KiBMagnification
30%401331.4
40%491221.6
50%581151.7
55%621111.7
60%671112.1
65%731122.1
70%811142.1
80%1021272.8
100%417417

In the course of this, it becomes clear that the changes are down to particular features in the image – a sprig of leaves on a twig of the tree (at 40% to 55%) in mid-stream, a fine detail of the spray of water droplets (at 60% to 80%) – which controlled the magnification needed to see glitches. At lower magnifications, the pivotal feature is simply unclear, or unseen; once the quality is good enough that no coarser detail shows flaws, the magnification at which flaws show is simply that at which the pivotal feature is clearly visible. Consequently, it is to be expected that other images may have different thresholds, due to their discernible features having different scales. Finally, for this rotated test image, let's look at third-scale images:

QualitySize/KiBSize/Quality/KiBMagnification 40%26641.4 55%33581.7 60%35582.2 65%38582.2 80%53652.9 100%197197

I find myself thinking that full-scale at a quality of 65% or so is about as good as there's any point saving, short of an actual thumbnail – and the test image doesn't work very well as a thumbnail; it's hard to see what it's a picture of at anything below 1/3 scale.

Now, I did that all with a rotated image, since that's what I initially opened the GIMP for; it remains to see how it compares with images that started out in the right orientation.

Plain test image

So next I selected an image that was the right way up to begin with to study. Since several of the pictures I've taken this year of spray bows (like rain-bows, but in the spray from a waterfall between where I work and where I live), I decided to select one with a faint but clear spray bow from a sunny day when Akerselva (the river) was running at roughly its normal flow-rate, yielding some delicate structure of the water-fall's froth against dark stone. First, I simply saved this with quality values of 45%, 55%, 65%, 75% and 85%. Sure enough, the last is bigger than the raw data, re-inforcing the hypothesis that it's at about 80%. Here are the data for these images, assuming 80% for the raw data:

QualitySize/KiBSize/Quality/KiBMagnification
45%2234962.3
55%3326043.0
65%362556
75%402535
raw445557
85%471554

Well, maybe the raw image is a little above 80%, then. The pivotal features in the image were delicate features in the spray-bow and the water-fall, exactly what I picked the image for. Again, the images generally tend to look better than the raw one at magnifications reasonably below the threshold at which their deficiencies become clear – I think the compression is effectively doing crude edge-detection, which gives crisper edges to features in the image. The image quality is also discernibly better than for the rotated image – in which quite a lot of the artefacts appeared to be cross-talk between the minor vertical artefacts from the raw image being saved at less than 100% and the horizontal artefacts from being re-saved after rotation. Again, 65% seems to work nicely.

Next, I tried scaling the image down to half size and saving at 85% quality; however, as before, any amount of zoom revealed deficiencies (albeit less prominent than before) relative to the original (at half the zoom, so as to match the scaled image). This time I shalln't even bother exploring the variation.

Of course, it should be noted that the comparison being done here depends on Opera's zooming, so what I've really found is that scaling down with the GIMP and then back up in Opera produces poor results when compared with simply scaling down (by not as much) in Opera. This should be no surprise; if the final image displayer has the raw data, it can do a better job of displaying it nicely scaled than it can if it has to undo some prior scaling.

Conclusion

My camera seems to save JPEG with a quality value a little over 80. If it used a higher quality value, the ones I need to rotate would come out discernibly better, but it's probably making a good pay-off between image size and quality. For practical purposes, I can save around a fifth of the image size by saving at 65% quality without appreciable loss of subjective quality. Scaling images down works fine until someone wants to zoom in on them, but having data adequate to display at larger scale than needed enables the user agent to do a better job of displaying at diverse scales in the range that is needed.


Valid CSS ? Valid HTML ? Written by Eddy.