Saturday, 23 March 2019

NZ Rail Maps: Gimp optimisation with SSD works well

A few posts back in label NZ Rail Maps, I commented on how to optimise Gimp to use the swap partition on the computer effectively. I have since been able to prove using a computer with only 8 GB of RAM that it is able to use the SSD to load much bigger images without much of a performance hit and this validates my ideas about having SSD based swap to make Gimp perform well with very large projects that contain a lot of aerial photography data because this is how the mosaics get made in NZ Rail Maps.

The reason a large SSD based swap partition is becoming so important is that it speeds up the process of creating map mosaics from historical aerial photography. It also works well with Qgis when I am loading these large numbers of aerial photo layers into the GIS to draw the maps. However it is mainly Gimp that benefits, especially as I am taking advantage of the optimisation of resources to be able to create projects of increasing size.

Generally these large projects will cover a relatively large area. For example my largest ever project, currently around 24 GB, has 128 layers at present and this covers around 17 km of physical track distance (Dunedin to Mosgiel) with aerial layers present for the entire distance as well as other areas like the Fernhill and Walton Park branches and the first section (to Mosgiel Township) of the Outram Branch. There are still more historical aerial images to add to this map and so far the computer is coping quite well with working on it with about half the SSD in use for swap. So I expect this project will be able to grow quite a bit more yet.

Compare that with the previous largest project on this computer (although it's possible it may have had less RAM then) around 12 GB, but without optimising Gimp. The issue is simply if you leave that tile cache setting at the default then the Linux swap partition will not get used at all. Instead, Gimp will use its own swap space, which on my system is in the home volume. This means a lot of noisy and slow hard disk swapping. 

This essentially helps me to build a case for buying a larger SSD specifically for swap use. At the moment there is one SSD in the system which is mainly used for the root partition and some other OS stuff and most of it is a swap partition about 100 GB. A 240 GB SSD would create a large amount of swap space. The other option is to buy a 480 GB SSD for the NUC and put its disk into the computer to be used as a 240 GB swap volume. Anyway the cost of a SSD is around $80-120, so it is on the list of priorities to hopefully achieve soon.