Monday, 18 November 2019

NZ Rail Maps: Large Area Raster Tiles At Reduced Quality Setting

Using Qgis to draw the maps, we've run against a few times some sort of limitation on the number of raster layers that it seems to have, which runs out a long time before hitting the actual resource limits of the computer it's running on. This in turn limits the number of raster layers that can be included in a project which has created quite a few challenges in creating map volumes. So a few times we have looked at ways of making bigger raster layers to reduce the physical number of them used. At the same time we have had a play with JPEG quality, reducing it from 100 to 50, which drastically cuts file size. 

For example, over Lyttelton, we have a mosaic with a resolution of 57600x28800 (1,658,880,000 pixels in total) and there are in total 48 tiles (12x4) at 4800x7200 resolution. The 48 tiles downloaded from LINZ are 600 MiB in total. For the 1967 era layers we produced a total of 32 exported tiles which also reached about 600 MiB in total at quality of 100. However the large area tile that we produced for this article covering the entire 48 tile space with quality 50 only uses 156 MB which is quite a lot smaller overall, for only an imperceptible loss of quality. We may even try lower quality settings but for now, 50 is where we will leave this, as it will have to be tested over a wide range of mosaics.

Creating the full size mosaics is also much faster as there is only one tile output for each era and this means we will be doing all the exports for Greater Christchurch Maps in future at full size for each mosaic project in order to speed up the exporting process. Some additional work may be needed to make the areas covered be full rectangles without gaps as these will generally be exported as black areas that we don't need to be in large tiles. As it happens, with the Lyttelton examples, a number of large tiles will have the Linz 2015 contemporary areas included around the edges by default as the historical areas often do not cover the full canvas.

The main challenges are where a base area is not a perfect rectangle, meaning it will have to be broken down into two or more areas that are, and the time that it takes Qgis to refresh its canvas with the larger layer size. At the moment these issues are not too challenging and we expect them to be able to be worked around satisfactorily. Making the sidecar files is extremely straightforward as all that is needed is to copy the sidecars for the top left tile of the original tile grid and rename it to suit the new file name specification, due to the fact that the only coordinates in the world file are at the top left corner and the software assumes it is just meant to keep drawing the tile until all pixels have been rendered.

We have also observed that Qgis doesn't seem to have an issue loading raster tiles from a WMTS server in our projects that also have local raster layers, so the option is there of perhaps working out how to have a WMTS server on our local network to serve the raster tiles, instead of being file based, but also to query the difference in architecture of the software, as it is evidently handling WMTS layers in a better way than local file based layers.