SpatialDB Advisor
Choosing a Tile Size · 570 days ago by Simon Greener
The most critical property of an image catalog that needs to be defined on its creation (using the Catalog Registry Tool) is the size of an individual tile.
The scientific basis for what I am about to say was an published paper by Dr Peter R. Lamb (CSIRO Division of Information Technology) entitled: Tiling Very Large Rasters. This paper was published in the Procedings of the 6th International Symposium on Spatial Data Handling, Edinburgh, Sept 1994. (The paper is not available on the web: you have to use a library to get a copy of the paper!)
I am going to do a Camtasia video on how to choose a tile size based on this paper with some practical examples. But here is the gist of what it will contains.
Determine the average display window of the GIS clients that access the image catalog and divide the window by 4.
For example, if the dominant visualisation application using an Image Catalog (or Oracle’s excellent GeoRaster technology) is a web application with an output image size of, say, 800×600 pixels that all image catalogs should be tiled at 200×150 pixels.
Pixel size, because it is a normalised measure it makes computing the tile size of an image catalog easy. However, most users of fat-client GIS applications don’t know the size of the display area in pixels.
To “map” this to a projected distance one would multiply this by the pixel size. So, large scale orthophotography at 1.5m/pixel the tile size expressed in meters would be: 300m x 225m; large scale cartographic data (eg 1:25,000) @ 12m/pixel would be 2400×1800m; 1:500,000 at 100m/pixel would be 20,000m x 15,000m.
Effects of getting the tile size wrong.
Peter’s paper includes a number of graphs that show access time plotted against tile size.
What is common across all these curves is that setting a tile size that is larger than the optimum incurs less cost than setting the tile size to be smaller than optimum. The curves become much steeper, faster with too small a tile size running the danger of generating high access times (the curve’s asymptote or limit is being approached).
If our image catalog tile size was set to 200×150 pixels and a fat client was being run at high resolution 1280×1024 (the optimum tile size for this being 320×256 pixels) or above then we might start to get on to the steeper part of the curve increasing disk activity and slower access times. In this case it might be better advice to divide the average display window by, say, 3. However, statistics (or estimations of actual access) will help determine whether this is a good idea. So, if the fat-client is generating only 5% of the accesses to the image catalog then one would not bother with a large tile size. But if the fat-client access was generating 50% of the access then … but, as with so many things, it all depends on context!
Finally, I have not discussed the effect that database block sizes (such as are used in Oracle database) and logical block read settings might have on the choice of tile size. One might also want to consider disk sector size (eg FAT32 vs NTFS etc).
Finally, for disk/file based image catalogs there is the question of the number of physical files within a folder/directory. This does have a major impact on performance. But such a discussion is for another day…
Simon
Comment [2]
Tile Map Service Specification · 570 days ago by Simon Greener
On the OSGeo website there is a new posting describing the Tile Map Service Specification
From the Document Scope:
“This standard specifies the behavior of a service that provides access to georeferenced maps in the form of a regularly spaced set of regularly sized squares at a finite number of scales. This standard specifies operations to retrieve a computer-readable description of the service instance and to retrieve particular tiles.
A Tile Map Service provides access to cartographic maps of geo-referenced data, not direct access to the data itself. This document standardizes the way in which map tiles are requested by clients, and the ways that servers describe their holdings.”
It is nice to see the use of tiled map libraries becoming more visible and getting some respect. Post MrSID and ECW there was, I think, a presumption that technology like image catalogs were “old technology” and we know what such a pejorative tag being “old” is in a buzzword driven world of modern IT.
(My view has always been that a professional in the geospatial industry should know the science behind the technology we use and be able to select the most appropriate for the job at hand. ECW, MrSID and JPEG2000 because of their use of wavelet compression techniques are appropriate for certain types of data – mainly photography and satellite imagery – and in certain types of application use but not for cartographic maps generated from vector data.)
It appears that the increased visibility of Tiled Map Services such as Google Maps etc, has meant that the issue of image catalog use is being revisited.
About time.
I have had a brief read of the specification and can see that an image catalog created using my Image Catalog Tool could be served over the web as a Tile Map Service via a suitable interface.
A few observations though. The specfication says:
“The Tiled Web Service provides access to resources, in particular, to rendered cartographic tiles at fixed scales.”
Why does the specification continually refer to the tiles having been cartographically rendered? Perhaps this is just semantics but I have never thought of an orthophoto or satellite image as being a “cartographic rendering”! Yet these can be stored in an image catalog and delivered via a Tiled Map Service.
Secondly the specification talks about “fixed scales” and even has these in the XML reponse:
<TileFormat width="256" height="256" mime-type="image/jpeg" extension="jpg" /><TileSets><TileSet href="http://tms.osgeo.org/1.0.0/vmap0/levelzero" units-per-pixel="0.703125" order="0" /><TileSet href="http://tms.osgeo.org/1.0.0/vmap0/levelone" units-per-pixel="0.3515625" order="1" /><TileSet href="http://tms.osgeo.org/1.0.0/vmap0/leveltwo" units-per-pixel="0.17578125" order="2" /><TileSet href="http://tms.osgeo.org/1.0.0/vmap0/levelthree" units-per-pixel="0.08789063" order="3" /></TileSets>Now my Image Catalog Tool does not enforce a single fixed scale for all tiles stored within it. Yes, it allows for the setting of a minimum pixel size but individual tiles could have greater resolution. This seems to me a more natural thing to do where image catalogs are being constantly added to. For example, at my last employer, spot aerial photography was done to cover forest operations as they commenced during each year. The spot aerial photography was done using digital cameras and low flying aeroplanes yet correctly orthorectified. The resultant imagery was, because of the method of capture, variable in scale (though the pixel resolution was approximately the same). Perhaps I should consider creating a property (managed by the Catalog Registry Tool) that enforces the minimum pixel size as the fixed scale for the catalog. (If any user of the tool wants this let me know.)
The specification also seems to assume that an image catalog underneath a Tile Map Service has a single area of coverage. That is, that the whole of the area defined in the bounding box xml:
<BoundingBox minx="-180" miny="-90" maxx="180" maxy="90" />has tiles and that the tiles contain real data.
My experience in using image catalogs within a dynamic business is that often this is not the case. Because of this I added in the automatic management of a <image_catalog>_boundary.shp shapefile. This shapefile contains a single multipart polygon that is the outer boundary of all existing tiles (in my image catalogs not all tiles have to exist). This is done by unioning all the tile boundaries together after any edit operation.
Just some simple observations on image catalogs and the new Tile Map Service Specification. Regardless as to my quibbles with the assumptions in this standard, I am genuinely pleased to see that the need for such as specification has resulted in it being created and published.
Simon
Image Catalog Tool · 571 days ago by Simon Greener
I thought I would start a dedicated section on my site for the Manifold GIS based Image Catalog Tool.
Image catalogs are used in ESRI technology, Manifold GIS and software such as the UMN MapServer. My image catalog tool, while not perfect, does (once you have mastered the tool and built some management practices around its use for your organisation) make the creation and management of image catalogs easier. It also facilitates the easy update of imagery in a catalog as well as the creation of single image mosaics from an image catalog.
The image catalog tool is more than just a tool to insert images into an image catalog. The tool comes with two components (a hyperlink means that there is an introductory video associated with the component or function being discussed).
The Catalog Registry Tool is used to create and manage an image catalog. Via this tool one can define an image catalog’s properties (metadata) and have the directory structure and shapefiles created. The metadata that is defined and managed by the Catalog Registry Tool makes the operation of the actual Image Catalog Tool far easier.
Once a catalog is defined one uses the Image Catalog Tool to add imagery to an image catalog. Adding imagery could entail the creation of new tiles or the updating of existing ones.
The Image Catalog Tool has many strengths over existing management tools.
- It is free (expect for a Manifold GIS license of USD250);
- It enables an image catalog’s image quality to improve over time. It does this because if an existing tile has a pixel resolution that is coarser than any new image being added to the catalog that tile is upgraded to the smaller pixel resolution of the new imagery. In this way an image catalog’s image quality only improves over time and does not degrade. (See discussion on “minimum pixel size” in the Catalog Registry Tool.)
- Single image mosaics can be generated from a set of image catalog tiles.
- Multiple tile storage and “world file referencing” formats are supported.
- A history record for each image that has been added to an image catalog can be created. The image’s spatial extent is saved as a footprint that can be visualised in a GIS.
- A boundary shape defining the actual extent of the data is automatically created after each execution of the tool. This is useful for visualising areas in which tiles exist before zooming into particular areas.
The Image Catalog Tool is based on published science. This science allows us to define a optimum tile size for an image catalog so that disk access times are kept to a minimum improving visualisation response times in internet and interactive GIS use.
Over time I will add videos and documentation on all aspects of the tool until such a time as Manifold.Net incorporate the ideas in the tool into the product.