Part 6:  Knowledge management and communication

Spatial and remote sensing data management review
J Lowry
Introduction
Over the last eight years, SSD has invested substantial resources in the acquisition and creation of spatial data. However, SSD has reached an important cross-roads, in terms of continuing to acquire, collect and crucially, manage spatial datasets. Unless an appropriate management system is put in place, the cost-effectiveness of acquiring, generating and maintaining spatial data will be reduced, as increasing resources will be required to store the data. At the same time, the value of data of unknown quality or of data lacking accurate documentation will steadily diminish. Conversely, the spatial datasets collated by SSD have the potential to significantly increase their value if dedicated resources were made available to support data management.
Consequently, the current status of data management protocols and the utilisation of network server space for handling of GIS and remote sensing data, and photographic images was reviewed internally over the period of February – June 2008. The review built on, and enhanced existing data management recommendations (eg Finlayson & Bayliss 1997) to develop new protocols and guidelines for spatial data management which reflected the current versions of software now in use, and current national and international data management standards and procedures.
The brief specifically called for the review project to:
•	Collate the procedures and protocols developed by the Spatial User Group (SUG) for spatial and remote sensing data, to produce an SSD operations manual for the handling of these types of data;
•	Oversee implementation of the partitioning of the Network Attached Storage (NAS) system to create a dedicated space for storage of spatial and remote sensing data;
•	Oversee SSD-wide implementation of the new data management operations protocols by negotiation of data migration timelines with the primary data custodians;
•	Establish a cataloguing and directory structure framework and place all primary acquired GIS and remote sensing datasets into the corporate server environment;
•	Develop and implement a strategy for offline or slower hard-drive archiving of completed project material – ultimately this could be applied to all non-current (say greater than 3 y old) project material stored in SSDX;
•	Develop and implement a strategy for cataloguing and storage of digital photographic images using as a basis the database developed by Andrew Esparon:
–	In the first instance implement a framework for storing of all future digital images on a common platform;
–	Develop a strategy for the progressive migration of historical images to the new platform;
•	Identify current impediments to effective sharing and access of GIS and remote sensing data with ERIN (Environmental Resources Information Network) in Canberra, and identify hardware upgrades needed to address this; and
•	Present a report to the Technical Data Management Group outlining the review findings and recommendations, and present a final consolidated report to the SMM for consideration.
Results
The key output from the project at the end of June was a data management plan, with a list of conclusions and short and long-term recommendations for the creation and management of spatial data. The report also identified and developed a number of standards and operating procedures for the creation and management of spatial data. These included the quality assessment and quality checking procedure for datasets as they are created and added to the corporate spatial database (Figure 1). 
The cost and resource implications of the recommendations contained in the review are currently being considered by management within SSD, with certain elements allocated a higher priority. Key recommendations made in the report include:
•	The need to recognise data management as an integral element of every project, and to ensure adequate resourcing is provided and recognised for this;
•	Signficant resources will be needed to ensure staff involved in the creation, manipulation and analysis of spatial data have adequate training and time to undertake and implement the data management protocols and QA/QC procedures recommended in the report;
•	The establishment of a dedicated data management position to oversee the management and storage of spatial data; 
•	If configured appropriately, existing hardware and software infrastructure could be used to implement an efficient data management environment which mirrors, on a smaller scale, the environments and standards developed by ERIN in Canberra. There are significant support benefits in ensuring the infrastructure and data management protocols are aligned or similar to those applied by ERIN.  
Steps for completion
The technical recommendations contained in the report are in the process of being implemented.
The critical element to ensure the success of the data management strategy is ensuring that sufficient resources (training, time allocation) are provided to all staff involved in the creation, manipulation and management of spatial data. 
The strategic staffing issue regarding coordination of data management in SDD is yet to be formally considered by the Senior Management Group. 

7. Fix Topology errors using appropriate Geodatabase ArcTools

10. Dataset ready to be imported into spatial database e.g. ArcSDE

2. Create Geodatabase(s) and import feature data

3. Set datum and projection (default standard should be GDA94 - Geographic coordinates)

5. Fix Geometry using ‘Repair Geometry’ tool from ArcTools

Errors identified

No errors identified

Run ‘Check Geometry’ again to make sure no errors remaining

8. Update metadata if required, to state nature of tests performed. Tests should be described under Lineage.

* the purpose of these steps is to ensure that datasets do not contain slivers, gaps, unwanted overlaps, unclosed polygons, missing attributes, self-intersecting polygons, unwanted dangling lines, and line and point features at an inappropriate density to the published data scale.

6. Build and check topology* of data using topological analysis tools

1. Rationalise data types to ESRI Shapefile format & geodatabases

Interim step whilst geodatabase skills developed and geodatabases established and populated.

9. Dataset review coordinated by data manager

Errors identified

4. Check Geometry* of datasets using “Check Geometry” tool from ArcTools

Figure 1  QA/ QC procedure for vector datasets
Acknowledgments
The spatial data review and related data management plan was produced through a consultative and iterative process with input provided by members of the eriss Spatial Users Group (SUG) and the eriss research and data management group (ERDMG). Valuable input has also been provided by Damian Woollcombe and Mike Maslen of ERIN, and Che Diggens of AMS Pty Ltd.
References
Finlayson CM & Bayliss B (eds) 1997. Data management systems for environmental research in northern Australia: Proceedings of a workshop held in Jabiru, Northern Territory, 22 July 1995. Supervising Scientist Report 124, Supervising Scientist, Canberra.