Methodology

Data Acquisition

Data published through the Geoportal will primarily be obtained from government agency websites free of charge. The selection and acquisition process will be thoroughly documented. Data may be found in a variety of original formats such as: spatial datasets (.shp, .gdb, geojson, etc.), non-spatial, tabular files (.dbf, .xslx, .dat. etc.) or other formats (such as .pdf, .ppt, .doc, jpg, bmp, etc.).

As the quality of the information processed and published through the Geoportal is a direct function of the quality of the data, it is critical that the data selection process consider only those data that have associated metadata.

Data Processing

Once the data has been selected it will be necessary to start a data normalization process. This process includes stages of profiling, diagnosis, processing, structuring, organization and documentation.

• Profiling of the data. Data profiling refers to the analysis of the data to clarify its structure, content, relationships and associated rules. Profiling helps understand anomalies and evaluate the quality of the data.
• Diagnosis and work plan. Once the profiling of the data has been carried out, a diagnosis and work plan will be prepared that prioritizes and specifies the activities and the time required for processing the datasets.
• Processing and structuring of the data. Processing and structuring of the data will be executed using previously defined procedures leveraging ETL tools, SQL queries and/or other defined functions.
• Organization of the data. This process involves designing classifications and rules to organize the data sets.
• Documentation of the data. Dataset documentation will be developed by applying a metadata schema to provide the dataset history and other attributes.

Data Warehouse

A data warehouse is a relational database that is designed for querying and analysis rather than for transaction processing. It usually contains historical data derived from transaction data, but it can include data from other sources. It separates analysis workload from transaction workload and enables an organization to consolidate data from several sources. In addition to a relational database, a data warehouse environment includes an extraction, transformation, and loading (ETL) solution, an online analytical processing (OLAP) engine, client analysis tools, and other applications that manage the process of gathering data and delivering it to business users.

Data Publication and Distribution

Once the data is in the data repository, the cartographic design process can be started for publication to the Geoportal, with the following main stages:

• Design of styles for the representation of geographic data. The cartographic design of each layer to be published in the Geoportal must be identified and documented. To document the symbology of the layer, a previously defined format must be used.
• Layer publication in the Geoportal. This activity involves loading the geographic data layers in the Data Repository along with their configuration for the correct deployment within the Geothemes of the Geoportal. Procedures defined for this purpose must be followed.
• Review and validation of data publication. A set of quality control tests must be carried out to verify that the data have been published correctly. For this purpose, defined procedures will be followed.

The delivery mechanisms of the Geoportal are diverse and are governed by information and communication technologies and international de facto standards set by the Open Geospatial Consortium (OGC). For example, the so-called 'Catalogs', also called Spatial Data Directories, facilitate the search of spatial data on the Internet. This is achieved through a number of servers that contain metadata about existing spatial data (some of which may not necessarily be available free of charge), which allow users to search, visualize, transfer, request, promote and disseminate spatial data from different sources via the Internet.

Other mechanisms associated with information and communication technologies are Web Services, such as Web Services of Maps, (WMS, Web Map Service), of Features (WFS, Web Feature Service), and of Catalogs (CSW, Catalog Service for the Web). Implementation of these services requires software tools that are free, meet OGC standards, and that can be integrated and accessible through the Geoportal.

Data distribution can occur using either of the following two methods: the first is file transfer protocol (FTP) under an agreement between the client and the Geoportal team. The second method is distribution as open data formats using various tools. Therefore, it is necessary that the Geoportal be implemented with tools that allow data file downloads in open formats.