The Automated Map. Public innovation for the generation of the National Topographic Map of Spain

Several reasons have prompted the National Geographic Institute of Spain (IGN-Spain) to implement an automatic process to generate the National Topographic Map 1: 25,000 (MTN) instead trough the traditional manual way, pointing out the growing lack of human resources, in addition to the search for a quick response to the increasing demand of updated geoinformation by the society. This new automated process provides an annual production of all the map sheets composing the MTN25 (4.019 files), what is an unprecedented time record, so that the users can quickly both download them from the Download Centre Website and visualize the maps through the visualization web services WMS and WMTS. This methodology is also applied to the creation of sheets for printed publication, whose final output requires a simplified manual editing process.


Introduction: why an automated map?
The production of cartography has undergone a strong evolution thanks to the emergence of new technologies, both in the geographic features capture (using direct and indirect methods) and in cartographic editing and drafting and in the products dissemination as well. This evolution can be observed in the aspects described below.

Data capture tools and methods
Data capture methods have evolved enormously in the field of surveying. Initially, classical methods based on direct data collection on the terrain were used; after that, the data collection was streamlined by applying photogrammetric techniques (from analogical and digital restitution to the orthophotographs production), up to the present, where digital elevation models are generated from LIDAR data. This technological evolution has promoted the accessibility to higher resolution data more with a higher frequency, which has directly impacted on the cartographic production flows.

Cartography production methods and workflows
The National Topographic Map [1] production, moreover being one of the legal mandates of the IGN-Spain, is a public task of a strategic nature as it is representing the cartography of a country. Therefore, when the first works began, a large part of the IGN-Spain's resources were allocated to this objective as the MTN was the most important product, in addition to being the basis from which other derived mapping products arose, as well as new spatial planning applications, engineering projects, etc. Consequently, the tasks of collecting, preparing and editing were carried out directly on the map sheets. However, currently the workflows have been reversed so that, if formerly the production rate was set by the map, nowadays it is subject to the rate of features updating that is carried out in the National Topographic Database 1:25.000 (BTN) [2], from which the map is currently generated.

Production environments
The technological environments have evolved in such a way that the tasks of capturing, preparing and editing have gone from being carried out with CAD tools, to being updated with GIS tools.

Fields of action
Until recently, the production unit for both the MTN and the BTN has been the map sheet. However, the update methodology applied to the data sources from which the map is generated uses other geographical areas, from the province to the whole national territory. These actions require updating a large number of MTN sheets that are affected.

Data dissemination
In a short time, it has gone from publishing printed editions of the map to disseminating large volumes of geographic data on web visualizing services (WMS/WMTS) and download tools. Currently, the MTN is also available through all these channels, so that it can be easily viewed alongside other data sets such as geographic databases and aerial or spatial imagery. This is one of the most important reasons why the map update should be as synchronized as possible with the other data sets.

Decrease in resources
Editing and preparing maps is a very expensive process. It demands highly specialized personnel because it has a very strong aesthetic component. The IGN-Spain has achieved in its history a large staff of draftsmen and cartographic specialists who have carried out this work, first with analogical methods, and later with digital processes. This workforce has been progressively reduced and much of the knowledge has been lost. On the other hand, the economic resources allocated for hiring have also suffered a significant reduction. Consequently, the volume of sheets that may be published is low compared to the diffusion of other data sets produced by the IGN.

High demand for up-to-date information
In addition, users demand more and more data updating, promoting the need of a faster and more efficient production method.

The need to make the IGN-Spain geographical databases visible
The MTN is a high demand product, known by the entire geographic information user community. The MTN is a very important window to view the updates produced in their original data sources. Consequently, the producers of these data also demand that their updates will appear as soon as possible on the map. The evolution of the IGN production process asks for a deep change in the MTN production model, almost imperative. Technological advances present an opportunity to carry out this transformation, largely oriented towards the automation of the cartographic editing, drafting and publication processes. Furthermore, implementing this disruptive change also implies assuming its pros and cons: the results will not get as much high-quality as the classic map sheets, in exchange for having a high publishing frequency, including the most recent feature updates made in the database.

Description of the workflow to create the automated map
The automated map is obtained by executing a flow of automated processes that try to emulate the traditional procedures used in the elaboration of a traditional map. It consists in two phases: the first one, where generalization and editing processes are applied until obtaining a first output in vector format, and the second one, where symbolization and rasterization techniques are applied until obtaining the final output of the map, in raster format in this case.

Generalization and edition processes of vector data
These automated processes apply geometric manipulation techniques to the geographical features stored in the databases, in order to make their cartographic representation legible: • Generalization to the display scale.
• Classification and symbolization according to a map legend. • Features edition to ensure readability.
• Features labelling with a specific label style and placement in order to identify the labelled item unambiguously. • Aesthetic quality, as far as possible, of the resulting map, making this point one of the most critical on the automated map. These action rules have been formalized through a series of synchronized automated generalization and editing operations within a workflow. The difficulty of automated generalization is known, among other reasons due to the choice of generalization parameters adjusted to different situations (parameters that a cartographer would naturally assume when generalizing by hand each area of the map). For this reason, throughout the entire automated flow, many algorithms analyse the context of each feature in order to decide the best treatment strategies.
Thus, for example, the building density will be decisive in the choice of the type of treatment to apply in each area containing buildings.
Another difficulty of automated generalization is the high dependency between map elements, triggering looping repercussions when acting on an element and messing up elements worked in previous steps. In order to overcome this drawback and avoid iterative processes, the flow is based on the execution of a sequence of processes ordered according to the restrictions on the alteration of the elements they modify. Thus, first of all, actions will be carried out on the features with the greatest modification restriction, and then, on features with less restriction. Finally, other difficulty of the automated generalization process is that it acts on large "alive datasets", and so, it is necessary to know their data models and the possible relationships existing between them. This highlights the importance of the consistency in the starting data sources. The automation of this process requires starting from harmonized data sources, with known and assured semantic and topological relationships. The implementation of this flow is a succession of synchronized processes implemented mainly in Feature Manipulation Engine (FME) [3] and some Python scripts from the ArcPy library. FME is a powerful tool for data conversion, sharing, transformation, validation and integration. It contains more than 500 transformers that perform different types of data operations (readers, writers, geometric operators, filters, etc.). These transformers can be displayed on a graphical canvas, and connected between them in order to model a workflow. Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. It is an ideal language for scripting and rapid application development in many areas on most platforms [14]. The flow is orchestrated by a main tool that sequentially executes each of the following processes or modules on the list of sheets to be processed:

Data sources
The first step consists in doing the data extraction from the following data sources (databases), clipped with the sheet frame to be processed: • BTN, once it has integrated the transport theme from the Transport Networks Database. • SIGLIM, administrative boundaries database.

Preliminary treatment
Preliminary treatment of the extracted features through Python scripts, using Arcpy libraries: • Buildings outline simplification, by eliminating details less than a superficial tolerance, since the building layer from the cadastre has too much detail for the map display scale. • Unification of all the carriageways in motorways.
• Progressive displacement of linear elements, starting from the most fixed such as hydrography or railway to the most mobile such as roads or paths. • Generalization of the urban roads network.

Mapping the data model
It entails a model translation, from the BTN data model to the symbolization one, in order to provide to the geographic features, the visual properties (colour, thickness, style, font)

Linear features process
Treatment of linear features that seek greater readability on the map, such as: • Generation of representative features of tunnels and bridges in communication routes. • Generalization of enclosures (walls) based on their density and arrangement on the map.

Building features process
Treatment of building features. It is a module with a high processing load since the buildings undergo numerous actions to achieve a legible representation at the output scale, while preserving the representativeness of the area. By symbolizing buildings and communication routes through punctual symbols and lines with sizes and widths perceptible to scale, overlaps are generated between these elements that are not actually produced.
The action strategies are based on a preliminary density analysis generating three kinds of partitions: • Low-density buildings, scattered type or isolated buildings: in this partition the process tries to separate punctual and linear elements to improve readability. • Medium-density buildings, housing type developments: filtering is performed to eliminate overlaps between symbols, maintaining the density pattern of the area.
• Buildings in high-density area: in this partition the objective is to maintain the urban skeleton that characterizes the urban area essence. In this way we can find building amalgamation, buildings clipping with the urban roads, or block replacements of the large buildings.

Altimetry
In this step a selection of the most representative elevation points is carried out.

Specific symbols
A series of actions are carried out to improve the readability and aesthetics of the map: • Reposition of some point symbols with regard to linear elements associated, for example, landmarks on roads. • Symbolization of areas through patterns of punctual symbols: solar plants are filled with symbols of solar panels or mine areas with the mine symbols.

Labelling
Labels are generated for map features with a predefined style, based on the classification of the labelled element. This module is subject to constant evolution process to solve implementation difficulties, due in part to the large number of labels coming from BTN and the absence of a label hierarchy for the same class.

Land cover process
Integration of the coverage layer from SIOSE: translation between the SIOSE data model and the MTN legend, and geometric corrections of coincident features, such as contour of water sheets or other boundaries.

Database store process
Storage of the process outputs in a PostGIS database: the geometries from the dgn files are stored, organizing the information in tables by topic, with attributes that facilitate subsequent symbolization.

Automated map generation process in raster format
This process is made up of the symbolization flow and the rasterization flow:

Symbolization flow
The following operations are carried out: • Semantic attribution: Some attributes are generated through a look up table on the data in PostGIS. Such attributes, as the layer and the description of the feature, allow for symbolise the map elements. • Import into MapPublisher: Through a template designed on Adobe Illustrator and FME, the information is loaded and the corresponding symbolization is applied to that template. • Layer structure. Display order: the layers are ordered according to their priority when displayed on the map. • Export to GeoPDF: Two format files are generated Adobe Illustrator.ai and geoPDF.

Rasterization flow
The raster embedded in the geoPDF is reached and by a series of threads within the general flow of rasterization, it is georeferenced and scaled. After this subprocess, the raster files corresponding to each sheet are written in geoTIFF format. In this geoTIFF, the shading obtained from the DTM has also been incorporated, masking it in some areas, such as roads and body of water. Once the shaded geoTIFF files have been obtained, they are compressed, obtaining images in ecw format. At the end of the automated process of each MTN sheet, the following files and formats are obtained:

Automated map process for printed edition: simplified editing
The goal and scope of the simplified editing is to generate editions of the MTN through a process optimized in time and resources. The data used are: the content of the map sheet obtained automatically with FME processes from the database sources, and the marginal information of the map sheet, which are metadata obtained and represented by automated processes. It is carried out through a manual process with a final edition of both, content and exterior of each map sheet, rigorously adjusted to a check-list tasks, in order to grant that the final result agrees the product specifications and the quality requirements for the printed publication:

Figure 3. Extract of items to review in the simplified editing process
The main differences between this method and the traditional process are the following: • Only editing work is carried out, that is, updating tasks are not carried out here, since it is assumed that the update was made in the data sources. • It implies less editing workload because: -It is part of a product in which an automated editing and generalization has already been carried out. -The editing criteria only attend to readability issues and to correct the results of the automated editing processes that do not give us a desirable solution, but not the traditional MTN specifications.  Production is carried out through contracts with a scope of approximately 800 sheets per year. For the management of such a high number of sheets, an information monitoring flow has been implemented in the DMZ (Demilitarized Zone) that is a perimeter network that allows organizations to protect their internal networks from untrusted traffic. This online project management system allows the monitoring of deliveries through requests, transferring files, Gantt chart, news, forums and technical documentation repository. In addition, for the file handling, the automation of packaging, registration of items in the different control databases and file transfer until they are sent to IGN print department, have been developed.
As a result of the simplified editing work, an improvement in the processes and tools of the automated map is being achieved. The doubts, troubleshooting, detected failures that are documented with the Redmine program, are so helpful to carry out improvements encountered in the process, in the checklist and in the tools developed to generate the automated map and the symbolized sheet.

Figure 6. MTN25 production evolution graph
In addition to the simplification of the process as a result of automatic production, it has also been achieved: • The design of a protocol focused on solving problems that the tool cannot solve yet, focused only on editing. • The implementation of the check-list protocol as a quality record, allowing traceability for sheets, either with or without Quality Control. • The check-list allows the implementation of a flow based on a batch quality control with acceptance or rejection, which represents an 80% reduction in reviewing times in the IGN. In conclusion, the simplified editing has a number of advantages: • Minimum time between the data updating and their publication in printed cartography format. • Less use of resources, eliminating overlaps between tasks. • Lower production costs. • Greater objectivity.
• Control of process times and results (quality record) • Possibility of Continuous Improvement (quality indicators). Although we must not forget that they are advantages even at the cost of reducing the quality of cartographic editing and increasing the risk that information with errors will not be detected.

High resolution map
Currently, a new higher resolution cartographic product is being developed for its publication through web services. It will also contain the information from the same data sources used in the automated map and it will have the symbology adapted to the resolution of each source, obtaining a result similar to the Automated Map a scale 1: 25,000, but with the geometries hardly generalizing or manipulating. The output of this High-Resolution Map is adapted for being viewed via web viewers, where the zoom removes the restriction of the visual limit of the paper, allowing the symbols and texts to be interpreted with smaller pictograms. These smaller symbols avoid applying automatic generalization and automatic editing processes to geometries to fit the available space. Thinking, for example, in the communication routes, they will have a thickness less than the width they occupy in reality, so that it will not be necessary to apply displacement processes to avoid overlaps as it is done in the 1: 25,000 automated map.  This use of smaller symbols greatly facilitates automatic labelling, because being the texts smaller, many more labels fit, avoiding overlaps. In short, this product has a series of advantages if it is compared the 1: 25,000 automated map, such as those listed below: • As generalization is not necessary, we do not lose the geometric detail of the sources of origin, obtaining a result closer to reality. That brings us to a higher resolution product: higher detail in vector format and smaller pixel size in raster format. • Labelling is greatly improved by being able to label more entities. • Implementation is simplified by not having to introduce automatic generalization and editing processes. This not only facilitates implementation and maintenance but also reduces the number of errors due to automation. As a disadvantage, there is a greater weight in the raster files obtained, which makes us consider other lighter formats such as Cloud Optimized GeoTIFF(COG).

Conclusions
With this public investment in the innovation of the generation of the most emblematic product of the IGN, the National Topographic Map, it is intended to respond to the high demand for geoinformation by society, which mainly prioritizes the speed of data publication. Without a doubt, having managed to generate the map automatically through processes that simulate the decision-making of cartographic operators is a milestone in the history of the IGN's cartographic production, and the stimulus to continue evolving. The next objective is to ensure that the cartographic quality of the final product is closer to that obtained by classical methods, in which authentic "works of art" were generated. The optimization of the cartographic representation in combination with the advantages provided by the automation of processes in terms of the degree of updating of the information and the increase in the frequency of publication of the entire series, makes the MTN a product that serves the requirements of its 21st century users.