Contents¶

Permission is granted to copy, distribute and/or modify this document under
the terms of the GNU Free Documentation License, Version 1.3 or any later
version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts and no Back-Cover Texts.  A copy of the
license is included in the Appendix section.

Preface¶

The most up to date version of this documentation can be found in HTML and PDF form on ReadTheDocs.

Recommended User Knowledge¶

Users¶

This user guide assumes that users of the Data Selector tool have:

General IT experience including use of Microsoft Windows.
Experience in the use of a relevant GIS application supported by the tool (currently ArcGIS or MapInfo), including selecting and querying features and attributes.
An understanding of the data held in the Recorder6 database that has been configured for use by the Data Selector tool.

Administrators¶

It is recommended that a person within each organisation is designated as the tool and database administrator. This person should:

Have an understanding and experience of IT systems management.
Understand relational database structures.
Have an good understanding of how to write Structure Query Language (SQL).
Become familiar with how the Data Selector tool has been configured within the organisation.
Have a good understanding of XML.

Reading Guide¶

This Preface explains a little about the Data Selector tool, the community of people who develop and use it, and the licensing conditions for using and distributing it. It also explains how to read this user guide.

Introduction explains why the Data Selector tool is needed, what it does and where it comes from.

Extending Recorder6 is a brief outline of how and how Recorder6 can be extended to enable the Data Selector tool to get the most out of your biological data.

Setting up the tool describes how to install and set up the Data Selector tool.

Running the tool describes how to run the Data Selector tool.

Writing SQL introduces how to write SQL queries to be used in the Data Selector tool. [Coming soon!]

FAQs has a list of commonly asked questions and their answers.

Appendix contains examples of the XML configuration files for MapInfo and ArcGIS, lists known issues with the tool and contains a copy of the GNU Free Documentation License v1.3 covering this guide.

Licensing¶

The code for the Data Selector tool is ‘open source’ and is released under the GNU General Public License (GPL) v3. Users are free to install it on as many computers as they like, and to redistribute it according to the GPLv3 license.

This guide is released under the GNU Free Documentation License (FDL) v1.3. Permission is granted to copy, distribute and/or modify this document under the terms of the license.

Please remember, however, that the tool cost a lot of money to develop and still requires further development and ongoing support. Hence any contributions towards costs would be gratefully received. Enquiries can be made via email to either Hester or Andy.

Useful links¶

Acknowledgements¶

Many thanks are due to all the LERCs in the south-east of England and their staff who have, and continue to, fund and contribute to the Data Selector tool. It takes many developers, testers and users to build a truly useful tool (especially users who care enough to test new releases, report bugs and discuss feature requests).

Conventions used in this user guide¶

The following typographical conventions are used in this manual:

Ctrl-A: Indicates a key, or combination of keys, to press.
Commit: Indicates a label, button or anything that appears in user interfaces.
Tools… –> About: Indicates a menu choice, or a combination of menu choices, tab selections or GUI buttons.
C:\Program Files (x86)\MapInfo\Professional: Indicates a filename or directory name.

Tip

Tips can help save time or provide shortcuts.

Note

Notes explain things in more detail or highlight important points.

Caution

Warnings where users should pay attention.

Introduction¶

Background¶

Many LERCs use Recorder 6 for entering, collating and querying records of species and habitats. Recorder 6 is a very versatile system and is the package of choice within the LERC community where efficiency, flexibility and scalability are critical. For querying data it not only has an inbuilt Report Wizard which will produce reports from the most common data tables, but it also comes with a reporting system (XML reports) which can be used to report on all tables in the database, either by running one of the extensive selection of XML reports installed with Recorder 6 or by adapting an existing XML report or writing your own.

However, although the Recorder database structure is well documented it is very complex and requires time and expertise to master it - this can make it appear complicated for many users. One solution is to create an SQL script that exports all of the regularly used data from the complex Recorder 6 tables into one or more simplified SQL database tables (currently MapInfo or ArcGIS). These tables can then be used for both standard and bespoke reporting and analysis, and more importantly can be used to plot the data as points and/or polygons for use directly in a GIS application.

Note

The Recorder database structure is based on the NBN data model which is documented in the Documentation Wiki on the Recorder web site.

To facilitate the querying of data from these new simplified tables a new interface was developed - the Data Selector tool. The Tool was originally developed for use with MapInfo for Greenspace Information for Greater London (GiGL) but is now available for use with both MapInfo and ArcGIS and is used by a number of LERCs.

Tool overview¶

The Data Selector Tool presents a simple user interface to extract information from Recorder 6 based on the selection criteria and desired format specified by the individual user. It is integrated into the user interface of the installed GIS application (currently MapInfo or ArcGIS) - appearing as a button or menu item. The tool itself has a simple interface (The Data Selector tool interface), requiring a minimum of input (a set of SQL clauses to be used in the selection, an output format and an output file destination are the only required items). Once set up, the tool communicates with both the GIS system and an associated SQL database to extract the required data. The output files can be created as MapInfo TAB format (.tab) files or ESRI shapefile (.shp) files (depending upon the installed GIS application), Comma Separated Value (.csv) files or plain Text (.txt) files.

The Data Selector tool interface

Queries used in the tool can be saved and loaded so they can be used again-and-again and run by all users. When the tool runs the query entered by the user will be passed to SQL Server to be executed against the selected SQL table. Any errors in the syntax of the query will be displayed in a pop-up message. The results of a successful execution will be saved in the required format and, if the output is a MapInfo .tab file a symbology will be applied to the records if required. The tool also generates a log of the execution results which details the steps that were taken during the process and the number of records selected and output. The process is discussed in this document in the section on using the tool.

Defining the default values for some options in the tool is done via a configuration document written in XML. Using this document the user can configure: * The default location of the output folder and the default output format. * The location of the default folder for saving and loading queries. * The default columns used for plotting non-spatial data, or re-plotting already spatial data (MapInfo version only). * Definitions of different sets of symbology to apply to output .tab files and the default set to use (MapInfo version only).

Using this configuration file, each individual LERC can tailor the Data Selector Tool to its individual requirements. Examples of the XML file are included in the Appendix, and the process of setting up this file is discussed in the section on setting up the tool.

Benefits¶

There are a number of clear benefits to using the Data Selector tool for carrying out data selections from Recorder6.

The tool enables the user to query and extract data directly from the Recorder6 database using a simple user interface within the GIS application without even needing to start the Recorder6 application.
Queries can be saved for future use which is particularly useful for increasing efficiency if a query is run on a regular basis.
Queries can be created by more experienced users in the LERC, or commissioned to be developed by a 3rd party, to meet specific requirements. It is even possible to share queries between LERCs (if based on a common set of attributes).
Query results can be plotted directly in GIS applications, as MapInfo (.tab) tables (MapInfo version only) and ESRI (.shp) shapefiles, for immediate visual and spatial analysis, as well as being output as spreadsheet (.csv) files and text (.txt) files.
The master SQL tables can be updated as and when required (for example, following a specific import of data or after amending or validating a set of records) or run on a regular basis by scheduling a task in SQL Server. This provides great control over when data is included in any data queries and extracts.
The format and content of the data in the SQL tables can be tailored to match each LERCs requirements and data holdings ensuring that only data that meets the necessary conditions is included in any queries and that data is presented exactly as expected.

Extending Recorder 6¶

SQL Server¶

Master Table¶

In order for the Data Selector tool to be able to extract species records from Recorder6 a SQL script must be run periodically to create and update a master table in the Recorder6 database. The SQL script will be specific to your requirements based on a whole host of considerations. For example:

Which columns/attributes to be defined in the table.
Which surveys, events, samples and occurrences to include in the table.
Whether any confidential surveys, species or occurrences should be included/excluded or flagged as confidential in the table.
Whether any sensitive species should have their precision altered in the table.
What species designations are to be included and how they should appear in the table (e.g. into which columns and using what abbreviations).
How any occurrence abundance/counts and associated measurement qualifier/types should appear in the table.
If zero abundance records are to be included in the table (in which case they would be clearly flagged as zero abundance).
If a cut-off date for specific taxonomic groups or species is used to flag them as historic records.
If the standard Recorder6 taxon group names or more bespoke group names are used (e.g. splitting ‘terrestrial mammal’ into ‘Mammals - Terrestrial (bats)’ and ‘Mammals - Terrestrial (excl. bats)’.
Whether unverified and/or unchecked records are included in the table.
How record dates are to appear (e.g. as vague date range or as just single dates).
If any specific records are to be flagged (e.g. bat records containing ‘roost’ or ‘hibern’ in any record/sample comments or measurements, or bird records containing ‘breed’, ‘bred’ or ‘nest’).

Note

Multiple master SQL tables, or multiple views on the same SQL table, can be created as required.

Spatial Data¶

If your Recorder6 database is running on a more recent version of SQL Server (i.e. SQL Server 2008 or later) then it supports ‘Geometry’ and ‘Geography’ spatial data type. In this case the master table can also be ‘spatialized’ by setting the geometry for all records as points and/or polygons. This enables spatial queries to be performed within SQL Server rather than the GIS application thereby reducing the work load in GIS and utilising the likely increased performance capabilities of many servers running SQL Server.

In order to ‘spatialize’ the master table additional steps in the SQL script are run to calculate the geometry of all records based on their grid reference. The geometry can be calculated as points and/or polygons based on the requirements of the LERC and how the data will be used. Once spatialized, records from the master table can be directly plotted and viewed as points or polygons in GIS. In addition, queries can be executed in SQL Server using the spatial location of the records in much the same way that spatial queries can be performed in GIS. This reduces the overheads in GIS and means that the number of records exported from SQL Server into GIS can be much reduced.

Note

If your Recorder6 database is running on an older version of SQL Server (i.e. SQL Server 2005 or earlier) then it does not support ‘Geometry’ and ‘Geography’ spatial data type. But don’t worry, all is not lost! Although the master table cannot be ‘spatialized’, the records can optionally be plotted as points and/or polygons by the Data Selector tool in GIS if the selected output type is a .tab or .shp file (depending on the host GIS application).

The Data Selector tool¶

Tool components¶

There are three component parts to the Data Selector tool that work together to automate the process described above:

Spatial data held in a SQL Server database (a stored procedure for its extraction is also required).
An XML configuration file that specifies how the tool is set up and how data will be saved by default.
The Data Selector tool itself.

The Data Selector tool is used within a GIS environment but it does not require any particular data layers to be preloaded in the GIS.

The Data Selector tool within the MapInfo environment

Tool workflow¶

The Data Selector tool requires minimum user input in order to perform queries once it is configured. The simple workflow is as follows (see Fig. 3.2):

The user selects which SQL master table to query.
The user lists all the attributes (columns) from the selected SQL table to return (or enters ‘*’ to return all attributes).
The user specifies any ‘Where’ selection criteria, if any, to apply when selecting records from the SQL table.
If required, the user can also specify any ‘Group By’ and ‘Order By’ criteria to apply when selecting records from the SQL table.
The user selects what output format should be created for the selected records.
Optionally, the user can choose one of the pre-defined symbology sets (only applicable for MapInfo .tab output files).
The user can also choose to spatially plot the data (or re-plot the data if it is already spatialized) as points and/or polygons.
There is also an option to clear the log file before use.
Once the user clicks OK the process starts.

The Data Selector tool workflow

In essence, the process that the tool follows is identical to the manual process a user would perform:

The required columns and records from the SQL table are selected based on the specified criteria.
The selected records are saved to the target file in the required output format.
If chosen, and the output file is a GIS layer, the records are spatially plotted as points and/or polygons.
If the output file is a MapInfo .tab file any chosen pre-defined symbology will be applied to the output records.
During the process the tool records its progress to a log file and, when the process finishes, this log file is displayed allowing the user to assess the success of the data selection.

Tool outputs¶

When the process finishes, the output is added to the GIS interface, either as a new GIS layer or as a non-spatial text table.

Output files¶

When the tool is used in MapInfo, output GIS layers can be generated in MapInfo (.tab) format or converted into an ArcGIS (.shp) shapefiles. When used in ArcGIS the tool will output GIS layers as ArcGIS (.shp) shapefiles or as file- or personal-geodatabase feature classes. An example of the output the tool can generate is showin in Fig. 3.3.

Example of a GIS spatial output from the Data Selector tool

Text file outputs can be generated in CSV format (Fig. 3.4) or TXT format.

Example of a text file output from the Data Selector tool

Finally, the log file details each step that was taken during the process, and gives some feedback about the outcome of the process. This includes reporting on the chosen options for the selection, the number of records that were selected, if the output contains spatial data, and if any symbology was applied to the output table (Fig. 3.5).

Example of a Data Selector tool log file

The following chapters, setting up the tool and running the tool, will guide you through setting up and operating the tool in such a way that these tool outputs meet the general requirements of data selection within your organisation.

Setting up the tool¶

Before the Data Selector tool will function, it needs to be installed and configured. It is recommended that the configuration is carried out first, although the steps are interchangeable for the ArcGIS implementation.

Configuring the tool¶

The configuration is stored in an XML file called ‘DataSelector.xml’, and there are some differences in the contents of this file between the MapInfo and the ArcGIS implementations of the tool. Please ensure that you are using the correct XML file, examples of both of which can be found in the Appendix. Attributes and settings are presented as nodes (beginning with a start node, e.g. <example>, and finishing with an end note, e.g. <\example>), with the value for the setting held between the <value> and <\value> tag.

Caution

The name of the configuration file must be ‘DataSelector.xml’. The tool will not load if a different name is used.

The XML file can be edited in a text editor such as Notepad or Wordpad, or using a more feature rich XML editor such as as Sublime Text. The configuration file contains general attributes of the tool. The structure is roughly the same for both implementations of the tool.

Caution

It is important that the structure of the file is maintained as it is presented in the Appendix. Any changes to the structure may result in the Data Selector tool not loading, or not working as expected.

Once editing has been completed and the edits have been saved, it is recommended that the configuration file is opened using an internet browser such as Internet Explorer which will help highlight any editing errors – only if the structure of the file is valid will the whole file be displayed in the internet browser.

Note

It is recommended that the configuration file is kept in a central (network) location, so that all users use the same configuration. Additionally, in case of the MapInfo implementation of the tool, it is essential that the configuration file is kept in the same folder as the compiled version of the tool.

Special characters in XML¶

The characters &, < and > are not valid within values and, so in order to be used, must be escaped with XML entities as follows:

<: This must be escaped with < entity, since it is assumed to be the beginning of a tag. For example, RecYear < 2010

>: This should be escaped with > entity. It is not mandatory – it depends on the context – but it is strongly advised to escape it. For example, RecYear > 1980

&: This must be escaped with & entity, since it is assumed to be the beginning of a entity reference. For example, TaxonGroup = 'Invertebrates - Dragonflies & Damselflies'

Setup for ArcGIS¶

General attributes for ArcGIS¶

The configuration file deals with a series of general attributes for the Data Selector tool. These general nodes specify where files are kept, which SQL Server to connect to, which SQL Server tables to make available and other overall settings. Details on these attributes (and their typical values where known) are given below. The list follows the order within which the attributes are found in the configuration file. This version of the configuration details is valid for the ArcGIS version 1.0.2 of the Data Selector tool.

LogFilePath: The folder to be used for storing log files. This folder must already exist.
FileDSN: The location of the File DSN that specifies to ArcGIS which SQL Server database to connect to.
ConnectionString: The connection string to establish an ADO connection to the source SQL Server database.
DefaultExtractPath: The default folder where output files will be stored. This can be overridden by the user when executing the tool.
DefaultQueryPath: The default folder where queries will be saved and loaded. This can be overridden by the user when executing the tool.
DefaultFormat: The default format of the output files to be created. Options available are ‘Geodatabase’, ‘Shapefile’, ‘CSV file’, ‘dBase file’ and ‘Text file’.
DatabaseSchema: The schema in the SQL Server database containing the source SQL tables. This is typically ‘dbo’.
IncludeWildcard: The Include wildcard for table names to list all the tables in SQL Server that can be selected by the user.
ExcludeWildcard: The Exclude wildcard for table names in SQL Server that should NOT be available for selection by the user. This enables temporary and user-specific tables to be hidden in the tool interface.
RecMax: Currently not used but must exist in XML.
DefaultSetSymbology: Currently not used but must exist in XML.
LayerLocation: Currently not used but must exist in XML.
EnableSpatialPlotting: Currently not used but must exist in XML.

Caution

All entries in the configuration file are case sensitive. Most common errors in the setting up of the tool are caused by using the incorrect case for entries.

Setup for MapInfo¶

General attributes for MapInfo¶

The configuration file deals with a series of general attributes for the Data Selector tool. These general nodes specify where files are kept, which SQL Server to connect to, which SQL Server tables to make available and other overall settings. Details on these attributes (and their typical values where known) are given below. The list follows the order within which the attributes are found in the configuration file. This version of the configuration details is valid for the MapInfo version 1.0.14 of the Data Selector tool.

ToolTitle

The title to use for the program in the MapInfo Tools menu.

LogFilePath

The folder to be used for storing log files. This folder must already exist.

FileDSN

The location of the File DSN that specifies which SQL Server database to connect to.

DefaultExtractPath

The default folder where output files will be stored. This can be overridden by the user when executing the tool.

DefaultQueryPath

The default folder where queries will be saved and loaded. This can be overridden by the user when executing the tool.

DefaultFormat

The default format of the output files to be created. Options available are ‘tab’, ‘shp’, ‘csv’ and ‘txt’.

DatabaseSchema

The schema in the SQL Server database containing the source SQL tables. This is typically ‘dbo’.

TableListSQL

The SQL statement used to list all the species tables in SQL Server that can be selected by the user.

RecMax

The maximum number of records what will be extracted in any one extract file.

DefaultSymbologySet

The default symbology set number that should be used for .tab files. The number corresponds to the order of the symbology sets defined in the MapInfoTables nodes. A value of ‘0’ (zero) sets the default to ‘<None>’ so that no symbology will be set by default.

DefaultSpatialPlotting

The default for whether the SQL database tables can be spatially plotted. If ‘Yes’ the interface will display options for specifying how the data will plotted. If ‘No’ the options for specifying how the data will plotted are hidden in the user interface.

Note

Even if the selected SQL Server table is spatially enabled it is possible to re-plot the data (for example, using a different grid size or as points instead of polygons, or vice-versa)

CoordinateSystem

The spatial coordinate system to use for mapping when plotting data.

DefaultSpatialColumns

This section defines the default SQL Server table columns to use for creating spatial data. It has the following entries:

XColumn: The default column containing the X co-ordinates (eastings). This can be overridden by the user when executing the tool.
YColumn: The default olumn containing the Y co-ordinates (northings). This can be overridden by the user when executing the tool.
SizeColumn: The default column containing the grid size (precision) to be plotted. This can be overridden by the user when executing the tool.

DefaultPointsPrecision

The default maximum precision for plotting polygons as points. Any records where the ‘SizeColumn’ is less than or equal to this value will be plotted as points. Any records where the ‘SizeColumn’ is greater than this value will be plotted as polygons. This can be overridden by the user when executing the tool.

DialogSize

Indicates the user interface dialog size for the tool. Options are ‘Norm’ or ‘Max’. It is generally recommended that ‘Max’ is used unless the dialog does not fit in the desktop.

Symbology attributes for MapInfo¶

MapInfoTables: This section defines the symbology sets available for applying to any MapInfo .tab outputs. It has the following entries:

All symbology sets are found within the <MapInfoTables> node. For each symbology set, which can be selected and applied to an output MapInfo .tab file, a new child node must be created. The node name (e.g. ‘SymbologySet1’) is not important but must be unique. A simplified example of a symbology set is shown in Fig. 4.1.

Example of symbology set attributes configuration (MapInfo)

Desc: An attribute describing the symbology set. This will appear in the interface drop-down list.
Symbology: Each set contains only one child node <Symbology>> under which multiple symbols can be specified. set Each symbol is specified between <Symbol> and </Symbol> tags and is defined by the following child nodes:
Clause: The clause that defines the records which will be assigned this symbol. This can be left blank to apply the symbology to all records with the same <Object> type specified below.
Object: The object type that is symbolised using this symbol (e.g. Region). Options are ‘Point’, ‘Line’ or ‘Region’.
Symbol: The style to be used for the symbol. This attribute only applies to Point objects.
Pen: The style to be used for the symbol border (outline). This attribute applies to Region objects.
Brush: The style to be used for the symbol infill. This attribute applies to Region objects.

Tip

In order to find the syntax for the Pen and Brush attribute, set the desired symbol for a polygon (region) layer through Options => Region style, then write Print CurrentBorderPen() in the MapBasic window and hit enter. The printed pen definition (e.g. 2,2,10526880) can be used in the Pen attribute. Repeat with Print CurrentBrush().

Caution

All entries in the configuration file are case sensitive. Most common errors in the setting up of the tool are caused by using the incorrect case for entries.

Setting up the SQL Server database¶

In addition to any SQL tables containing records to be extracted using the Data Selector tool, an auxiliary table must also be present in the SQL Server database in order for the tool to be able to select data from tables held in SQL Server. This table is as follows:

Spatial_Tables table

This table contains information about any SQL data tables that may be used by the tool. The table has the following columns:

Valid date and time format specifiers¶
Column	Description
TableName	The name of the data table
OwnerName	The database owner, usually `dbo`
XColumn	The name of the column holding the X coordinates of the record
YColumn	The name of the column holding the Y coordinates of the record
SizeColumn	The name of the column holding the grid size of the record (in metres)
IsSpatial	Bitwise column (1 = Yes, 0 = No) defining whether the table is spatially enabled
SpatialColumn	If the table is spatially enabled, the name of the geometry column (e.g. `SP_GEOMETRY`)
SRID	The name of the spatial reference system used to plot the records
CoordSystem	The coordinate system of the spatial data in the table
SurveyKeyColumn	The column containing the survey key for each record

Note

The British National Grid SRID value is Earth Projection 8, 79, "m", -2, 49, 0.9996012717, 400000, -100000 Bounds (-7845061.1011, -15524202.1641) (8645061.1011, 4470074.53373)

Caution

This table must be filled out correctly for each SQL table that is available to the Data Selector tool.

Note

A number of stored procedures that are used by the tool for selecting the required records must also be present in the SQL Server database. To obtain copies of these procedures please contact Hester or Andy.

Installing the tool¶

Installing in ArcGIS¶

Installing the tool in ArcGIS is straightforward. There are a few different ways it can be installed:

Installation through Windows Explorer¶

Open Windows Explorer and double-click on the ESRI Add-in file for the Data Selector tool (Fig. 4.2).

Installing the Data Selector tool from Windows Explorer

Installation will begin after confirming you wish to install the tool on the dialog that appears (Fig. 4.3).

Installation begins after clicking ‘Install Add-in’

Once it is installed, it will become available to add to the ArcGIS interface as a button (see CustomisingToolbarsArcGIS).

Note

In order for this process to work all running ArcMap sessions must be closed. The tool will not install or install incorrectly if there are copies of ArcMap running.

Installation from within ArcMap¶

Firstly, open the Add-In Manager through the Customize menu (Fig. 4.4).

Starting the ArcGIS Add-In Manager

If the Data Selector tool is not shown, use the Options tab to add the folder where the tool is kept (Fig. 4.5). The security options should be set to the lowest setting as the tool is not digitally signed.

The ‘Options’ tab in the ArcGIS Add-In Manager

Once the tool shows in the Add-In Manager (Fig. 4.6), it is available to add to the ArcGIS interface as a button (see CustomisingToolbarsArcGIS).

The ArcGIS Add-In Manager showing the Data Selector tool

Customising toolbars¶

In order to add the Data Selector tool to the user interface, it needs to be added to a toolbar. It is recommended that this customisation is done inside a document, but it can be done so that the toolbar always appears in ArcGIS (see Fundamentals of Saving your Customizations for an explanation of how customisations are stored within ArcGIS).

Starting Customize Mode in ArcGIS

Customising toolbars is done through the Customize dialog, which can be started either through the Add-In Manager (by clicking Customize, see Fig. 4.6), or through choosing the ‘Customize Mode…’ option in the Customize Menu (Fig. 4.7).

Once this dialog is open, select or clear the check box ‘Create new toolbars and menus in the document’ as required in the Options tab (Fig. 4.8).

Customising the document in ArcGIS

It is recommended that the button for the Data Selector tool is added to a new toolbar. Toolbars are created through the Toolbars tab in the Customize dialog, as shown in figures Fig. 4.9 and Fig. 4.10.

Adding a new toolbar in ArcGIS

Naming the new toolbar in ArcGIS

Once a new toolbar is created and named, it is automatically added to the ArcMap interface as well as to the Customize dialog (Fig. 4.11. In this case the toolbar was named ‘TestToolbar’).

New toolbar added to the ArcGIS Interface

As a final step the Data Selector tool is added to the toolbar. This is done from the Command tab in the Customize dialog (Fig. 4.12). Click on Add-In Controls and the Data Selector tool will be shown in the right-hand panel.

Finding the Data Selector tool in the add-in commands

To add the tool to the toolbar, simply drag and drop it onto it (Fig. 4.13). Close the Customize dialog and save the document. The Data Selector tool is now ready for its final configuration and first use.

Adding the Data Selector tool to the new toolbar

In order to function, the tool needs to know the location of the XML configuration file. The first time the tool is run, or whenever the configuration file is moved, a dialog will appear asking for the folder containing the XML file (Fig. 4.14). Navigate to the folder where the XML file is kept and click OK. If the XML file is present and its structure is correct, the Data Selector form will be shown. Even if the tool is not run at this time, the location of the configuration file will be stored for future use.

Locating the configuration file folder

Installing in MapInfo¶

To install the tool in MapInfo, make sure that the configuration of the XML file as described above is complete, that the XML file is in the same directory as the tool MapBasic application (.MBX). Then, open Tool Manager in MapInfo by selecting Tools --> Tool Manager... in the menu bar (Fig. 4.15).

The Tool Manager in MapInfo 12 or earlier

In the Tool Manager dialog, click Add Tool…, then locate the tool using the browse button ... on the Add Tool dialog (Fig. 4.16). Enter a name in the Title box (e.g. ‘DataSelector’), and a description if desired. Then click Ok to close the Add Tool dialog.

Adding a tool in Tool Manager

The tool will now show in the Tool Manager dialog (Fig. 4.17) and the Loaded box will be checked. To load the tool automatically whenever MapInfo is started check the AutoLoad box. Then click Ok to close the Tool Manager dialog.

The Data Selector tool is loaded

The tool will now appear as a new entry in the Tools menu (Fig. 4.18).

The Data Selector tool menu

Note

The name that will appear in the Tools menu is dependent on the ToolTitle value in the configuration file, not the name given when adding the tool using the Tool Manager.

Running the tool¶

Instructions¶

The operation of the Data Selector tool is explained in this section. While the interface is similar between the ArcGIS and MapInfo implementations of the tool, there are some differences. These are pointed out where relevant throughout this document.

As discussed in the Setting up the tool section, the Data Selector tool is operated from the GIS application. It relies on the SQL tables containing any data that may be selected and a configuration document for setting up the tool. Therefore, before running the tool, ensure the following conditions are met:

The XML configuration document has been set up correctly and is named correctly.
The Data Selector tool has been installed and set up.

Opening the form¶

To open the Data Selector tool in ArcGIS, click on the Data Selector tool button (Fig. 5.1).

Launching the Data Selector tool (ArcGIS)

To open the Data Selector tool in MapInfo, select Tools… -> Data Selector in the Tools menu (Fig. 5.2).

Launching the Data Selector tool (MapInfo)

If there are any structural issues with the XML document, the tool will display a message with the error it has encountered, and not load any further.

Example warning message displayed for any XML structural issues

Provided that the XML document is otherwise correct, the form will display (Fig. 5.4).

The form displaying the available SQL tables and options (MapInfo)

Using the form¶

Select the required source table in the SQL Tables box (Fig. 5.5).

Select the required source table in the SQL tables list

Select the output file format to contain the selected records (Fig. 5.6).

Select the output file format

The following options are available in MapInfo:

MapInfo (.tab) file
ESRI (.shp) file
Comma-separated (.csv) file
Tab-separated (.txt) file

The following options are available in ArcGIS:

Geodatabase
Shapefile
Text file (tab delimited)
CSV file

If you have selected ‘MapInfo (.tab) file’ as the output file format (only available in MapInfo) you can optionally also select one of the pre-defined symbology sets to apply to the output file (Fig. 5.7). Symbology sets are specified in the configuration file

Select a symbology set to apply to the records (MapInfo only)

If you have selected ‘ESRI (.shp) file’ (available in ArcGIS or MapInfo) or ‘MapInfo (.tab) file’ (available only in MapInfo) as the output file format you can optionally also select if the records should be spatially plotted or re-plotted (Fig. 5.8). In ArcGIS, the records are automatically plotted if a geometry field is included.

_images/SelectSpatialPlottingMapInfo.png

Select a symbology set to apply to the records (MapInfo only)

Finally, select whether the log file should be overwritten, and whether the spatial columns should be deleted after use (MapInfo only). Now the form is set up to enter and run your query.

Entering queries¶

Specify the columns and SQL clauses that you wish to use to query the selected table. All syntax should adhere to SQL Server SQL syntax, and it is important that the correct part of the query is entered in the correct text box. Bear in mind when writing your query that the tool is designed to run a SELECT query only, and that it will not execute an action query. Simply enter the columns you wish to select in the Columns box, including any changes in name (e.g. CommonName as Name), your selection clause in the Where box, and your Group By and Order By clauses in their relevant boxes as appropriate. A number of examples are shown in Fig. 5.9 (a simple query including geometry), Fig. 5.10 (a spatial query using grouping) and Fig. 5.11 (a non-spatial, or tabular, query).

Note

If a Group By clause is included for a query that includes a spatial element, the geometry must be grouped too using SQL specific syntax (see Fig. 5.10 for an example using an aggregate function).

Note

It is not necessary to include key words such as SELECT, WHERE, GROUP BY and ORDER BY. Doing so will make the tool fail.

Example of a simple query using geometry (ArcGIS)

Example of a query using spatial grouping (ArcGIS)

If no spatial element is selected in the query, the output will be tabular. An example of such a query is shown in Fig. 5.11, where a simple report is generated of the number of records for each taxon group.

Example of a summary query with tabular output (ArcGIS)

Running the query¶

Once the correct query has been entered and the output format selected, and the plotting and symbology defined (MapInfo only), check whether the log file should be overwritten using the Clear Log File check box. Then, click OK. You will be prompted for an output file (Fig. 5.12). Enter a name for your output and click Save. If an existing file is selected, the tool will prompt you for permission to overwrite this, or to choose a different name. Once the output name has been selected the tool will begin the process.

Enter the name for your output file.

Depending on which version of the tool you are using, and whether you are generating a spatial or a tabular output, the output will be created and shown as follows:

If you are using MapInfo and are creating a spatial output layer, it will be added to the view and shown.
If you are using ArcGIS and are creating a spatial output layer, it will be split into point and polygon layers as required and added to the view in a grouped layer.
If you are using MapInfo and are creating tabular output in CSV or TXT format, it will not be added to the view.
If you are using ArcGIS and are creating tabular output, or MapInfo and creating tabular output in .tab format, it will be added to the view and shown.

An example of spatial and tabular outputs in ArcGIS is shown in Fig. 5.13 and Fig. 3.4.

Example output from a spatial query (ArcGIS; Zoomed in to maintain confidentiality)

Example output from a non-spatial query (ArcGIS)

Once the process has finished, a messagebox will ask you whether you wish to close the form (Fig. 5.15). After you have made your choice the log file is shown (Fig. 5.16) so you can check the process has completed correctly, and the results will be added to the screen as discussed above. Note that the log file entries are different between ArcGIS and MapInfo versions.

User is asked whether the form should be closed after the process finishes

The log file is shown when the process finishes (ArcGIS)

Loading & saving queries¶

Queries can be saved for future use, so once a query has been written and saved it can be loaded and re-run by any user at any time and it will output results from the most up-to-date copy of the database.

Queries can be saved or loaded by using the Save and Load buttons (Fig. 5.17). They are saved as a simple text file with minimal formatting.

Save and load queries (MapInfo)

Frequently Asked Questions¶

This is a list of Frequently Asked Questions about the Data Selector tool. Feel free to suggest new entries!

General questions¶

How do I get a copy of the tool?

The source code as well as the installer setups can be downloaded from GitHub (MapInfo or ArcGIS). Please ensure that you use the correct configuration file, examples copy of which are also included with the releases.

Can several people use the tool at the same time?

Any number of users can use the tool if they have a copy of the tool installed or loaded in their own copy of ArcGIS or MapInfo. Where outputs are written to a central (network) location, it is important to ensure that no two users are saving to the same files at the same time as this can lead to unexpected results.

Does the tool work with QGIS?

Currently only ArcGIS and MapInfo implementations of the tool exist. However, if funding was available the tool could be adapted to also support QGIS.

Operating the tool¶

One of the SQL Server tables I want to use isn’t showing in the form. How do I get it to show up?

This issue will arise if the layer isn’t being selected by the SQL statement in the XML configuration document. Please refer to the setup section and adjust the TableListSQL statement, and the Include and Exclude wildcards as appropriate.

I got an error message about a SQL error

If there is an error in any of the selection criteria (i.e. the ‘Columns’, ‘Where’, ‘Group By’ or ‘Order By’ fields) then a SQL error message may appear. The contents of the message will vary between MapInfo and ArcGIS but both will contain some information about the nature of the SQL error. The details of the error are generated by SQL Server and will generally be specific to the nature of the error (e.g. ‘Incorrect syntax near …’ or ‘Invalid column name …’), but they can sometimes be more cryptic.

How do I learn how to use SQL syntax?

There are some excellent free online resources available to learn how to write SQL queries. Some examples are The W3 Schools tutorial and the Tutorialspoint SQL tutorial. When writing your queries bear in mind that the tool is set up to run a SELECT query only.

Tool issues¶

How do I report a new bug or propose a change in the tool?

Please check the existing known issues and change requests on the LERCAutomation pages on GitHub (ArcGIS, MapInfo) before reporting/proposing new issues or changes. If you have a new issue or request you can submit it there and it will be picked up by the developers. Alternatively, you can email suggestions to Hester or Andy.

Appendix¶

Example XML file for ArcGIS¶

Below is an example of XML that might be used to set up the Data Selector tool in ArcGIS. Note, many of the settings have been included for illustration only and it is up to each user or LERC to ensure the system is configured to their requirements.

<?xml version="1.0" encoding="utf-8"?>

<!--
WARNING: This file should be changed carefully and a backup should be
taken before any changes so that they can be backed out.  Changed lines
can also be commented out as below.
-->

<!--
This config file contains all the variables used by the DataSelector
ArcObjects add-in.

The 'configuration' node is the 'root' node and signifies the start of the
contents of the configuration file.

The 'DataSelector' node contains all of the entries relating to the
ArcObjects add-in variables.

Each entry relates to a file, folder, table name, column name or SQL
statement used by the ArcObjects add-in to select and export species data
for use in GIS or as a spreadsheet.
-->

<configuration>
<DataSelector>

  <!-- The existing file location where log files will be saved with output
       messages -->
  <LogFilePath>
    <value>H:\DataSelector\LogFiles</value>
  </LogFilePath>

      <!-- The path to the DSN configuration file that contains the connection
       details for the SQL Server database -->
  <FileDSN>
    <value>H:\DataSelector\Config\TVERCConn.sde</value>
  </FileDSN>

  <!-- Connection string for ADO connection (to allow stored procedures to
       be run with parameters) -->
  <ConnectionString>
    <value>Server=SONY\SQLEXPRESS; Database=NBNData;
           Integrated Security=True;</value>
  </ConnectionString>

  <!-- The existing file location where extracts will be saved by default -->
  <DefaultExtractPath>
    <value>H:\DataSelector\Extracts</value>
  </DefaultExtractPath>

  <!-- The existing file location where queries will be saved and loaded
       by default -->
  <DefaultQueryPath>
    <value>H:\DataSelector\Queries</value>
  </DefaultQueryPath>

  <!-- The default format of the output files to be created -->
  <DefaultFormat>
    <value>Geodatabase</value>
  </DefaultFormat>

  <!-- The schema used in the SQL Server database -->
  <DatabaseSchema>
    <value>dbo</value>
  </DatabaseSchema>

  <!-- the Include wildcard for table names to list all the species tables
       in SQL Server that can be selected by the user to extract from -->
  <IncludeWildcard>
    <value>*</value><!--*TVERC_Spp_* -->
  </IncludeWildcard>

  <!-- the Exclude wildcard for table names that should NOT be used for
       species tables in SQL Server that can be selected by the user to
       extract from -->
  <ExcludeWildcard>
    <value>*TVERC_Spp_*_*</value>
  </ExcludeWildcard>

</DataSelector>
</configuration>

Example XML file for MapInfo¶

Below is an example of XML that might be used to set up the Data Selector tool in ArcGIS. Note, many of the settings have been included for illustration only and it is up to each user or LERC to ensure the system is configured to their requirements.

<?xml version="1.0" encoding="utf-8"?>

<!--
WARNING: This file should be changed carefully and a backup should be
taken before any changes so that they can be backed out.  Changed lines
can also be commented out as below.
-->

<!--
This config file contains all the variables used by the DataSelector
MapBasic tool.

The 'configuration' node is the 'root' node and signifies the start of the
contents of the configuration file.

The 'DataSelector' node contains all of the entries relating to the
MapBasic tool variables.

Each entry relates to a file, folder, table name, column name or SQL
statement used by the MapBasic tool to select and export species data for
use in GIS or as a spreadsheet.
-->

<configuration>
<DataSelector>

  <!-- The title to use for the program in the Tool menu -->
  <ToolTitle>
    <value>Data Selector</value>
  </ToolTitle>

  <!-- The existing file location where log files will be saved with output
       messages -->
  <LogFilePath>
    <value>D:\Andy\GiGL\Data Selector\Logs</value>
  </LogFilePath>

  <!-- The location of the File DSN that specifies which SQL Server database
       to connect to -->
  <FileDSN>
    <value>D:\Andy\GiGL\Data Selector\SQL Server\NBNData.dsn</value>
  </FileDSN>

  <!-- The existing file location where extracts will be saved by default -->
  <DefaultExtractPath>
    <value>D:\Andy\GiGL\Data Selector\Extracts</value>
  </DefaultExtractPath>

  <!-- The existing file location where queries will be saved and loaded by
       default -->
  <DefaultQueryPath>
    <value>D:\Andy\GiGL\Data Selector\Queries</value>
  </DefaultQueryPath>

  <!-- The default format of the output files to be created -->
  <DefaultFormat>
    <value>tab</value>
  </DefaultFormat>

  <!-- The schema used in the SQL Server database -->
  <DatabaseSchema>
    <value>dbo</value>
  </DatabaseSchema>

  <!-- The SQL statement used to list all the species tables in SQL Server
       that can be selected by the user to extract from -->
  <TableListSQL>
    <value>Select table_name From information_schema.tables Where table_name
           Like 'GiGL[_]Spp[_]%' And table_name Not Like 'GiGL[_]Spp[_]%[_]%'
           Order By table_name</value>
  </TableListSQL>

  <!-- The maximum number of records what will be extracted in any one
       extract -->
  <RecMax>
    <value>10000000</value>
  </RecMax>

  <!-- The default for whether the symbology should be set for .tab files
       or not -->
  <DefaultSymbologySet>
    <value>2</value>
  </DefaultSymbologySet>

  <!-- The default for whether the SQL database tables will be spatially
       plotted -->
  <DefaultSpatialPlotting>
    <value>No</value>
  </DefaultSpatialPlotting>

  <!-- The spatial coordinate system to use for mapping -->
  <CoordinateSystem>
    <value>Earth Projection 8, 79, "m", -2, 49, 0.9996012717, 400000,
           -100000</value>
  </CoordinateSystem>

  <!-- The table columns used for creating spatial data -->
  <DefaultSpatialColumns>
    <XColumn>
        <value>Eastings</value>
    </XColumn>
    <YColumn>
        <value>Northings</value>
    </YColumn>
    <SizeColumn>
        <value>GRPrecision</value>
    </SizeColumn>
  </DefaultSpatialColumns>

  <!-- The default maximum precision for plotting polygons as points -->
  <DefaultPointsPrecision>
    <value>100</value>
  </DefaultPointsPrecision>

  <!-- The User interface dialog size ("Norm" or "Max") -->
  <DialogSize>
    <value>Max</value>
  </DialogSize>

  <!-- The symbology sets available for applying to any MapInfo .tab
       outputs -->
  <MapInfoTables>
    <SymbolSet1 Desc="Standard Points/Polygons">
        <Symbology>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION &lt;= 10</Value>
                </Clause>
                <Object>
                    <Value>Point</Value>
                </Object>
                <Symbol>
                    <Value>2,65,255,12,MapInfo Dispersed Group,0,0</Value>
                </Symbol>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION = 100</Value>
                </Clause>
                <Object>
                    <Value>Point</Value>
                </Object>
                <Symbol>
                    <Value>2,64,255,14,MapInfo Dispersed Group,0,0</Value>
                </Symbol>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION &lt;= 10</Value>
                </Clause>
                <Object>
                    <Value>Region</Value>
                </Object>
                <Pen>
                    <Value>2,2,10526880</Value>
                </Pen>
                <Brush>
                    <Value>5,10526880</Value>
                </Brush>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION = 100</Value>
                </Clause>
                <Object>
                    <Value>Region</Value>
                </Object>
                <Pen>
                    <Value>2,2,10526880</Value>
                </Pen>
                <Brush>
                    <Value>5,10526880</Value>
                </Brush>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION = 1000</Value>
                </Clause>
                <Object>
                    <Value>Region</Value>
                </Object>
                <Pen>
                    <Value>2,2,10526880</Value>
                </Pen>
                <Brush>
                    <Value>5,10526880</Value>
                </Brush>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION = 2000</Value>
                </Clause>
                <Object>
                    <Value>Region</Value>
                </Object>
                <Pen>
                    <Value>2,2,10526880</Value>
                </Pen>
                <Brush>
                    <Value>6,10526880</Value>
                </Brush>
            </Symbol>
            <Symbol>
                <Clause>
                    <Value>GRPRECISION = 10000</Value>
                </Clause>
                <Object>
                    <Value>Region</Value>
                </Object>
                <Pen>
                    <Value>2,2,10526880</Value>
                </Pen>
                <Brush>
                    <Value>1,10526880</Value>
                </Brush>
            </Symbol>
        </Symbology>
    </SymbolSet1>
  </MapInfoTables>

</DataSelector>
</configuration>

GNU Free Documentation License¶

                GNU Free Documentation License
                 Version 1.3, 3 November 2008


 Copyright (C) 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.
     <http://fsf.org/>
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

0. PREAMBLE

The purpose of this License is to make a manual, textbook, or other
functional and useful document "free" in the sense of freedom: to
assure everyone the effective freedom to copy and redistribute it,
with or without modifying it, either commercially or noncommercially.
Secondarily, this License preserves for the author and publisher a way
to get credit for their work, while not being considered responsible
for modifications made by others.

This License is a kind of "copyleft", which means that derivative
works of the document must themselves be free in the same sense.  It
complements the GNU General Public License, which is a copyleft
license designed for free software.

We have designed this License in order to use it for manuals for free
software, because free software needs free documentation: a free
program should come with manuals providing the same freedoms that the
software does.  But this License is not limited to software manuals;
it can be used for any textual work, regardless of subject matter or
whether it is published as a printed book.  We recommend this License
principally for works whose purpose is instruction or reference.


1. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that
contains a notice placed by the copyright holder saying it can be
distributed under the terms of this License.  Such a notice grants a
world-wide, royalty-free license, unlimited in duration, to use that
work under the conditions stated herein.  The "Document", below,
refers to any such manual or work.  Any member of the public is a
licensee, and is addressed as "you".  You accept the license if you
copy, modify or distribute the work in a way requiring permission
under copyright law.

A "Modified Version" of the Document means any work containing the
Document or a portion of it, either copied verbatim, or with
modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of
the Document that deals exclusively with the relationship of the
publishers or authors of the Document to the Document's overall
subject (or to related matters) and contains nothing that could fall
directly within that overall subject.  (Thus, if the Document is in
part a textbook of mathematics, a Secondary Section may not explain
any mathematics.)  The relationship could be a matter of historical
connection with the subject or with related matters, or of legal,
commercial, philosophical, ethical or political position regarding
them.

The "Invariant Sections" are certain Secondary Sections whose titles
are designated, as being those of Invariant Sections, in the notice
that says that the Document is released under this License.  If a
section does not fit the above definition of Secondary then it is not
allowed to be designated as Invariant.  The Document may contain zero
Invariant Sections.  If the Document does not identify any Invariant
Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed,
as Front-Cover Texts or Back-Cover Texts, in the notice that says that
the Document is released under this License.  A Front-Cover Text may
be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy,
represented in a format whose specification is available to the
general public, that is suitable for revising the document
straightforwardly with generic text editors or (for images composed of
pixels) generic paint programs or (for drawings) some widely available
drawing editor, and that is suitable for input to text formatters or
for automatic translation to a variety of formats suitable for input
to text formatters.  A copy made in an otherwise Transparent file
format whose markup, or absence of markup, has been arranged to thwart
or discourage subsequent modification by readers is not Transparent.
An image format is not Transparent if used for any substantial amount
of text.  A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain
ASCII without markup, Texinfo input format, LaTeX input format, SGML
or XML using a publicly available DTD, and standard-conforming simple
HTML, PostScript or PDF designed for human modification.  Examples of
transparent image formats include PNG, XCF and JPG.  Opaque formats
include proprietary formats that can be read and edited only by
proprietary word processors, SGML or XML for which the DTD and/or
processing tools are not generally available, and the
machine-generated HTML, PostScript or PDF produced by some word
processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself,
plus such following pages as are needed to hold, legibly, the material
this License requires to appear in the title page.  For works in
formats which do not have any title page as such, "Title Page" means
the text near the most prominent appearance of the work's title,
preceding the beginning of the body of the text.

The "publisher" means any person or entity that distributes copies of
the Document to the public.

A section "Entitled XYZ" means a named subunit of the Document whose
title either is precisely XYZ or contains XYZ in parentheses following
text that translates XYZ in another language.  (Here XYZ stands for a
specific section name mentioned below, such as "Acknowledgements",
"Dedications", "Endorsements", or "History".)  To "Preserve the Title"
of such a section when you modify the Document means that it remains a
section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which
states that this License applies to the Document.  These Warranty
Disclaimers are considered to be included by reference in this
License, but only as regards disclaiming warranties: any other
implication that these Warranty Disclaimers may have is void and has
no effect on the meaning of this License.

2. VERBATIM COPYING

You may copy and distribute the Document in any medium, either
commercially or noncommercially, provided that this License, the
copyright notices, and the license notice saying this License applies
to the Document are reproduced in all copies, and that you add no
other conditions whatsoever to those of this License.  You may not use
technical measures to obstruct or control the reading or further
copying of the copies you make or distribute.  However, you may accept
compensation in exchange for copies.  If you distribute a large enough
number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and
you may publicly display copies.


3. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have
printed covers) of the Document, numbering more than 100, and the
Document's license notice requires Cover Texts, you must enclose the
copies in covers that carry, clearly and legibly, all these Cover
Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on
the back cover.  Both covers must also clearly and legibly identify
you as the publisher of these copies.  The front cover must present
the full title with all words of the title equally prominent and
visible.  You may add other material on the covers in addition.
Copying with changes limited to the covers, as long as they preserve
the title of the Document and satisfy these conditions, can be treated
as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit
legibly, you should put the first ones listed (as many as fit
reasonably) on the actual cover, and continue the rest onto adjacent
pages.

If you publish or distribute Opaque copies of the Document numbering
more than 100, you must either include a machine-readable Transparent
copy along with each Opaque copy, or state in or with each Opaque copy
a computer-network location from which the general network-using
public has access to download using public-standard network protocols
a complete Transparent copy of the Document, free of added material.
If you use the latter option, you must take reasonably prudent steps,
when you begin distribution of Opaque copies in quantity, to ensure
that this Transparent copy will remain thus accessible at the stated
location until at least one year after the last time you distribute an
Opaque copy (directly or through your agents or retailers) of that
edition to the public.

It is requested, but not required, that you contact the authors of the
Document well before redistributing any large number of copies, to
give them a chance to provide you with an updated version of the
Document.


4. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under
the conditions of sections 2 and 3 above, provided that you release
the Modified Version under precisely this License, with the Modified
Version filling the role of the Document, thus licensing distribution
and modification of the Modified Version to whoever possesses a copy
of it.  In addition, you must do these things in the Modified Version:

A. Use in the Title Page (and on the covers, if any) a title distinct
   from that of the Document, and from those of previous versions
   (which should, if there were any, be listed in the History section
   of the Document).  You may use the same title as a previous version
   if the original publisher of that version gives permission.
B. List on the Title Page, as authors, one or more persons or entities
   responsible for authorship of the modifications in the Modified
   Version, together with at least five of the principal authors of the
   Document (all of its principal authors, if it has fewer than five),
   unless they release you from this requirement.
C. State on the Title page the name of the publisher of the
   Modified Version, as the publisher.
D. Preserve all the copyright notices of the Document.
E. Add an appropriate copyright notice for your modifications
   adjacent to the other copyright notices.
F. Include, immediately after the copyright notices, a license notice
   giving the public permission to use the Modified Version under the
   terms of this License, in the form shown in the Addendum below.
G. Preserve in that license notice the full lists of Invariant Sections
   and required Cover Texts given in the Document's license notice.
H. Include an unaltered copy of this License.
I. Preserve the section Entitled "History", Preserve its Title, and add
   to it an item stating at least the title, year, new authors, and
   publisher of the Modified Version as given on the Title Page.  If
   there is no section Entitled "History" in the Document, create one
   stating the title, year, authors, and publisher of the Document as
   given on its Title Page, then add an item describing the Modified
   Version as stated in the previous sentence.
J. Preserve the network location, if any, given in the Document for
   public access to a Transparent copy of the Document, and likewise
   the network locations given in the Document for previous versions
   it was based on.  These may be placed in the "History" section.
   You may omit a network location for a work that was published at
   least four years before the Document itself, or if the original
   publisher of the version it refers to gives permission.
K. For any section Entitled "Acknowledgements" or "Dedications",
   Preserve the Title of the section, and preserve in the section all
   the substance and tone of each of the contributor acknowledgements
   and/or dedications given therein.
L. Preserve all the Invariant Sections of the Document,
   unaltered in their text and in their titles.  Section numbers
   or the equivalent are not considered part of the section titles.
M. Delete any section Entitled "Endorsements".  Such a section
   may not be included in the Modified Version.
N. Do not retitle any existing section to be Entitled "Endorsements"
   or to conflict in title with any Invariant Section.
O. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or
appendices that qualify as Secondary Sections and contain no material
copied from the Document, you may at your option designate some or all
of these sections as invariant.  To do this, add their titles to the
list of Invariant Sections in the Modified Version's license notice.
These titles must be distinct from any other section titles.

You may add a section Entitled "Endorsements", provided it contains
nothing but endorsements of your Modified Version by various
parties--for example, statements of peer review or that the text has
been approved by an organization as the authoritative definition of a
standard.

You may add a passage of up to five words as a Front-Cover Text, and a
passage of up to 25 words as a Back-Cover Text, to the end of the list
of Cover Texts in the Modified Version.  Only one passage of
Front-Cover Text and one of Back-Cover Text may be added by (or
through arrangements made by) any one entity.  If the Document already
includes a cover text for the same cover, previously added by you or
by arrangement made by the same entity you are acting on behalf of,
you may not add another; but you may replace the old one, on explicit
permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License
give permission to use their names for publicity for or to assert or
imply endorsement of any Modified Version.


5. COMBINING DOCUMENTS

You may combine the Document with other documents released under this
License, under the terms defined in section 4 above for modified
versions, provided that you include in the combination all of the
Invariant Sections of all of the original documents, unmodified, and
list them all as Invariant Sections of your combined work in its
license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and
multiple identical Invariant Sections may be replaced with a single
copy.  If there are multiple Invariant Sections with the same name but
different contents, make the title of each such section unique by
adding at the end of it, in parentheses, the name of the original
author or publisher of that section if known, or else a unique number.
Make the same adjustment to the section titles in the list of
Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History"
in the various original documents, forming one section Entitled
"History"; likewise combine any sections Entitled "Acknowledgements",
and any sections Entitled "Dedications".  You must delete all sections
Entitled "Endorsements".


6. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other
documents released under this License, and replace the individual
copies of this License in the various documents with a single copy
that is included in the collection, provided that you follow the rules
of this License for verbatim copying of each of the documents in all
other respects.

You may extract a single document from such a collection, and
distribute it individually under this License, provided you insert a
copy of this License into the extracted document, and follow this
License in all other respects regarding verbatim copying of that
document.


7. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate
and independent documents or works, in or on a volume of a storage or
distribution medium, is called an "aggregate" if the copyright
resulting from the compilation is not used to limit the legal rights
of the compilation's users beyond what the individual works permit.
When the Document is included in an aggregate, this License does not
apply to the other works in the aggregate which are not themselves
derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these
copies of the Document, then if the Document is less than one half of
the entire aggregate, the Document's Cover Texts may be placed on
covers that bracket the Document within the aggregate, or the
electronic equivalent of covers if the Document is in electronic form.
Otherwise they must appear on printed covers that bracket the whole
aggregate.


8. TRANSLATION

Translation is considered a kind of modification, so you may
distribute translations of the Document under the terms of section 4.
Replacing Invariant Sections with translations requires special
permission from their copyright holders, but you may include
translations of some or all Invariant Sections in addition to the
original versions of these Invariant Sections.  You may include a
translation of this License, and all the license notices in the
Document, and any Warranty Disclaimers, provided that you also include
the original English version of this License and the original versions
of those notices and disclaimers.  In case of a disagreement between
the translation and the original version of this License or a notice
or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements",
"Dedications", or "History", the requirement (section 4) to Preserve
its Title (section 1) will typically require changing the actual
title.


9. TERMINATION

You may not copy, modify, sublicense, or distribute the Document
except as expressly provided under this License.  Any attempt
otherwise to copy, modify, sublicense, or distribute it is void, and
will automatically terminate your rights under this License.

However, if you cease all violation of this License, then your license
from a particular copyright holder is reinstated (a) provisionally,
unless and until the copyright holder explicitly and finally
terminates your license, and (b) permanently, if the copyright holder
fails to notify you of the violation by some reasonable means prior to
60 days after the cessation.

Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.

Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License.  If your rights have been terminated and not permanently
reinstated, receipt of a copy of some or all of the same material does
not give you any rights to use it.


10. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the
GNU Free Documentation License from time to time.  Such new versions
will be similar in spirit to the present version, but may differ in
detail to address new problems or concerns.  See
http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number.
If the Document specifies that a particular numbered version of this
License "or any later version" applies to it, you have the option of
following the terms and conditions either of that specified version or
of any later version that has been published (not as a draft) by the
Free Software Foundation.  If the Document does not specify a version
number of this License, you may choose any version ever published (not
as a draft) by the Free Software Foundation.  If the Document
specifies that a proxy can decide which future versions of this
License can be used, that proxy's public statement of acceptance of a
version permanently authorizes you to choose that version for the
Document.

11. RELICENSING

"Massive Multiauthor Collaboration Site" (or "MMC Site") means any
World Wide Web server that publishes copyrightable works and also
provides prominent facilities for anybody to edit those works.  A
public wiki that anybody can edit is an example of such a server.  A
"Massive Multiauthor Collaboration" (or "MMC") contained in the site
means any set of copyrightable works thus published on the MMC site.

"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0
license published by Creative Commons Corporation, a not-for-profit
corporation with a principal place of business in San Francisco,
California, as well as future copyleft versions of that license
published by that same organization.

"Incorporate" means to publish or republish a Document, in whole or in
part, as part of another Document.

An MMC is "eligible for relicensing" if it is licensed under this
License, and if all works that were first published under this License
somewhere other than this MMC, and subsequently incorporated in whole or
in part into the MMC, (1) had no cover texts or invariant sections, and
(2) were thus incorporated prior to November 1, 2008.

The operator of an MMC Site may republish an MMC contained in the site
under CC-BY-SA on the same site at any time before August 1, 2009,
provided the MMC is eligible for relicensing.


ADDENDUM: How to use this License for your documents

To use this License in a document you have written, include a copy of
the License in the document and put the following copyright and
license notices just after the title page:

    Copyright (c)  YEAR  YOUR NAME.
    Permission is granted to copy, distribute and/or modify this document
    under the terms of the GNU Free Documentation License, Version 1.3
    or any later version published by the Free Software Foundation;
    with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.
    A copy of the license is included in the section entitled "GNU
    Free Documentation License".

If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts,
replace the "with...Texts." line with this:

    with the Invariant Sections being LIST THEIR TITLES, with the
    Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.

If you have Invariant Sections without Cover Texts, or some other
combination of the three, merge those two alternatives to suit the
situation.

If your document contains nontrivial examples of program code, we
recommend releasing these examples in parallel under your choice of
free software license, such as the GNU General Public License,
to permit their use in free software.

Index¶

Index