Import Points

To import points directly, on the menu select ‘Import’ and then ‘Points’.

Import Points menu option

Then enter a name and description for the data set (which will be used as the name of the active layer), enter a description of the data set (which will be used in the layer metadata).

Press the ‘Next’ button, then browse to find the CSV file containing the points.  

Name the layer and add an optional description for the CSV file

Upload Points

Upload (import) points is an option when the Spatial Portal can accept any point data. The points don’t have to be species. For example, one study wanted to map the locations of current and proposed wind farms. When ‘upload/import points’ is an option, a dialogue screen similar to that shown below will be displayed.

Select ‘Upload points’ and then press the ‘Next’ button, then browse to find the CSV file containing the points.

Upload any type of point data

Once the points have been loaded:

• The points are mapped
• The data is listed as a mapped layer using the name entered for the import/upload points dialogue screen, or for tools/export the name of the csv file
• Metadata will be created from the uploaded information
• The layer is available for sampling of environmental and/or contextual values, scatterplots and prediction

The CSV File

The import or upload points option uses a comma separated value (CSV) formatted file containing the following fields:

• Record identifier (any text),
• Longitude (decimal degrees),
• Latitude (decimal degrees – note negative values for south of the equator) and
• Up to 256 additional fields containing any alphanumeric values.

The first three columns are mandatory and the order cannot be altered i.e. RecordId, Longitude, Latitude. NOTE: Where more than the mandatory three columns are provided, a header row with column titles should be provided.

For example:
RecordId, Longitude, Latitude, PH, SoilDepth, Permeability, Slope

Any of the additional fields can be used to facet (display a breakdown of different groups of values) or further filter records by creating new layers. For more info »

A standard suite of contextual layer values are appended to the uploaded records. They can also be used for faceting and filtering. For more info »

The contextual layers are:

• Country boundaries
• Australian States and Territories
• LGAs (Local Government Areas)
• IBRA Regions
• IMCRA Regions
• Land use
• Vegetation types – native
• Vegetation types – present

A simple identifier and points only file example (demo_CSV_upload.csv):

Demo CSV upload file

Mapped points

The points in the simple demonstration file are shown on the map below.

Import points mapped

Showing the standard suite of contextual layer values added to the facet

For more information on faceting and filtering »

Metadata

When the layer metadata icon icon is selected in the layers list for the imported points or a created layer the metadata is displayed.

Import or Upload Points metadata for the simple sample set

A facet is a specific aspect or attribute of an species occurrence record that can be broken down into different groups of values, represented by different colours in the legend and on the map.

One or more of these different groups (or facets) can be selected, or filtered out, to create new layers.

Faceting species attributes

The current list of contextual layer facet options includes:

• Lifeform
• Scientific Name
• Species
• Genus
• Family
• Order
• Class
• Phylum
• Kingdom
• Data Provider
• Institution
• Year
• Month
• Collection
• Basis of Record
• Country Boundries
• Australian States and Territories
• LGA Boundries
• IBRA Region
• IMCRA Region
• Vegetation types – Present
• Vegetation types – Native

When you choose a faceting option, a coloured legend will appear below the layering list and map options. Each legend class can be selected or deselected using the checkboxes to the left of the class name.

The following example shows occurrence data for all species within the current extent. It is then faceted on Species name which is shown using different coloured points as per the legend. The species Platycercus elegans has been selected using the facet checkbox and all points of occurrence data for that species are highlighted with a red circle on the map.

Faceting on species

Alternatively as a suggestion, you might be interested in separating the data into specific States and/or Territories, or by LGA Boundary.

Scatterplots also cater for faceting.

For more information, see Scatterplot faceting »

Filtering

Species occurrence records once faceted, can also be filtered, e.g., one or more facet classes selected, to create new layers only conforming to those classes. This is known as filtering.

For example, by choosing to facet on ‘Year’, you can define a particular time period using the sliding bar under the legend or by typing in the year range boxes. Alternatively you can click in the appropriate year checkboxes. Once one or more classes are selected, a ‘Create layer’ button appears above the legend. Pressing this button, creates a new layer selecting the records that comply.

Faceting and creating a new layer by filtering on a year range

Sample file for faceting and filtering

To experiment with the power of faceting and filtering, download the sample file Demo_Random_100x9v.csv.

Example file Demo_Random_100x9v.csv

Results from importing the Sample File Demo_Random_100x9v.csv

For more information on how to import or upload points »

Import points mapped. Showing the faceting options.

Import Points choose a couple of facets

Faceting on the imported field group ‘lab05′ by clicking a couple of the checkboxes. The corresponding points are highlighted on the map.

These checked facets can be used for filtering to create a new layer, by pressing the ‘Create layer’ button.

Import Points create a new facet layer

In the example shown above both layers are displayed, but the new layer based on the chosen facets has had the size of its points increased.

Faceting in Scatterplots

Legends in the Spatial Portal allow the user to modify the display of the mapped layer. For more information on the Common Layer Functions »

However, the Scatterplot is different. Manipulating the legend for a Scatterplot not only effects the points on the map (geographic), but the points on the Scatterplot (environmental) graphical display as well.

To activate the legend in the Scatterplot, click on the ‘Species display setting’ button. This creates a floating window that renders changes to the points in both spaces on the basis of selected legend properties. The user has to press the ‘Apply’ button to activate the changes. Again, faceting is one of the options.

Note: Filtering by facets is not available. Scatterplots have the ability to create new layers based on records within or without defined environmental boundaries, not based on contextual attribute values of the records themselves. For more information on Scatterplots »

In the image below the Scatterplot is faceted on ‘Basis of Record’.

Scatterplot faceted on Basis of Record

In the next image the Scatterplot is faceted on ‘Institution’.

Scatterplot faceted on Institution

The Previous Analysis can be run in an entirely different session, browser or even PC.

Background

Each model is assigned a unique identifier that allows it to be displayed in subsequent sessions e.g. Model reference number: 1310016793060.

The identifier name varies from tool to tool:

• Classify – Reference number
• Predict – Model reference number
• Points to Grid – Analysis id

This identifier is found in the metadata of the layer. The Restore Previous Analysis tool uses the identifier to restore a model created earlier.

Prediction layer - click the metadata icon for the Model reference number

When the layer metadata icon icon is clicked in the layers list the metadata popup is displayed for the Classification, Prediction, or Points to Grid (optionally produces Occurrence Density and Species Richness) layers. The metadata can be displayed in a separate window.

The Metadata shows the Model reference number to restore a model at a later time

Run the Previous Analysis Wizard

From the menu option, select ‘Tools’, and then ‘Previous analysis’.

Enter the Model reference number given previously in the metadata and press the ‘Next’ button.

Previous Prediction Metadata

Enter the reference number to restore a previous analysis

The results again become available as a downloadable zip file, and the model is restored.

Maxent (predict) restored zip file of results

Example of a restored prediction model

Nearest Locality

From the menu option, select ‘Tools’, and then ‘Nearest locality’.

Nearest locality menu option

The nearest feature tool dialogue window instructs the user to click on the map to generate the nearest localities.

Click on the map for the five nearest localities

The named gazetteer features are shown by markers on the map and in a table that includes latitude and longitude, distance to the feature, and compass bearing. The latitude and longitude of the point clicked is prominently displayed.

Continue to click elsewhere on the map to add new markers. The five latest gazetteer features are shown in the table. Click the ‘remove all markers’ link to clear all.

Nearest localities to the clicked hand pointer

These gazetteer features could help the user decide which gazetteer polygon to use in adding an area.

Select one of the above locations using the gazetteer

The latitude, longitude of named features and the actual point clicked could help with the task of building a list of points to load into the Spatial Portal. See Import Points ».

If chosen correctly, the tool could also be used to find the nearest winery

Exported areas can be re-imported later into the Spatial Portal if you wish to reuse common shapes, or used in external GIS products (such as Google Earth, uDig or Diva GIS).

Export My Area

The user defined area, generated via digitising a polygon on the map and shown here in red, can be exported from the Spatial Portal in three different standard formats.

Export Area Menu Option

From the menu option, select ‘Export’, and then ‘Areas’.

If you have a one or more predefined active areas mapped, then these will be available as an option to export, e.g., ‘My Area’ or ‘Barossa’.

Select an Export area option

Multiple areas (mapped layers) can be combined into a single export file using the ‘All area layers’ option. In this case, both the ‘My Area’ and ‘Barossa’ layers will be exported as a single file.

Select to export all predefined area layers

Press the ‘Next’ button.

Select an export area format option

Select one of the three area export formats:

• Shapefile
• KML
• Well Known Text (WKT)

Export Shapefile

Export Area Shapefile

Pressing the ‘Next’ button produces the Shapefile zip.

Export Shapefile zip file

Shapefile Zip Contents

The Shapefile is an common open standard developed by ESRI that can be imported into many GIS products.

Shapefile loaded into uDig

Export Keyhole Markup Language (KML)

Export Area KML

Pressing the ‘Next’ button produces a zip file containing the KML file.

Export KML zip file

The KML file contained within the zip file.

An example KML file

The KML file is then available for loading into Google Earth or Google Maps or other software that accepts KML.

KML loaded into Google Earth

Export Well Known Text (WKT)

Export Area Well Known Text (WKT)

Pressing the ‘Next’ button produces a zip file containing the WKT text file.

Export WKT zip file

The WKT text file contained within the zip file.

Contents of the WKT file

The WKT is a very simple and readable text format that is also supported by a range of products. Two useful things about WKT are that the coordinates can be directly edited, and because it is simple text, it is likely to be able to be read well into the future.

Think of this tool as driving a needle through through the occurrence locations and reading off all the values of a user-selected environmental and contextual layers. In other words, the occurrences define the geographic locations where the layers are sampled.

The data produced from Sampling contains records of all the species occurrence details (mainly in Darwin Core fields), data quality checks and with chosen environmental and contextual values appended to the record.

Export Point Sample menu option

From the Menu Option, select ‘Export’ and then ‘Point Sample’.

Select an area to analyse

Note the ‘Define new area’ will involve an extra step (please refer to Add Area for additional information).

If you have a one or more predefined areas already mapped, then these will be available as an option to restrict the sample to any one of these active area layers, e.g., ‘My Area’, or in this case the named area ‘Queanbeyan’.

Select species, point locations or LSIDs

Sampling, select one of the following methods:

• All species. More info »
• Search for species by common or scientific name (can be a higher taxonomic level, such as genus, family or order). More info »
• Search for multiple species by scientific or common name. More info »
• Upload points. More info »
• Upload an assemblage. More info »

The Upload points and Upload assemblage have an additional step to upload a user’s file. Follow the links for more information. Otherwise you will be presented with a screen like that below:

Select the environmental and contextual layers to be sampled at species locations

Environmental data will return a numeric value, for example a value of the mean annual temperature at each of the point locations. Sampling contextual layers will return a category/class, for example, a land-use category such as ‘Forestry’ or a State / Territory such as ‘Victoria’.

The download data file also contains TRUE/FALSE values for a range of data quality checks processed by the Atlas, or issues raised by users (e.g., ‘Suspected outlier’). Each column will only appear, if at least one record has a value of TRUE. See the Google Spreadsheet below for more details.

There are several ways to easily select multiple layers, or to reuse a layer set from a previous session. A pre-set best 5 terrestrial layer set is also available. See Selection of multiple layers »

Clicking on the icon (to the RHS of the name column) when selecting environmental or contextual layers to sample, shows the metadata describing the layer.

Metadata for the layer Precipitation - annual (Bio12)

Press the ‘Next’ button on Step 3 Sampling to access the Download dialogue window.

Export Point Sample download dialogue window

Fill in the fields and then press the ‘Download All Records’ button.

Following the selection of the layers, a zip file is produced containing the occurrence records as a data.csv file, and the citation details of each of the data providers as citation.csv.

Export point sample data zip file

Example files contained in the data.zip:

• All species occurrence records for the Queanbeyan LGA: data.csv file (3 MB)
• Citation for data providers providing the data.csv: citation.csv file (16 KB)

NOTE: Data files can be potentially very large so they are zipped for efficient downloading. The citation file is included to ensure data providers are acknowledged accurately. You will need an application that can unpack zip files.

The Google Docs Spreadsheet contains the fields given in the data.csv occurrence download file. The fields are based mainly on Darwin Core terms. Not all fields have a Darwin Core equivalent.

Example of a saved data.csv file

Example of a saved citation.csv file

Analyses Using the Samples File

The data.csv file can be used in a wide range of analysis packages. For example, the ‘R’ language and environment for statistical computing and graphics.

‘R’ compiles and runs on a wide variety of UNIX platforms, Windows and MacOS. For more information see (http://www.r-project.org/).

Instructions for ‘R’ loading

The data.csv file downloaded from the spatial portal can be used directly in the command line in ‘R’. It takes only two lines of ‘R’ script, or four, if comments are included (as shown below). Paste them into the ‘R’ console. Remember to change the path to reflect where your data.csv file is kept.

Sample ascii text file for loading data.csv into ‘R’ – r_samples_load.txt (1 KB)

##### Reads the results of a download from the ALA spatial portal into
##### R dataframe 'data.csv' from default path, currently C:/Data/R
samples<-data.frame()
samples<-read.table("C:/Data/R/data.csv", sep=",", header = T)


1. A grid cell by species composition matrix. The number of occurrences of each of the species within each of the grid cells is tabulated and a CSV-format file produced.
2. An occurrence density map. In this case and the species richness map, a moving average is used to average the number of occurrences in the extent of the moving average and the result placed at the centre grid cell.
3. A species richness map. Calculated on the same basis as the occurrence density but the value is the average count of the number of different species.

The sites by species matrix is probably the most common data structure for ecological data analysis. The number of species occurrences and the species richness (unique number of taxa) are independently averaged over a moving window of grid cells.

From the menu option, select “Tools” and then “Points to Grid”.

The user chooses an area to apply the species to grid. Defining a new area adds an area using one of 14 different options and involves an extra step to define the area of interest.

Click the “Next” button.

Any predefined area (My Area) listed as a map layer in the top left layers list can be used for a Points to Grid report.

A single taxon (species or higher taxonomic level) or an assemblage of taxa (multiple taxa) can be used. Either search for the scientific or common names one by one, or paste in a set of scientific or common names for an assemblage (see Step 2 figures above and below).

Alternatively, the user can Upload LSIDs, a CSV text file containing a list of Life Sciences Identifiers (LSIDs) for identifying scientific taxon names.

The user sets the size of each grid cell in decimal degrees, and selects from a range of choices of grid cell matrix size to make up the moving average window configuration (e.g. 9×9 grid cells, each with a length of your nominated grid cell size in decimal degrees). The occurrence density and species richness average is calculated for the centre cell using all the cells in the moving average window. The window then shifts cell by cell to calculate the next set of averages. The moving average and sites (grid cells) by species matrix is calculated for each grid cell in the defined area.

The user must choose to produce one or more of the following:

• Sites by Species CSV report
• Occurrence Density layer
• Species Richness layer

It can take several minutes to prepare the data and layers. The time taken is proportional to the size of the area defined and the size of the grid cells used.

Sites by Species CSV report

A Sites by Species CSV report is produced, giving a matrix of grid cell sites, each with their Latitude and Longitude, by species occurrence count for each taxa per cell. This matrix can be quite sparse so the zip file that is saved may be up to > 95% of its original size.

Example Sites by Species CSV report

Occurrence Density layer

Example Occurrence Density layer

Shows a layer of the average number of species (or other taxa) occurrences at the centre of each grid cell area based on the moving window parameters (occurrence density). The colour legend gives the ranges of values for the Occurrence Density layer.

Note Occurrence Density can be considered to reflect the survey effort. It is generally greater along roads and near sites of higher human population.

Species Richness layer

Example Species Richness layer

Shows a layer of the average assemblage species richness (uniqueness count) at the centre of each grid cell area based on the moving window parameters.

The colour legend gives the ranges of values for the Species Richness layer.

The method used is called “MaxEnt” for Maximum Entropy and was developed by Stephen Phillips.
See (Maxent software for species habitat modeling).

To start a Prediction, select from the Menu Option, “Tools”, then “Predict”.

MaxEnt looks at where species occur in terms of location and environment, and also where the species does not seem to occur. A model is built using the available species and environmental data and the result is essentially, a probability surface showing where the model suggests the species could occur.

You need considerable expertise in modelling, to understand the environmental variables and areas to factor into the prediction tool. Only then can you extrapolate any meaning into the results of the predictions.

Important: The area used for the prediction will be the defined active area. If you have not specifically defined an active area, then the current map extent in the mapping window forms the active area. This is extremely important in prediction as you will not want to extrapolate too far beyond the environmental conditions where the species occurs. It is strongly recommended that you circumscribe the active area to fit the occurrence locations as well as cover the area of interest (which may extend beyond the mapped occurrences). This is always the trade-off!

Run the Prediction Wizard

Step 1 of 5 - Select an area for analysis

(Please refer to Add Area for additional information).

Step 2 of 5 - Select Species (or genus, family or order).

The auto-complete section mechanism is the same as on the Add to Map ->Species. Type the first three letters using either a scientific or common name and a list of suggestions is produced. Select a species (or higher taxonomic class) by clicking on it. The number of occurrences is given alongside the species (or higher taxonomic class).

Selected species will be mapped, if not mapped previously. See Add to Map ->Species.

The last mapped species (or genus, family or order) is included as a species selection choice in Step 2 of 5 of the wizard. (See image below).

Species can also be uploaded using two other methods:

• Upload points. More info »
• Upload LSIDs. More info »

The last mapped species (Macropus rufus) is included as a species selection choice

Step 3 of 5 - Select your environmental layers

Prediction is mainly based upon environmental layers, but after some research a new version of MaxEnt is able to accept a subset of the contextual layers as predictors. The contextual layers available for prediction are:

• Land cover
• Land use
• Vegetation – condition
• Vegetation types – native
• Vegetation types – present

The selection of appropriate layers is extremely important if an effective model is to be built. The Spatial Portal has around 300 layers covering an extremely wide range of environmental and contextual variables that experts believe could have some control on the distribution of organisms. Many of these variables / layers are highly correlated. If highly correlated layers are used, the model will be weighted accordingly by those variables, regardless of how ‘intelligent’ the method. It always pays to know your organisms well! Know what environmental and contextual factors can influence or control their distribution.

NOTE: The relationships between the layers for the colour coding has been calculated at the national extents. Layers may therefore be more or less related at different scales.

Every layer added to the model may improve it, or over-fit it. To assist, the Spatial Portal has built a form of correlation between all its environmental layers. When a layer is added to the model, the Portal examines the relationship between it and all other environmental layers. It then colour codes the remaining layers in ‘traffic light’ colours. Green against a layer suggests that there appears to be (not necessarily ‘is’) little correlation between that layer and the selected layer. Orange is intermediate while red suggests that there is a fairly high correlation. Remember that while there may be a high correlation, a layer may still have a subtle different factor that may be important. When a new layer is added, the colours are re-calculated on the basis of the closest relationship to any existing layer added to the model.

Other traps for the unwary include using occurrences that have a wide range of observation dates. In this case, the model may be poor because of migration, changed environments or even evolution. A key issue is that the species should be in equilibrium with the environmental variables. The worst case may occur with an invasive species where it’s spread may be rapid and where it is far from reaching ‘environmental equilibrium’.

Step 4 of 5 - Select Optional MaxEnt parameters

• Jacknife in MaxEnt – selecting this option examines the significance of the environmental variables – one at a time. Jacknife tests identify environmental variables with highest gain when used in isolation. High-gain variables have the most useful information when used by themselves. The environmental variable that decreases the gain the most when it is omitted has the most information, that is not obviously present in the other variables.
• Each model is assigned a unique identifier that allows it to be displayed in subsequent sessions e.g. Model reference number: 1310016793060. This value is given in the metadata of the layer. The Restore Prior Analysis tool uses this identifier to restore a model created earlier. It can be from a totally different session, browser or even PC.
• Create response curves. This option creates a graph for each layer value (x-axis) against probability (y-axis). These curves are useful for evaluating how well the selected layers have captured the species environment.
• Random test percentage. This option allows the user to define the percentage of the occurrences for testing against the output model.
• See in the Referenced Links a tutorial on running MaxEnt.

Step 5 of 5 - Enter a name for the Prediction Model

Click on the ‘Next’ button to run MaxEnt.

The map window becomes grayed-out and a Prediction data preparation and running time dialogue popup appears on the screen, to indicate how long it should take before the MaxEnt output is displayed.

Note: Building a MaxEnt model can take some time. The time is dependent on a combination of factors: the number of occurrences, the size of the geographic extent of the model, and the number of environmental layers selected.

Prediction estimated generation time dialogue box

Opening My Prediction Results Results, including environmental response graphs, are available in the downloadable zip file

The map is not included in MaxEnt results zip file, as might be expected. The reason for this omission is to hide the location of potentially sensitive species. The Spatial Portal does however display the prediction map – and as with all mapped objects, this layer is added to the list of active layers.

Map window displays the prediction probability layer

Metadata

When the layer metadata icon icon is clicked in the layers list the metadata popup is displayed for the Prediction layer. The metadata can be displayed in a separate window.

The metadata and the results from the prediction. Click to see the Model Reference Number.

The Model Reference number used to restore prior analysis is shown at the top of the metadata.

The metadata contains the methods and modelling used to create the prediction. Response graphs show the impact each of the chosen environmental layers had on the prediction. An image of the prediction probability map layer is also included.

A Case Study

A case study on using the MaxEnt Prediction tool to predict the distribution of the Greater Glider, was prepared by Dr Jane Elith, of The University of Melbourne.

Read the Case Study »

Referenced Links

• Phillips, S.J., Dudík, M., A and Schapire, R.E., Maximum entropy approach to species distribution modeling PDF (185 KB)
• Phillips, S.J., Anderson, R.P. and Schapire, R.E., 2006. Maximum entropy modeling of species geographic distributions. Ecological Modelling, vol. 190, pp. 231-259. PDF (902 KB).
• Phillips, S.J., A Brief Tutorial on Maxent. Word (4.3 MB).