We’re changing the URLs used on the LINZ Data Service

Shortly, the LINZ Data Service (LDS) will be migrating to the Koordinates cloud platform. As part of this migration, we need to change the URLs we use on LDS. This will happen overnight on Tuesday 29 August (8pm - 2am).

Most LDS customers will not be affected, and you don’t need to do anything.

Redirects will be in place for all items, so existing URLs, links, Query APIs, and most web services will continue to work as before.

A small portion of our web services and APIs will be impacted. If you use these, please see more information about this change and how it may impact you.

LDS URL changes general FAQs

Audience: All LDS users

These FAQs provide general information on what is changing, when and how, for customers who use the LINZ Data Service for downloading and viewing data.

Do I have to do anything?

Most customers don’t need to do anything, and you shouldn’t notice any changes.

A small number of our web services and APIs will be impacted. If you use these services you may need to update your API usage with the new identifiers. We’ve contacted those customers who may be affected.

What’s an identifier?

Identifiers are numbers used to designate a unique item, such as a data layer, table, document or set. These are visible in dataset URLs and displayed in the ‘Technical details’ panel for each dataset.

For example, the identifier for NZ Property Titles is 804:

https://data.linz.govt.nz/layer/804-nz-property-titles/

What’s changing?

The identifier numbers you see in your URLs are going to be a bit bigger. Layer ‘729’, for example, will become layer ‘50729.’

We’ll be putting in place redirects for all items, so existing URLs, links, Query APIs, and most web services will continue to work as before. These redirects will be supported until at least 1 Aug 2018.

What web services and APIs will be impacted?

A small number of web service and API uses will not redirect. This means, if you use any of the services listed below you may need to take action to manage the impact of this change:

  • Site-wide GetCapabilities requests (WFS, WMTS, WMS)
  • CS-W GetRecords requests
  • Site-wide, Group or Location Atom feeds

If you use any of these services and have not already heard from us, see the additional information below on how this may impact API customers.

Why is this change happening?

The identifiers need to change because they overlap with identifiers that already exist internally within the Koordinates cloud platform. For example, a dataset with the identifier ‘729’ already exists on both the LINZ Data Service and the Koordinates cloud platform.

Will the LINZ Data Service be available on 29 August?

The identifier change will take place outside of New Zealand working hours on 29 August, to minimise any disruption for our customers. It is expected LDS will be unavailable between 8-11pm NZST on that day.

I have bookmarked the URL of a favourite dataset in my browser. Will this bookmark still work?

Yes. Our page redirects will mean you’ll be able to access the datasets as before.

Will I still be able to access my recent downloads via my download history?

Yes. This change will not impact your ability to view and access the listing of your downloads from the last 7 days.

I have chosen to receive an email notification when my download is ready. Will I still be able to access my downloads via this email?

Yes. This change will not impact your ability to view and access any downloads via email.

I have used the link tool to save my own map view. Will these URLs still work? 

Yes. The redirects will also work for any saved map view (location of interest, collection of datasets and crop) you have created. This applies to both the full and short link versions.

What is Koordinates?

Koordinates is our technology partner. Koordinates is a New Zealand-based technology company that has worked in open data publishing for nearly a decade. Koordinates provide data service sites for many NZ government agencies, including Land Information New Zealand, Stats NZ and the Ministry of Defence.

Why is the LINZ Data Service migrating?

By migrating to a cloud installation, the LINZ Data Service will be easier to maintain and upgrade, and more readily ‘scalable’ should we wish to publish more data. There are also likely to be noticeable performance improvements.

What security testing has been performed?

Thorough penetration testing of the Koordinates platform has been performed by Quantum Security, along with a detailed security assessment.

What if I notice any problems?

Please get in touch with our support team at linzdataservice@linz.govt.nz You can also contact our technology partners, Koordinates, at support@koordinates.com

 

LDS URL change impacts for API customers

Audience:WxS/API customers

The following provides summary information and FAQs on the nature and scope of the impact of the URL change, to assist web service and API customers evaluate and manage any impact to their work.

On the evening of Tuesday 29 August, the LINZ Data Service will be migrating to the Koordinates cloud platform. As part of this migration, we need to change the unique identifiers we use on LDS to avoid any duplication of IDs with other datasets published on the Koordinates cloud platform.

We’ll be putting in place redirects for all items, so existing URLs, links, Query APIs, and most web services will continue to work as before. However, our testing has revealed that a small number of services will be impacted.

Impact for some web service and API requests

A small number of web service and API uses will not redirect. This means, if you use any of the services below you may need to take action to manage the impact of this change:

  • Site-wide GetCapabilities requests (WFS, WMTS, WMS)
  • CS-W GetRecords requests
  • Site-wide, Group or Location Atom feeds

Mapping identifiers

The identifier migration will see the LDS unique identifiers of current datasets change to be in a new range. Datasets (layers/tables) will have 50000 added to their ID number. Sets will have 4700 added. This only applies to datasets and other items in existence before 29 August. After this date, the allocation of IDs to new datasets will not follow this pattern. For example:

Type

Old identifier

New identifier

Layer

https://data.linz.govt.nz/layer/772-nz-primary-parcels/

https://data.linz.govt.nz/layer/50772-nz-primary-parcels/

Table

https://data.linz.govt.nz/table/1567-nz-property-titles-list/

https://data.linz.govt.nz/table/51567-nz-property-titles-list/

Set

https://data.linz.govt.nz/set/87-aims-street-address-set/

https://data.linz.govt.nz/set/4787-aims-street-address-set/

Site-wide GetCapabilities requests

LINZ Data Service WFS, WMTS and WMS support site-wide GetCapabilities requests providing you with a listing of all WFS, WMS or WMTS available datasets across the service. For example:

https://data.linz.govt.nz/services;key=[your=api-key]/wfs?service=WFS&request=GetCapabilities

Our testing has shown that requests that rely on site-wide GetCapabilities documents to access layers will fail following the site ID migration. These requests will only return the new IDs. If you have a saved project file or workbench and try to re-load the file following the ID migration, it will fail to load at all or have load errors.

Dataset-specific GetCapabiltities requests are not affected by this change and will redirect with no known issues. For example:

The layer-specific GetCapabilities request for NZ Property Titles:

https://data.linz.govt.nz/services;key=[your-api-key]/wfs/layer-804?service=WFS&request=GetCapabilities

will redirect (without you needing to do anything) to:

https://data.linz.govt.nz/services;key=[your-api-key]/wfs/layer-50804?service=WFS&request=GetCapabilities

What you need to do

After the migration, if you have any saved project files in your GIS or CAD software or FME workbench processes that rely on site-wide GetCapabilities requests, you will need to update these with the new identifiers. The simplest way to do this is to open the file and recreate your web service connection.

To minimise disruption, if possible, you may wish to change these prior to the migration on 29 August, by following the mapping logic described above (e.g. 772 – 50772). However, you will not be able to test or run requests with the new identifiers until after the migration.

You may also wish to take the opportunity to update your projects so that they request single layer versions of the GetCapabilities URLs, rather than rely on the top-level GetCapabilities document. These types of connections are easier to maintain and are more efficient to process client-side.

CS-W GetRecords requests

The LDS CS-W catalogue service provides access to metadata about our datasets. Unfortunately, redirects will not work for the LINZ Data Service CS-W.

Following the ID migration, GetRecords results from the CS-W catalogue web service will return new GUIDs (ISO metadata Globally Unique Identifiers) - old GUIDs will no longer be used. Requests will also return the new dataset IDs. GetRecordById requests for old identifiers will return ‘not-found’ responses.

What you need to do

If you are using our CS-W service to harvest metadata from LDS into your own application, you will need to remove previous records and re-harvest all records following the migration. It is important to note that as well as LDS dataset identifier IDs, GUIDs will also be changing and that any replication that has occurred to applications needs to be re-synced with new GUIDs.

Site-wide, Group or Location Atom feeds

LDS provides a number of aggregate Atom feeds to allow you to check for updates to our public datasets. These include:

The site ID migration will have the following changes to site-wide or category Atom feeds. Following migration:

  1. Site-wide feeds will only contain URLs with the new layer IDs – old layer IDs will not be present in the response
  2. In any feeds captured prior to the migration, the old layer IDs will still work as a redirect

What you need to do

If you’re using the site-wide feeds and are persisting layer IDs in the process, we recommend that you update your script to reference the new layer IDs following the mapping logic described above (e.g. 772 – 50772).

In future, you may wish to change your process to use the Koordinates Data Catalogue API for determining layer changes or table metadata. This will support private and public layers.

Changes to layer revision history feeds

Layer revision history feeds are not expected to be impacted. Feeds of layer revisions will continue to support the old URLs, return old dataset identifiers and GUIDs within the content. Links to each revision within an individual feed will be updated to point to the new IDs, but be redirected without issue.

For example, the following feed will continue to be available as is:

https://data.linz.govt.nz/feeds/layers/804-nz-property-titles/revisions/

Then links to individual revisions within the feed such as:

https://data.linz.govt.nz/table/804/history/305/

will be redirected to:

https://data.linz.govt.nz/table/50804/history/305/

Web service and API impact FAQs

I have established processes that use primary keys and other in-dataset identifiers. Will these be impacted?

No, the change will not impact any identifiers within datasets.

What web services and APIs will be impacted by this change?

Our testing has shown that most web services and APIs will continue to work as before, with the use of redirects. Only the following customer-facing services will be impacted:

  • Site-wide GetCapabilities requests (WFS, WMTS, WMS)
  • CS-W GetRecords requests
  • Site-wide, Group or Location Atom feeds

Other website service or API uses such dataset-specific web service requests and atom feeds, dataset revisions (changesets) and the Query API will not be impacted.

Can I make any changes before the migration on 29 August?

The move to the cloud platform (which is triggering these identifier changes) will be a cut-over, rather than a transition over time. You can plan for the change and even alter your processes to work with the new identifiers, if desirable. However, you will not be able to test or run any processes using the new identifiers until after the migration has taken place.

That is why we are giving you notice of the change and providing suggestions on what to do.

I’m using site-wide GetCapabilities requests in my application. Will I be affected?

Configurations relying on GetCapabilities requests will need to be updated with the new identifiers. This is because requests which return available layers across the LINZ Data Service will only return the new dataset IDs, resulting in load errors and failure.

I’m using dataset-specific GetCapabilities requests. Will I be affected?

Dataset-specific GetCapabiltities requests are not affected immediately by this change and will redirect with no known issues. These redirects will operate until at least 1 August 2018. To avoid any issues at a later date, we suggest update your requests over the next few months.

How do I know if I’m using site-wide GetCapabilities requests in my application?

If you aren’t sure whether your web service connection is using the site-wide or dataset-speific request URL, you will need to open your project or connection to review the format of the URL.

If the URL looks like this and when you initially established the connection you needed to scroll through a list of all available datasets, then you are using a site-wide URL and will need to update your connection after the migration:

https://data.linz.govt.nz/services;key=[your=api-key]/wfs?service=WFS&request=GetCapabilities

If the URL includes the ID of an individual dataset and you initially had only one dataset to choose from when establishing your connection, then you are using a dataset-specific URL and do not need to make any changes:

https://data.linz.govt.nz/services;key=[your-api-key]/wfs/layer-787?service=WFS&request=GetCapabilities

I’m using site-wide web services via AutoCAD, FME, QGIS or ArcGIS. What should I do?

After the identifier migration, AutoCAD, ArcGIS Desktop and QGIS users of WMS/WMTS/WFS web service may need to re-link layers when opening projects. FME users should check that any processing workflows are functioning correctly.

I use the CS-W web service. What should I do?

Unfortunately, redirects will not work for the LINZ Data Service CS-W. Following the ID migration, GetRecords results from the CS-W catalogue web service will return new GUIDs (ISO metadata Globally Unique Identifiers) and new dataset IDs.

If you are using our CS-W service to harvest metadata from LDS into your own application, you will need to remove previous records and re-harvest all records following the migration.

What is the impact on the Atom feeds?

Following the migration, LDS site-wide and category Atom feeds will only contain URLs with the new layer IDs – old layer IDs will not be present in the feed. However, any previous references to old layer IDs will still work as a redirect. If you’re using the site-wide feeds and are persisting layer IDs in the process, we recommend that you update your script to reference the new layer IDs.

Will layer revision feeds be impacted?

Layer revision history feeds are not expected to be impacted. Feeds of layer revisions will continue to support the old URLs and return old identifiers within the content. Links to each revision within an individual feed will be updated to point to the new IDs, but be redirected without issue.

Last Updated: 29 August 2017