Australia's first 3rd Generation open data site - from the ACT

The ACT government today announced the soft-launch of their new open data site,  dataACT, through their equally new  Government Information Office blog.

In my view this is now the best government open data site in Australia.

What makes it the best?

• Data is available in a range of common reusable formats - from JSON and RDF through RSS and XML - as well as CSV and XLS for spreadsheet users.
• Visualisation tools are built into the site, so data is not only useful to data scientists and programmers, but to the broader public who can chart and map it without having to leave the site.
• The built-in embed tool allows people to take the data and rapidly include it in their own site without any programming knowledge.
• Users can reorder the columns and filter the information in the site - again without having to export it first, and
• discussions are built into every dataset by default.

It follows a 'generational' path for open data I've been talking about for awhile.

Most open data sites start as random collections of whatever data that agencies feel they can release as a 'quick win', to meet a government openness directive. They then progressing through more structured sites with rigour and organisation, but still only data, through to data and visualisation sites which support broader usage by the general community and finally into what I term 'data community sites', which become collaborative efforts with citizens.

In my view dataACT has skipped straight to a 3rd Generation data site at a time when other governments across Australia are struggling with 1st or 2nd Generation sites.

Well done ACT!

Now who will be the first government in Australia to get to a 4th Generation site!

Read on for my view of the generations of open data sites:

1st Generation: Data index

• Contains or links to 'random' datasets, being those that agencies can release publicly quickly. 
• Data is released in whatever format the data was held in (PDF, CSV, etc) and is not reformatted to web standards (JSON, RDF, etc).
• Some datasets are released under custom or restrictive licenses.
• Limited or no ability to discuss or rate datasets
• Ability to 'request datasets', but with no response process or common workflow

2nd Generation:  Structured data index

• Some thought regarding selective datasets, but largely 'random'
• More standardisation of data formats to be reusable online
• More standardisation of data licenses to permit consistent reuse
• Tagging and commenting supported (as in a blog for the site), with limited interaction by site management
• Workflows introduced for dataset requests, with agencies required to respond as to when they will release, or why they will not release, data
• Ability to list websites, services and mobile apps created using data

3rd Generation: Standardised data index

• Standardisation of data formats with at least manual conversion of data between common standard formats 
• Standardisation of data licenses to permit consistent reuse
• Tagging and commenting supported, with active interaction by site management
• Data request workflows largely automated and integrated with FOI processes
• Ability to filter, sort and visualise data within the site to broaden usage to non-technical citizens
• Ability to embed data and visualisations from site in other sites
• Ability to list, rate and comment on websites, services and mobile apps created using data

4th Generation: Data community

• Strategic co-ordinated release of data by agencies to provide segment-specific data pictures of specific topics or locations
• Standardisation of data formats with automatic conversion of data between common standard formats
• Standardised data licenses
• Tagging, commenting and data rating supported, with active interaction by site management and data holding agencies
• Data request workflows fully automated and integrated with FOI processes with transparent workflows in the site showing what stage the data release is up to - (data requested, communicated to agency, considered by agency, approved for release, being cleaned/formatted, legal clearances checked, released/refused release)
• Support for data correction and conversion by the public
• Support for upload of citizen and private enterprise datasets
• Ability to filter, sort and visualise data, including mashing up discrete datasets within the site to broaden usage to non-technical citizens
• Ability to request data visualisations as a data request
• Supports collaboration between hackers to co-develop websites, services and mobile apps using data
• Integrates the capability to run hack events - potentially on a more frequent basis (form/enter teams/submit hack proposals/submit hacks/public and internal voting/Winner promotion)

5th Generation: Integrated data platform

• A common platform for all national, state and local data, with the capabilities for each jurisdiction to make use of all Generation 4 features.
• Integrated mapping environment for all levels of government, enabled with all available open data.

Standardising content across government (or why does every agency have a different privacy policy?)

Every government website serves a different purpose and a different audience, however there are also standard content every site must have and legislation and standardised policies they must follow.

This includes content such as a privacy policy, legal disclaimer,  terms of use, accessibility statement, copyright, social media channels, contact page, information publication (FOI) pages and so on. It also includes the navigational structure and internal ordering of pages and the web addresses to access this content (such as for 'about us' pages).

So is there a case to standardise the templates and/or content of these pages and where to find them in websites across government?

I think so.

From an audience perspective, there is a strong case to do so. Citizens often use multiple government websites and it makes their experience more streamlined and efficient if they can find what they need in a consistent place (such as www.agency.gov.au/privacy), written in a consistent format and, where possible, using identical or near identical language.

It would also save money and time. Rather than having to write and seek legal approval for the full page content (such as for privacy information), only agency-specific parts would need writing or approval. Websites could be established more rapidly using the standard content pages and lawyers could focus on higher value tasks.

To put a number on the current cost of individually creating standard, if you assume it cost, in time and effort, around $500 to develop a privacy policy and that there are around 941 government websites (according to Government's online info offensive a flop), it would have cost up to $470,500 for individual privacy policies for all sites. Multiple this by the number of potentially standardisable pages and the millions begin adding up.

Standardisation could even minimise legal risks. It removes a potential point of failure from agencies who are not resourced or have the expertise to create appropriate policies and expose themselves to greater risks - such as over poorly written legal disclaimers which leave them open to being sued by citizens.

In some cases it may be possible to use the same standard text, with a few optional inclusions or agency-specific variations - such as for privacy policies, disclaimers, accessibility statements, terms of use, and similar standard pages.

In other cases it won't be possible to use the same content (such as for 'about us' pages), however the location and structure of the page can be similar - still providing public benefits.

Let's take privacy policies specifically for a moment.There's incredible diversity of privacy policies across Australian Government websites, although they are all subject to the same legislation (the Privacy Act 1988) and largely cover the same topics (with some variation in detail).

While this is good for lawyers, who get to write or review these policies, it may not be as good for citizens - who need to contend with different policies when they seek to register for updates or services.

Many government privacy policies are reviewed rarely, due to time and resource constraints, which may place agencies at risk where the use of new tools (such as Youtube, Slideshare and Scribd) to embed or manipulate content within agency sites can expose users unknowingly to the privacy conditions of third party sites (see how we handled these in myregion's privacy policy with an extendable third party section).

So, how would government go about standardisation? Although effectively a single entity, the government functions as a group of agencies who set their own policies and manage their own risks.

With the existence and role of AGIMO, and the WebGuide, there is a central forum for providing model content to reflect the minimum standard agencies must meet. There are mandatory guidelines for agencies, such as for privacy, however limited guidance on how to meet it. A standard privacy policy could be included and promoted as a base for other agencies to work from, or even provided as an inclusion for sites who wanted to have a policy which was centrally maintained and auto-updated.

Alternatively web managers across government could work together, through a service such as GovDex, to create and maintain standard pages using a wiki-based approach. This would allow for a consistently improving standard and garner grassroots buy-in, plus leverage the skills of the most experienced web masters.

There's undoubtably other ways to move towards standardised pages, even simply within an agency, which itself can be a struggle for those with many websites and decentralised web management.


Regardless of the method selected, the case should receive consideration. Does government really need hundreds of versions of what is standard content, or only a few?


Examples of government privacy policies (spot the similarities and differences):

• http://health.gov.au/internet/main/publishing.nsf/Content/privacy_security.htm
• http://www.immi.gov.au/functional/privacy.htm
• http://www.ag.gov.au/Pages/Disclaimers/Privacystatement.aspx
• http://www.dpmc.gov.au/privacy.cfm
• http://daff.gov.au/about/privacy
• http://www.deewr.gov.au/Pages/Privacy.aspx
• http://www.centrelink.gov.au/internet/internet.nsf/legal/privacy.htm?OpenDocument&nav=footer