Showing posts with label freedom of information. Show all posts
Showing posts with label freedom of information. Show all posts

Friday, December 28, 2012

2013 Free our data - a great presentation from Pia Waugh

Open Data advocate Pia Waugh spoke recently on the topic of freeing government data at Ignite Sydney 9 (an event where speakers get five minutes and 20 slides to say their piece).

It provides a strong view as to why governments need to open up data to the community and is definitely worth viewing and sharing.

2012 Why do agencies struggle with FOI and open data so much?

The linked email conversation (in a blog post), Freedom of Information Request for Classification Data, provides an interesting insight into the struggles government agencies are having with FOI and open data and with the difficulties applicants are having accessing data which should be available in reusable formats.

In this case information which is publicly available and searchable has been made less accessible by an agency in their site (breaking a site scraper). Then after the developer asks the agency for access to the data under FOI the agency (after several delays) offers to make it available for $4,000.

As far as can be determined from the information provided, the process for releasing the data - which is already in a database - simply requires a single SQL command.

The appearance is that the agency is being badly let down by its IT systems or staff - or that it is unwilling to provide the data.

Either situation is a sad reflection on the agency and on the commitment of the government to openness.

I'll keep tracking this request - as are also a number of people in the open data space - to see how it is resolved, and how long it takes to do so.

Saturday, December 15, 2012

2012 OpenAustralia Hack(s)fest on FOI - for hackers, media, activists & FOI gurus

The OpenAustralia Foundation will be holding the first Australian Hack(s)fest as part of the countdown to the launch of their new FOI assistance site, designed to make it easier for ordinary Australians to put in FOI requests to Commonwealth agencies.

The event, being held in Sydney at Google's office, will be held on the weekend of 17-18 November.

For more details and to register, visit: www.openaustraliafoundation.org.au/2012/11/05/youre-invited-to-our-freedom-of-information-hacksfest/

2013 But we're the experts! Why the 'internal expert' democratic governance model is gradually failing and what can be done about it.

Most public sector agencies are designed as centres of expertise on policy and service delivery.

By gathering, or training, experts in a given topical area and marshalling and directing this expertise to resolve specific issues and goals, agencies have been designed to design and deliver effective and sound policy and service delivery solutions to governments for communities.

Sure these powerhouses of expertise consult a little on the fringes. They access academia and business to provide 'fringe' expertise that they cannot attract into their agencies and engage with NGOs, community groups and individual citizens to check that service delivery solutions meet the 'on-the-ground' needs of specific communities.

This is necessary for fine-tuning any policy or service solutions to meet specific needs, where cost-effective to do so.

However the main game, the real policy powerhouse, are the government agencies themselves, who take on the roles of researcher, think tank, gatekeeper, designer and deliverer through their central pool of expertise.

This is a longstanding - even 'traditional' approach to governance. It was designed and adopted in an era where geography, communication and education limited the extent and access to expertise in a nation or community. Where, often, many people were disempowered politically and economically through limited access to information and knowledge.

Consider Australia at Federation in 1901. 

The new Commonwealth Government, in addressing national issues, had to serve a population of 3.7 million people (smaller than Victoria's population today), with an average age of 22 years old, dispersed over 7.7 million square kilometres.

There was no telephone, radio, television or internet, however the overland telegraph, which gave Australia high-speed communication with the world, was 30 years old, having been completed in 1872 and extended to Perth in 1877. While most communication travelled at the speed of a fast horse, train or ship, it was possible to share information across Australia the speed of light, though at the rate of only a few messages at once. This telegraphic networked served as Australia's communication backbone for almost another fifty years, until telephones became popular after World War II from 1945.

In 1901 Australia had one of the highest literacy rates in the world (80%) with school compulsory to 13 years old, though attendance was not enforced, many remote communities didn't have access to schools and Indigenous Australians were excluded. Literacy meant basic reading and writing, with the ability to add and subtract - the books issued to 13yr olds today would have been far beyond the ability of the majority of students in 1901.

The majority of Australia's 22,000 teachers hadn't attended a teacher's college, generally serving an apprenticeship as 'pupil teachers' and few 'technical colleges' existed to teach advanced students.

In 1901 there were only 2,600 students at Australia's four universities (0.1% of our population) and CSIRO wasn't even an idea (formed 1926). There wasn't a record of how many Australians had received a university education until the 1911 Commonwealth census, which reported 2,400 students at university and 21,000 'scholars' (with their level of education undefined).

In this environment, expertise was rare and treasured. Governments employed the cream of Australia's graduates and were almost the sole source of expertise and thinking on policy issues that the new nation had to address.

The 'government as centre of expertise' model made sense, in fact it was the only viable way to develop a system capable of administering the world's smallest continent and one of the largest, and most sparsely populated, nations.


Jump forward a hundred and ten years, and Australia is one of the most connected nations on the planet, with 98% of our 22 million citizens having instant access to the world through the internet and virtually every Australian having access to telephones, radio and television.

Education is compulsory to 15 or 17, with the majority of teachers tertiary educated and school attendance strictly enforced - including for Indigenous Austalians. We have about 41 universities, ten times as many as in 1901, as well as over 150 other tertiary institutions, with over 21% of Australians having received tertiary education.


As a result, the 'government agency as expert' model is failing.

Across our population there's far more expertise outside of government than within. Governments struggle to attract and retain talent in a global market, hamstringing themselves by restricting employment to Australian citizens, while the commercial sector internationally will happily take Australia's best trained minds and put them to use elsewhere in the world.

Despite this, government's basic model has barely changed. Agencies are structured and act as 'centres of expertise' on policy and service delivery topics.

True, there's a little more interaction with academia, with business and even with citizens. However agencies remain structured as 'centres of expertise' for policy design and service delivery, designed to serve communities with limited communication or education - limited capability to do for themselves.

This model may remain effective in certain parts of the world, in nations where literacy is low, geography remains a barrier and communication infrastructure is weak - like Papua New Guinea, regions in the Amazon and some other remote areas and developing nations.

However in developed nations, with high literacy, substantial tertiary education, where geography is no limit to communication and access to media and internet are almost universal, does the approach retain the same merit?

A 'centre of expertise' approach also has many downside risks which are necessary features of the system, providing the separation and public trust required by governments to operate in this way.

For example, when government agencies structure themselves as the experts, they need to maintain a level of mystique and authority to justify public trust that they are providing the best advice and solutions. 

Just like a religion has special rituals, and restaurants rarely let you see how their kitchens operate in order to preserve the trust of their worshippers and customers, government agencies conceal their day-to-day operations from scrutiny to maintain a mystique of expertise and create a clear separation between the 'agency business' of government and the external workings of society. 

This often involves keeping policy development processes hidden behind a wall of secrecy, bureaucratic language and bizarre semi-ritualistic procedures. 

This is also why, despite FOI and other approaches, governments largely remain secretive about their processes for designing policy. They can be messy, which may reduce trust and call into question the expertise of the agency or government.

As a result, if you're not a policy expert, in most countries it is unlikely that ordinary citizens have much knowledge of how an agency has developed a given policy, who was involved (formally or informally) or why certain decisions were reached. These activities are done behind closed doors - in the confessional, behind the kitchen wall, backstage - with all their inherent messiness, testing of 'dangerous' ideas and economic modelling of who wins and who loses with any specific decision treated as confidential and secret knowledge.

This leads to a second issue and a rationalisation. As the public isn't aware of how a specific policy or service was developed, generally seeing only the final 'packaged' solution, agencies can reasonably and logically argue that the majority of the public have little to add to the policy process. 

'Expert' policy officers can argue that; the public doesn't have sufficient context, doesn't have all the facts, doesn't understand the consequences of decisions or the trade-offs that had to be made.

And of course this is true. Because the public were not part of the process, they did not go on the same journey that the public sector 'experts' went on to reach a particular policy conclusion.

The public is told 'trust us, we're the experts', and again this is indeed true. Only the policy insiders had the opportunity to become the experts, all others were kept outside the process and therefore can never fully understand the outcome. 

Success in implementing policies and services relies on the public trusting agencies and governments to be the experts. To trust them to do their jobs as the 'experts' who 'know better' than the community. However the 'secret agency business' of policy and service design can feed on itself. Government may attempt to keep more and more from their citizens as, from their perspective, the more they reveal the less the public trust agencies. 

This is a tenuous approach to trust in modern society, where scrutiny is intense and every individual has a public voice.

If the agency policy experts, in their rush to meet a government timetable, overlooked one factor, or misunderstood community needs, a policy can quickly unravel and, like an emperor with no clothes, the public can rapidly lose faith and trust in government to deliver appropriate solutions.

In this situation it is rare that an expertise-based agency or government will be willing to publicly admit that they misunderstood the issue, convenes the people affected and expertise in the community and discusses it until they have a workable solution. It does happen, but it is the exception not the rule.

Instead, the first reaction to external scrutiny is often to protect their position and justify why the public should trust them. They may draw the wagons round, either seeking to bluff their way through ('you don't understand why we made these decisions, but trust us'), 'hide' the failure under a barrel (it was a draft, here's the real policy), or to tell the public that the agency will fix the issue ('trust us this second time'). 

In some extreme examples, governments may even cross lines to protect their perceived trust and reputation - concealing information or discrediting external expertise in order to justify the expertise inside their walls and try to regain public trust.

There are other risks as well to the government as expert model. Policy experts, who have worked in the field a long time, may not accept the expertise of 'outsiders' who appear to be interloping on their territory ' who are they to tell us what we should do'. Agency experts may become out-of-date due to not working in a field practically for a long time, they may hire the wrong experts, or simply not hire experts at all and attempt to create them. 

In all these cases, agencies have a strong structural need to preserve public trust and their integrity - which may often exhibit itself as 'protecting' their internal experts from external scrutiny, or otherwise attempting to prevent any loss of reputation through being exposed as providing less than good advice.

These risks mean that the government as expert model is under increasing pressure.

A more educated and informed citizenry, with high levels of access to publication tools means that every public agency mistake and misstep can be identified, scrutinised, analysed and shared widely.

Each policy failure and example of a government agency protecting itself at the expense of the community. Each allegation of corruption, fraud or negligent practice - whether at local, state or national level - contributes to a reduction in trust and respect that affects most, if not all, of government. 

Of course this government as expert model hasn't completely failed. There are areas that the community isn't interested in, where the government is indeed the expert or where it would be dangerous to release information into the public eye - where we do have to trust the governments we elect to act in our best interests without the ability to scrutinise their decisions. These areas are shrinking, but some are likely to always remain.

However the model started fraying around the edges some time ago and we see it represented today in the increasing lack of respect or trust in government. 

Citizens don't compartmentalise these failures in ways that governments hope they will, often seeing them as systemic failures rather than individual issues.

As a result citizens trust governments less, have less faith that governments can develop appropriate policies and services and turn even more scrutiny onto agencies - even when unwarranted.

The failures of the government as expert model are only likely to grow and extend, with greater scrutiny and greater pressure on agencies to perform. This, unfortunately, is likely to lead to more errors, not less, as governments seek to make faster decisions with fewer internal resources, less experts, less time.

So how does this failing model get resolved? What are the alternatives approach that governments can adopt to remain effective, relevant and functional in a society with high literacy, education, access to information and almost universal capability to publicly analyse government performance?

In my view the main solution is for governments, except in specific secure topics, to turn themselves inside out - changing their approach from being policy and service deliver 'centres of expertise' to being policy and service delivery 'convenors and implementors'.

Rather than seeking to hire experts and design policy and services internally, agencies need to hire people who can convene expertise within communities and from stakeholders, marshalling it to design policy and codesign service and focus on supporting this process with their expertise in structuring these approaches to fit the realities of government and implementing the necessary solutions.

This approach involves an entirely transparent design process (for both policies and services), making it possible to inform and engage the community at every step.

Within this approach, government agencies gain the trust of the community through managing the process and outcomes, not through being the expert holding the wisdom. The community doesn't need to trust a black box process, it comes on the journey alongside the agency, developing a deeper and richer trust and support for the outcomes. As a result, the energy of the community is aligned to support the agency in making the policy succeed, rather than being disengaged, or actively opposing the policy and leading to failure.

Government becomes an active participant and enabler of the community, reducing the cost of communicating information and influencing citizen ideas as citizens are influenced through their participation or observation of the proces.

This approach does require substantial education - both within government and within the community - to ensure that all participants are aware and actively engaged in their new roles. It can't, and shouldn't, be introduced into all agencies overnight and there are some policy requirements where security should take precedence and processes cannot be as fully revealed.

However the approach could be introduced relatively easily (and some governments around the world have done this already). For instance, a government could select three to five issues and put together taskforces responsible for taking a collaborative approach to deliver specific policy or service solutions. 

These taskforces provide a 'public secretariat' for managing community and stakeholder involvement, acting as facilitators, not operators, to marshal community engagement in the design process.

This could even be done at arms length from a government, with taskforces drawing on expertise from outside public sector culture to avoid accidental imposition of elements of a central command and control model, provided they include core skills from the public sector necessary to ensure the policies or services developed can be effectively and practically implemented by government.

This process would test the public policy design model, capturing learnings and experiences - not from a single process run once, but from an parallel process, with multiple taskforces running at the same time to test the real-world impact in a reduced timeframe. Learnings from the taskforces would be aggregated and used to build a more complete understanding of how to adopt the approach more widely within agencies.

Provided governments committed to the outcomes of these processes, and selected issues of interest to the community, this approach would provide solid evidence for the effectiveness (or otherwise) of a facilitation/implementation, rather than an internal expertise, approach to governance.

Alongside this moderated approach to public policy development, a complementary citizen-led policy engagement approach could be introduced using an ePetition or ePolicy methodology.

Mirroring the approach taken in other jurisdictions, where the community is given a method to propose, develop and have debated in parliament, citizen policy and legislation, this would provide another route for citizens to engage with and understand the complexity of policy development and build an alternative route for high-attention issues for which governments are not prepared to take immediate action.

This approach has been adopted in several forms overseas, such as the ePetition approaches in the US and UK, where any petition with sufficient votes receives the attention of the government and, in the case of the UK, is debated in Parliament.

more rigorous model is used in Latvia, where citizens are supported to design actual legislation online and, if they can marshall sufficient support, their bills go to parliament, they get to speak on them and the parliament votes them up or down.

This last model is beginning to be introduced in Scandinavian countries and Switzerland has long had a similar process, pre-dating the internet, which allows greater participation by citizens in decisions.

So, in summary, the 'internal expert' model designed for use in nations with limited literacy and education and poor communication, is failing to serve the needs of highly educated and connected nations, such as Australia, leading to increasing citizen concern and plummeting trust in governments.

To address this, governments need to adapt their approaches to suit the new realities - environments where there are more experts outside of government than inside and where citizens can universally scrutinise governments and publish facts, analysis and opinions which serve to increasingly force governments into difficult and untenable positions.

The key changes governments need to make is to turn themselves 'inside out' - exposing their policy and service delivery design and development processes to public scrutiny and engagement and becoming facilitators and implementors of public policy, rather than the expert creators of it.

While some areas of governance need to remain 'black boxes', many can be opened up to public participation, building trust with communities by bringing citizens on the journey with agencies to reach the most practical and appropriate solutions.

This will rebuild trust in governance and allow governments to improve their productivity and performance by tapping a greater range of expertise and building an easier path to implementation, where citizens support agencies, rather than oppose them.

2013 Open Australia launches 'Right to Know' FOI request site

The process of requesting documents from the government, though enshrined in Australian law, can be difficult for many people - taking significant time, causing significant frustration and resulting in inconsistent and substandard outcomes - as I found when asking government agencies some simple question about which public social media channels they operated and which web browsers they used.

My experience saw 25% of agencies requested never respond to the request, several more withhold information on the web browsers they used as 'commercial in confidence' (although this information is automatically transmitted to every website visited by their staff) and several others ask for up to $800 for finding the documents that provided the answer.

In many cases there was little or no information on the FOI process on agency websites and in many cases agency FOI staff contacted me to explain their FOI processes (which differed substantially amongst agencies) - adding significant cost to a process where this information could be more clearly publicly explained through agency websites.

This is, of course, a single example and doesn't represent the bulk of FOI requests - for which there's no consolidated statistical data on whether the system works well and the satisfaction of people making requests (as agencies don't assess this or aggregate this information - as they do for most other customer service processes), making it very hard to quantify how effectively the FOI system is performing other than via anecdotal views, such as mine above.

OpenAustralia has now launched a website designed to simplify the FOI process for ordinary people, making it easier for Australians to understand what they request from which government agencies and the process of making and completing these requests, as well as collecting information on the complexities and difficulties of these processes.

The website, Right to Know, is based on a UK model which has been in operation for a number of years and offers both better transparency in the FOI process as well as potentially streamlining the process for government agencies, where they will have to spend less of their valuable FOI officer time on explaining how to make valid FOI requests.

The site will also help capture released documents and collect data on the complexity and issues with FOI processes, helping the government to improve them in the future.

The site is launching at a time when the government is reviewing the operations of the revised FOI Act, and it will also be interesting to see what this review recommends - and what is ultimately done with these recommendations.

I'm hopeful that the government and public service will look on the launch of Right to Know as a positive step that supports the goals of FOI, rather than considering it as yet another impost on their operations, even if long-term FOI stalwards such as Allan Rose believe that the culture change necessary in the public service to support FOI are yet to occur.

Monday, May 21, 2012

Standardising content across government (or why does every agency have a different privacy policy?)

Every government website serves a different purpose and a different audience, however there are also standard content every site must have and legislation and standardised policies they must follow.

This includes content such as a privacy policy, legal disclaimer,  terms of use, accessibility statement, copyright, social media channels, contact page, information publication (FOI) pages and so on. It also includes the navigational structure and internal ordering of pages and the web addresses to access this content (such as for 'about us' pages).

So is there a case to standardise the templates and/or content of these pages and where to find them in websites across government?

I think so.

From an audience perspective, there is a strong case to do so. Citizens often use multiple government websites and it makes their experience more streamlined and efficient if they can find what they need in a consistent place (such as www.agency.gov.au/privacy), written in a consistent format and, where possible, using identical or near identical language.

It would also save money and time. Rather than having to write and seek legal approval for the full page content (such as for privacy information), only agency-specific parts would need writing or approval. Websites could be established more rapidly using the standard content pages and lawyers could focus on higher value tasks.

To put a number on the current cost of individually creating standard, if you assume it cost, in time and effort, around $500 to develop a privacy policy and that there are around 941 government websites (according to Government's online info offensive a flop), it would have cost up to $470,500 for individual privacy policies for all sites. Multiple this by the number of potentially standardisable pages and the millions begin adding up.

Standardisation could even minimise legal risks. It removes a potential point of failure from agencies who are not resourced or have the expertise to create appropriate policies and expose themselves to greater risks - such as over poorly written legal disclaimers which leave them open to being sued by citizens.

In some cases it may be possible to use the same standard text, with a few optional inclusions or agency-specific variations - such as for privacy policies, disclaimers, accessibility statements, terms of use, and similar standard pages.

In other cases it won't be possible to use the same content (such as for 'about us' pages), however the location and structure of the page can be similar - still providing public benefits.

Let's take privacy policies specifically for a moment.There's incredible diversity of privacy policies across Australian Government websites, although they are all subject to the same legislation (the Privacy Act 1988) and largely cover the same topics (with some variation in detail).

While this is good for lawyers, who get to write or review these policies, it may not be as good for citizens - who need to contend with different policies when they seek to register for updates or services.

Many government privacy policies are reviewed rarely, due to time and resource constraints, which may place agencies at risk where the use of new tools (such as Youtube, Slideshare and Scribd) to embed or manipulate content within agency sites can expose users unknowingly to the privacy conditions of third party sites (see how we handled these in myregion's privacy policy with an extendable third party section).

So, how would government go about standardisation? Although effectively a single entity, the government functions as a group of agencies who set their own policies and manage their own risks.

With the existence and role of AGIMO, and the WebGuide, there is a central forum for providing model content to reflect the minimum standard agencies must meet. There are mandatory guidelines for agencies, such as for privacy, however limited guidance on how to meet it. A standard privacy policy could be included and promoted as a base for other agencies to work from, or even provided as an inclusion for sites who wanted to have a policy which was centrally maintained and auto-updated.

Alternatively web managers across government could work together, through a service such as GovDex, to create and maintain standard pages using a wiki-based approach. This would allow for a consistently improving standard and garner grassroots buy-in, plus leverage the skills of the most experienced web masters.

There's undoubtably other ways to move towards standardised pages, even simply within an agency, which itself can be a struggle for those with many websites and decentralised web management.


Regardless of the method selected, the case should receive consideration. Does government really need hundreds of versions of what is standard content, or only a few?


Examples of government privacy policies (spot the similarities and differences):