Municipal Open Data Policies – San Diego Regional Data Library

Since 2006, an accelerating number of US cities have made a formal commitment to transparency by requiring publication of government data. This analysis studies ten documents cities have created to publicize and formalize Open Data policies, and includes recommendations for San Diego area governments to create their own policies.

Also available in PDF.

This report studies eight cities that have taken public action on developing Open Data programs, from the earliest program in Washington DC to Chicago’s ordinance of late last year. The study also includes the White House Open Data Directive, which is very similar to other documents. These documents take a variety of forms, from City Council resolutions to changes to municipal laws. The cities, their type of action, and how much detail is in their statements is summarized in Table 1.

City	Link	Form	Date	Definition & Detail
Washington DC	http://bit.ly/YMJvEp	DCStat Policy	June 1, 2006	High
Vancouver	http://bit.ly/171n1pK	Resolution	May 1, 2009	Low
Portland	http://bit.ly/11U9T1R	Resolution	September 1, 2009	Low
San Francisco (E)	http://bit.ly/10lNWr6	Executive Order	October 1, 2009	Low
White House	http://1.usa.gov/ZErVoO	Directive	December 1, 2009	Mid
San Francisco (O)	http://bit.ly/WZPaJI	Ordinance	October 1, 2010	Mid
Austin	http://bit.ly/ZrLgpq	Resolution	December 1, 2011	Low
New York	http://on.nyc.gov/10dUHLW	Ordinance	March 1, 2012	High
Philadelphia	http://1.usa.gov/ZEseQt	Executive Order	April 1, 2012	Low
Chicago	http://bit.ly/YWKYXT	Executive Order	December 1, 2012	High

Table 1. Cities and links to their policies

San Francisco appears twice, once for its executive order, and once for its ordinance.

This analysis shows that all of these documents present a motivation for the Open Data policy, and that there is a narrow range of motivations, with “transparency” being the most common. Motivations are presented in the first section of this report. The remaining sections present features of the documents—the clauses, terms, stipulations, or recommendations that the documents make about the Open Data Policy. These features are grouped into the following categories:

Staff and organization
Administration
Form of publication
What data is released
Soliciting feedback

Some of the documents have unusual features that are not found in others; these features will be discussed in a separate section. The final section presents recommendations for San Diego area municipalities to create their own policies.

Declaring Intent and Implementing the Policy

In many cases, a city’s open government policy begins with a city council resolution or executive order from the Mayor, and is followed by an ordinance. Resolutions tend to be composed primarily of motivations, with a small number of vague features. Executive orders also have more motivations than features, but provide a stronger mandate for the city to embark on a process to define and implement an Open Data Policy. For most cities an ordinance, a change to the city’s laws, should be the goal of Open Data advocacy; it is only when Open Data policies are translated into ordinances that the policies will have both the mandate and longevity that citizens should expect, although some cities can achieve the same permanence with an executive order.

Motivations

Most of the Open Data documents motivate the policy, usually in a preamble that states findings using many “whereas” clauses. The most common motivations were:

Transparency. Increasing government openness and the availability of information about government and its proceedings.
Participation. Allowing and encouraging citizens to participate in government and civic life.
Collaboration. Improving the extent to which city departments work together, or the city government works with private organizations.
Innovation. Interest in civic applications, new ideas, and new solutions to problems.
Progress. Civic development, economic improvement, or other aspects of community growth.

Table 2 summarizes which of the documents mentioned each of these motivations.

	Transparency	Participation	Collaboration	Innovation	Progress
Washington DC	x	x			x
Vancouver	x	x	x	x
Portland	x	x	x	x	x
San Francisco (E)	x	x	x	x
White House	x	x	x
San Francisco (O)	x	x	x	x	x
Austin	x	x	x	x
New York	x	x	x	x	x
Philadelphia	x	x	x
Chicago	x	x		x	x

Table 2. Motivations

Staff and Organization

All of the ordinances, most of the directives, but few of the resolutions address how departments will alter their staffing to accommodate new obligations under the Open Data policies. Among those that do, the major changes were:

Department Data Officer. Departments are directed to assign a person to be the liaison or manager for the department’s open data efforts, usually reporting to an advisory committee.
Chief Data Officer. Establish a role to head open data efforts across the municipal governments. Chairs the advisory committee, if it exists.
Advisory Committee. Create an advisory committee at the municipal level to oversee the entire program.

Table 3 summarizes the changes to staff.

	Department Data Officer	Chief Data Officer	Advisory Committee
Washington DC	x	x
Vancouver
Portland
San Francisco (E)
White House	x	x	x
San Francisco (O)			x
Austin
New York
Philadelphia		x	x
Chicago	x	x	x

Table 3. Staff and organization

Administration

Most policies specify administrative effort to manage the program. The major categories of administrative efforts include:

Regular Reporting. Departments must report on release schedules, compliance, or other aspects of implementing the program.
Reasonable Effort. The document states that departments shall make a “reasonable effort” to release datasets.
Develop Guidelines. Data-producing departments, the central IT department, or the advisory committee are instructed to create guidelines for technical or administrative policies.
No Restrictions. There shall be no restrictions on who can access or use data, and the website where the data is published shall not require registration before accessing data.
Privacy Preserved. The document explicitly states that when publishing data, existing requirements regarding privacy remain in force. Data that would not have been releasable to the public previously due to privacy controls still cannot be released.
Keep Current. Departments are instructed to ensure that after data is published, it is updated on a regular basis.

Table 4 summarizes changes to administration.

	Regular Reporting	Reasonable Effort	Develop Guidelines	No Restrictions	Privacy Preserved	Keep Current
Washington DC					x
Vancouver			x	x	x
Portland			x		x
San Francisco (E)	x	x	x		x
White House	x		x	x	x
San Francisco (O)	x	x	x		x
Austin		x	x		x
New York	x	x	x	x	x	x
Philadelphia			x		x
Chicago	x	x			x

Table 4. Administration

Form of Publication

Nearly all of the policies specify that data will be released in an open, non-proprietary format. Some also mention Open Source or explicitly name the URL where data will be published.

Open, Useful. The document includes language that describes the formats in which data will be published as “open,” “searchable,” “usable,” “easily analyzable,” or using similar language.
Specifies Portal. The document explicitly names the URL where the data will be published.
Open Source. The document mentions Open Source software as a goal or motivation.

Table 5 summarizes different types of publication.

	Open, Useful	Specifies Portal	Open Source
Washington DC
Vancouver	x		x
Portland	x		x
San Francisco (E)		x
White House	x	x
San Francisco (O)		x
Austin	x		x
New York	x
Philadelphia	x	x
Chicago	x

Table 5. Publication

What Data is Released

The biggest variation in the documents is in how they describe what data will be released. Some specify that all the data a department controls will be released, while others specify specific datasets.

All Datasets Controlled. Departments are required to make a reasonable effort to release all of their data, or justify what data they can’t release.
Specified Datasets. The document lists the datasets that will be released.
Identify Datasets. Departments are required to identify which datasets they will release.
Required Release. Departments are given a schedule to release some high-value datasets.
Voluntary Increase. Departments can waive exemptions and release data that they could choose to withhold.

Table 6 summarizes different ways of deciding what should be released.

	All Datasets Controlled	Specified Datasets	Identify Datasets	Required Release	Voluntary Increase
Washington DC		x
Vancouver			x
Portland
San Francisco (E)
White House			x	x
San Francisco (O)	x
Austin	x			x
New York	x
Philadelphia		x	x	x
Chicago		x		x	x

Table 6. What data is released

Soliciting Feedback

Many of the documents require that the government or departments either solicit feedback from the public, or require that the data repository have a feature for the public to submit feedback.

Table 7 summarizes which cities required solicitation of feedback from the public.

	Solicit Feedback
Washington DC
Vancouver
Portland
San Francisco (E)	x
White House	x
San Francisco (O)	x
Austin
New York	x
Philadelphia	x
Chicago

Table 7. Soliciting feedback

Unusual Features

A few of the documents have features that are not included in the other documents.

The San Francisco ordinance has a clause that refers to making datasets “available to a generic license” and specifically mentions Creative Commons. The City IT department may choose a license that would govern all data releases. Most other documents, particularly those with the “No Restrictions” feature, mandate that the data releases won’t require a license. It appears that the documents that require there to be no license are using the term only in the sense that a license restricts how the data may be used, while San Francisco refers to open licenses that explicitly grant the user rights.

The Portland resolution discusses, almost exclusively, Open Source, which is similar but orthogonal concept to Open Data.

Many documents define “Open Data” as being searchable, but the New York ordinance calls out external searchability in a separate clause.

Although it is an executive order, Chicago’s document is unusual because it is very detailed, with extensive definitions and mandating reports, new staff, and other changes.

Vancouver discusses Open Data, Open Standards and Open Source, and intends to integrate with a Canadian system for sharing data called ICIS. The resolution also directs the city to openly license software that the city develops.

Recommendations

Motivations

Include all 5 motivations. Transparency and participation are core motivations for every policy. Progress, including economic and social development, is an important reason for making data open. Collaboration is another powerful statement about openness, and it emphasis the sort of private-public integration that has resulted in many successful data efforts. Innovation is not a goal, but a required component of pioneering new ways of running government.

Staff and Organization

Use all three organizational structures: a Chief Data Officer, Department Data Officers, and an Advisory Committee.

In San Diego, many of these changes partly exist. The Chief Data Officer’s most significant role is to oversee compliance to plans and guidelines, so the position could be handled by the existing Director of Open Government. Department Data Officer is not a full-time position and can be simply a liaison to a committee. The existing Rules committee has responsibility for issues related to open data, so it, or a sub-committee, may be able to serve as the Advisory Committee.

What Data is Released

Require inventory and release. Require departments to inventory all data that they control, and to identify the effort required to make each dataset available for release. Require justification for any datasets the department does not think should be released.

Set a schedule. Set a schedule for both the initial inventory and the first three datasets that the department publishes.

Make inventory public. Require that the inventory document itself be released as data, and invite the public to participate in determining the priorities to release datasets that are not yet available.

Include SANDAG data. Include in the set of data considered for release datasets that are available to the city but are controlled by other organizations. In particular, crime incidents and traffic counts, which are controlled by SANDAG, should be considered a high priority for release.

Err on the side of openness. All data that a department controls should be considered releasable unless there is a good justification for why it is not.

Require contractors and partner agencies to release data that the city uses. Don’t allow commercial or intergovernmental relationships to become barriers to publishing public data.

Administration

Respect privacy. As all other cities have discovered, considering privacy and existing restrictions on releasing information is a requirement for any new policies.

Reasonable effort. Use a “Reasonable Effort” standard to guide departments in how much effort to expend, but also require justification for why a dataset cannot be released. These justifications should be public and open for comment.

Learn from SANGIS. Work with SANGIS and the departments that contribute data to it to identify best practices that have been successful.

Require open formats. Require that datasets be released in the most open forms possible. CSV is preferred to Excel, and Excel is preferred to PDF.

Create guidelines. Require that the Chief Data Officer and the Advisory Committee produce a guideline document for both the administrative and technical aspects of releasing data. Define a schedule by which the documents will be completed, and another schedule for when departments will implement the guidelines.

Require updates to data. No dataset that is regularly updated within the department should be updated on the public repository less frequently than once per quarter.

Use an open source repository. Set up a data portal, preferably CKAN. The site should be hosted on the city’s domain name at http://data.<citywebsite>. For instance, the San Diego data portal should be at http://data.sandiego.gov.

Conclusion

Creating an Open Data policy is a powerful commitment to openness, transparency, and public participation, and with advancing technology, it is a policy that is easy to implement and manage. Open data projects have been run not only at many cities, but many states as well, providing a base of experience that gives cities a straightforward way to demonstrate their desire for better, more efficient government.

Motivations

Staff and Organization

Administration

Form of Publication

What Data is Released

Soliciting Feedback

Unusual Features

Recommendations

Motivations

Staff and Organization

What Data is Released

Administration

Conclusion

See Also