Shared Lessons from the Case Studies

Many departments across state government already publish data. However, truly “open” data is different, both in how it is published and in the expectations that the public places on it. To understand these differences, agencies beginning open data projects can benefit by learning from the experiences of the agencies and departments that have gone before. Drawing on the experiences of two such early adopters – the California Health and Human Services Agency and the State Controller’s Office – provides a few key lessons:

  • To open your data, you have to know your data

  • Data can be messy

  • Strong, involved, executive sponsorship is critical

  • An inclusive governance team can develop comprehensive policies

  • Start from a strategic strong point

  • Create opportunities for learning, training and asking questions

  • Use events to build momentum

  • Learn from other organizations

To open your data, you have to know your data. A successful open data project is about publishing data the public can use. It is necessary to know what data you have, what data the public likely will find most useful, and how to publish that data so that privacy is protected while still maximizing its value to the public. For example, the Controller’s Public Pay project was responding to an immediate need, so the required data was easier to identify. The Health and Human Services’ Data Portal project used an internal data inventory along with the knowledge of departmental data stewards to identify high-value datasets. Both agencies also categorized their datasets into three levels of confidentiality–low-risk/public data, medium-risk/possibly publishable data, and high-risk/not publishable data.

Data can be messy. In long-running programs data needs may have changed over time. New fields might have been added or existing fields may be coded differently for different years. Information stored in older database programs may be difficult to export. To prepare datasets for publication the Health and Human Services Agency and the Controller’s Office addressed many issues and documented details describing data collection, history and limitations. After release, they planned for and solicited public feedback to help them continue improving data quality and usefulness.

Strong, involved, executive sponsorship is critical. Neither of the programs profiled in this paper would have happened without clear, committed support by executive leaders. In many cases, implementing a robust open data program means altering existing business processes (e.g. how information is collected, catalogued and published) and shifting workloads in the short term. Making these changes without executive support is extremely difficult, especially if any skeptics of the program are in decision-making roles. The result of executive vision is twofold. First, it convincingly demonstrates to staff that this program is possible and important. Second, it results in clear processes that provide direction and support to staff implementing the program. Executive support requires the head of an organization to provide clear expectations for the quality of the data, and it also requires that executive to take responsibility for the data once it is published.

An inclusive governance team will facilitate comprehensive policies. Open data draws on many different areas of expertise. It is best to involve both those with decision authority as well as those with hands-on experience in managing and publishing data. The team should include the executive sponsor, the project manager, the chief information officer, legal affairs, department or unit heads, public affairs, subject matter experts, and data stewards. In addition to project oversight, the governance team will need to establish procedures for publishing data – including how to review data prior to publication. Because of the need for detailed review, the open data governance team will typically engage in more hands-on decision making at the beginning of the project. At the start, sign-off authority for dataset publication should be at the highest level needed to make the organizations comfortable. Once evaluation processes are operational, the governance team can transition to a more traditional role of setting project goals and providing oversight.

Start from a strategic strong point. Nothing builds momentum better than an early success. At the start, efforts should focus on a test case that maximizes the chances for a successful release. This means building a team that supports the goals of the open data project, and – as much as possible – has the necessary skills and experience already in place. It also means selecting the right pilot data to begin publishing. For example, Health and Human Services identified data across all 12 departments that had already been approved for public release. This allowed them to focus on the tasks of prepping data for an open environment and communicating how this data could be used. The Controller’s Office chose a different challenge – creating a new data set, built from data that the Controller had not collected up to that point. However, this risk was manageable because the Bell Scandal provided clear evidence of the need for such data, and provided the additional pressure the team needed to convince reluctant local governments to report the data.

Create opportunities for learning, training and asking questions. As the project rolls out, staff and stakeholders will have questions and concerns. Providing space for staff and stakeholders to learn, adjust to changes in regular business processes, receive training and have their questions taken seriously is important for positive cultural change. The Controller’s Office and Health and Human Services adopted multiple strategies to meet this need. Both organizations emphasized an agile approach. The Controller’s Office, for example, focused on a single new dataset that its experienced project team could focus on in detail. As they published the data they “learned by doing,” iteratively improving their process and product. Because of their large size, Health and Human Services had to emphasize more formal training. As each department has moved to the front of the queue, the project management team provided ongoing training and mentorship.

Use events to build momentum. Events play an important role in creating enthusiasm among staff and opening communication with internal and external stakeholders. Conferences, brown bags and road shows provide stakeholders across departments and agencies opportunities to meet and share their opinions about open data. They also act as mechanisms to communicate agency goals to members of the community, receive feedback about what works and what doesn’t, and build external partnerships.

Learn from other organizations. Other organizations inside and outside of state government have developed open data projects. For the most part, they are eager to share their successes and their challenges. They often have, and will share, governance documents, code, contract pricing and partnership arrangements. This is a movement built on collaboration and transparency. Organizations new to open data can leverage that culture to jump-start a new program.