In 2010, LA Times investigative reporters revealed that city officials in Bell, California were paying themselves exorbitant salaries compared to other cities. The “Bell Scandal,” as it came to be known, raised immediate questions about whether that city was unique or whether the public just did not know about corruption taking place in other cities. The State Controller’s Office, which maintains fiscal control over $100 billion in receipts and disbursements of public funds a year, believed it could help answer that question by providing more transparency.
By the time the story broke, the Office had already initiated projects aimed at transparency in government. In 2002, it provided the Sacramento Bee, as well as other news agencies, with state employee salary information that the newspaper later made public. In 2009, the Contorller’s Office made information available online about registered warrants, also known as IOUs, which the state had used to pay vendors during the recent budget shortfall. Institutional knowledge gained from these previous projects, combined with an executive-level commitment to transparency and a dedicated staff, enabled the Controller’s Office to launch this time-sensitive project using existing resources.
The weekend after the LA Times story ran, the executive team came together for a half-day conference call to determine the path forward. The Controller laid out a goal to make city and county salary, wage and benefit data public, and the team quickly began discussing how they would undertake this task. They quickly reassigned a project manager from the information technology department to the project and then created a 90-day project plan. The project required support from nearly every department, and the governance team reflected this. The team included the Chief of Staff to the Controller, Director of Communications, Chief Counsel, Chief of Administration, Chief Operating Officer, Chief Information Officer, Chief of Audits and data stewards from the Accounting and Reporting Units. The team was passionate about the project and the potential result–making government more transparent for all Californians. The project plan laid out clear roles and responsibilities for each member of the team and each member of the team also knew that this project was a critical priority for the Controller. This meant that they had the support they needed to realign internal priorities so that the Public Pay project, as it had been named, had the resources it needed to meet its goals.
Ninety days later the team launched the first version of publicpay.ca.gov. Over the following months and years, the team updated that first version with more data and more tools. Today, the website provides a comprehensive open dataset that is used by journalists, local governments, auditors, unions and the public. The Controller’s Office was able to turn this idea into a public tool in only 90 days because it had:
Executive leadership from the Controller;
A governance structure already in place as a result of previous transparency projects;
A commitment to using internal resources; and
An agile development and publication approach that focused on quick sprints to launch for each update over the next several years.
The Controller’s Office made a strategic choice to launch the first version of the site in only three months. They could have waited six months or a year, which would have resulted in a site with more features. Instead, they focused on an iterative approach, publishing a dataset that was downloadable and that users could manipulate. The first version was a basic, tabular form that users could search. It did not have APIs or all the visualizations that publicpay.ca.gov has now, over five years later, but the data was open to the public, and that was the critical need the Controller’s Office wanted to meet initially.
The benefit of this approach was immediately apparent. Journalists, the League of Cities, and the California Association of Counties provided timely feedback to the press office and the team was then able to integrate that into later updates. The public and other stakeholders also provided feedback via a button on the website. Over time, the Controller added employee salaries from special districts, California courts, California Community Colleges, California State Universities and the University of California. In response to feedback, technical staff also edited documentation to include less technical jargon.
As the team added more tools to the site, they designed them for use by the general public. Power users, they knew, could download the entire dataset and use their own tools. However, these stakeholders made up only a small part of the potential audience for the data. To meet the needs of other users, the team designed easy point-and-click analysis and visualization tools that provided more opportunities for everyone to use the data.
When the Controller’s Office launched the first Public Pay site, they intentionally designed a basic system. As they and stakeholders had new ideas to improve the project, they would regularly add functionality. The team would assess each request and if they could easily integrate the change, they did. However, if the change was of a certain size, the Controller’s Office would integrate it into the set of design requirements they were developing for a full redesign, which they launched two years later.
In undertaking the redesign, the team wanted to integrate advanced analytic capability into the project. However, they also knew they did not have the capacity to do this in-house. To meet this need, they brought in a vendor using a competitive bid process. The team was able to find the right contractor for their needs because they had well-defined requirements resulting from the knowledge they gained designing the first version of the site and the benefit of two years of user feedback.
As the team developed later open data projects, like Track Prop 30 and By The Numbers, these sites differed from Public Pay because the team had more time to plan for how they would implement the Controller’s vision. Having more time to plan resulted in different development scenarios: the team continued to develop in-house, but also hired vendors for services where it made sense based on their business needs.
The most significant hurdle the project team overcame was building the dataset. At the time, cities and counties did not submit employee salary information to the Controller’s Office. Within a very short period, the governance team had to determine which data to request and ensure that data they received would be consistent across municipal boundaries. Job titles, for example, often differ from one city to another. They had to account for people who might hold more than one position, a potential problem exposed by the Bell Scandal. The team also had to ensure they were not releasing personally identifiable information that would compromise the safety and welfare of law enforcement personnel and victims of crime. These challenges were exacerbated by a compressed timeframe that left little room for error.
The Controller’s Office had to be ambitious to meet its deadline for the Public Pay project. It was successful, in part, because the leadership team had already worked together on the project to document the state’s IOUs the year before. The experiences the team had through Public Pay and the IOU project helped them work more quickly and efficiently during subsequent open data projects like the By The Numbers. Even though they had to build a new dataset for Public Pay, the team limited their risk by publishing information in phases. They first published data and provided basic tools, and then they enhanced their tools and brought in special district information including school districts. Executive leadership from the Controller as well as his Chief of Staff was also a critical element in the project’s success. Realigning or finding resources is an easier task when the head of an organization believes in the project. Similarly, the staff were committed to the project. They were a group of “can do” people eager to launch successful projects that would provide financial transparency for the public and other stakeholders.
The team used an agile, collaborative approach to meet these challenges. They met every day at 4 p.m. and adhered to an aggressive project management schedule. They built out pieces of their project iteratively–from policy to database structure–and made sure that the entire governance team had an opportunity to quickly view and respond to each piece. Very few of the challenges had easy solutions, but the team had previously built trust by working together. This helped them solve problems collaboratively and quickly. For example, the Controller’s Office had to determine how it would collect the salary and benefit data from cities and counties. It believed it had the authority to compel this information, but many cities were initially resistant. They were concerned about employee privacy and the legitimacy of the Controller’s authority. The Controller’s Office was able to broker an acceptable solution for many local governments by requesting that cities and counties submit information from employees’ W-2 forms. To protect the privacy of law enforcement and victims of crime, the Controller requested position numbers without names. At the same time, municipalities that did not submit data in a timely fashion found themselves on a public list of non-compliant entities and facing potential fines from the Controller. Though many cities did not initially see the benefit of the data, they later found that public salary data was beneficial for them. For instance, instead of conducting costly salary studies before pay negotiations, city managers found they could use the data from the Public Pay site instead. This also created the opportunity for the municipalities and the unions to start discussions with the same neutral data information in front of them.
The team used agile methodology to launch the Public Pay project. However, this approach was also representative of the organization’s entire open data portfolio. The team learned from each project, using that information to inform later projects. The IOU project helped shape Public Pay and Public Pay helped to create the protocols for the Track Prop 30 and By the Numbers projects. Each experience built on the one(s) before, and a core team learned together what worked and what did not.
A new Controller was elected to the office in 2015, and while there has been turnover among team members, several of the original team remain. That team is now determining what next steps to take with the existing open data sites. They are also formalizing structures by writing governance documents. In government, it is standard for a new administration to review a project, its outcomes and the resources needed to continue it. Within California state government, the open data projects at the Controller’s Office are the first to span two different elected leaders. As state agencies and other constitutional offices look to implement open data projects, knowing that executive leadership may or will change in the coming years, they can also learn from Controller’s Office staff as it bridged two administrations.
Staff at other agencies may also look to the staff that left the Controller’s Office, many of whom moved with the former Controller once he was elected to the post of State Treasurer. At the Treasurer’s Office, a team composed of former Controller staff and existing Treasurer staff are now developing new open data projects. They are bringing the lessons learned from the previous projects into a new organization and using them to more quickly launch new open data programs, such as DebtWatch.