Logo for The Department of Better Technology.

Rewiring Government

The Department of Better Technology helps governments deliver great digital services to the people who depend on them.

Interview: Justin Erlich on gaining insights from open data

Screendoor logo

Streamline the process behind your online forms.

Learn about Screendoor →

Quote from Justin Erlich, excerpted from the transcript below.

In the first episode of our podcast, Joshua Goldstein, our CEO, talks to Justin Erlich, the Data and Technology Advisor to California Attorney General Kamala Harris. We discuss Kamala Harris’ launch of OpenJustice, one of the most high-profile criminal justice transparency initiatives in the country, especially relevant given the public debate around racial bias in policing.

Use the player above to listen, or subscribe on iTunes and Google Play! You can also add our RSS feed to your favorite podcast app. If you like this episode, rate and review us on iTunes, and tell your friends.

A lightly edited transcript is below.

Justin Erlich, welcome to the Rewiring Government podcast!

Thanks for having me.

Tell us what you do at the California Department of Justice.

I’m a Special Assistant Attorney General and I advise the California Attorney General on technology and data, both from a policy perspective and also operationally.

Clearly, criminal justice is an enormously hot topic right now. There’s a lot of discussion in both research communities and the public around racial bias in policing. And in this context, California’s Attorney General launched OpenJustice, perhaps the most high-profile criminal justice transparency initiative in the country. Can you tell us what OpenJustice is, and talk about its origin story?

OpenJustice is a transparency initiative that seeks to hold ourselves accountable for how the criminal justice system is doing, and improve public policy for California. It came around in the context of all the viral videos [around racial bias in policing,] and media stories highlighting that they pointed to real issues in the criminal justice system. We discovered there was a real paucity of data. People didn’t really know the size and the scope of the problem. There were just a lot more questions being asked.

The Attorney General and our office sat down, and we asked ourselves how could we contribute to the debate. In California, we sit on a treasure trove of data. We aggregate data from state and local law enforcement agencies across the state, and we had not been using, analyzing, or publishing it to the extent that we might. We really wanted to embrace the ethos of open data, and start to release and analyze it in a way that could meet public demand, and really help spark what we are calling data-driven criminal justice reform.

When you look at OpenJustice, you see information about interactions between police and the public, criminal activity, incarceration. What are some of the ways you see this data shaping the broader narrative? How do you see OpenJustice being used?

I think it stems from trying to provide a fuller picture and more context to the stories we’re reading about. If you take the anecdotes that we read, we know that there are real issues, and our hope is that putting out the data can help identify where there are problems, and make clear where there aren’t. We can then better narrow in on the policy prescriptions that will really help.

We started by releasing a few important data sets around arrest rates, law enforcement officers killed or assaulted, deaths in custody, and arrest-related deaths. We tried to look at those data points while also pulling in important contextual data, like the demographics and population of the cities in which these incidents were happening, in order to really get a sense of the full story.

I think we’re just starting to see that there are some real problem areas in certain jurisdictions that need a closer look. But I think there’s another story too: a lot of jurisdictions are doing what cops are supposed to be doing, helping increase public safety and doing the job they signed up for. The real value of using this data is gaining a real understanding of where we should focus more deeply.

OpenJustice looks and feels a lot different than a typical DOJ initiative you would see in the past. What was the implementation process like? What had to change [in your process], and what were some of the challenges in making this happen? Also, where do you see OpenJustice’s role going forward?

I think the project’s success lay in having the Attorney General fully behind the mission. They made clear that this was a priority, and identified a core set of folks to form a team who would drive this forward. OpenJustice was going to be the top responsibility or priority of this small team. It felt different because, ultimately, we spun up a small startup within the government.

We pulled a couple of web developers from other projects to help. We had a full-time product manager, myself, and we basically built a minimum viable product using our core assets of data. Then we started getting user focus groups and early feedback about what they liked and what they didn’t.

We are truly embracing the startup ethos and trying to develop in an agile fashion. This wasn’t about releasing one set of data. This is about continually providing updates and new releases as we analyze more data, and we’ll look to be responsive to the interests of our various constituents.

To give one example, this started really just purely around [publishing] data. But, when we started telling some of the stories around that data, it became clear that not everyone had a full understanding of the judicial process and the various steps in the criminal justice lifecycle. So we’ve now started to develop content: showing infographics or other media to not only help explain the data behind criminal justice performance, but also the process itself. It’s becoming more of a broader transparency effort, just based on our first round of feedback. Going forward, we’ll continue to be responsive to how we think this platform can be most useful.

You’ve sort of alluded to this, but one thing I’ve noticed is that open data initiatives can often change the data production and consumption habits inside the organization releasing it. And that’s often one of the greatest payoffs. Is that something you’ve seen?

For example, California’s racial profiling law requires police to report race and other demographic characteristics of folks stopped by officers. It’s one of the many data-intensive areas in criminal justice. As some of these new requirements and opportunities come along, are there ways to glean lessons from OpenJustice to make sure that some of these new data production and consumption efforts can be done in a more streamlined fashion?

Yeah, it’s a great question. Something that became very clear to us, and this is an important part of data transparency efforts, is the more you start using data early on, the more you realize there are issues with reporting and collection which we’ve never really needed to address. Because ultimately, [the data] hadn’t been used that much, and so [these issues] weren’t ever noticed. The moment you start really using that data is the moment that you start to kick the tires on the whole system.

It’s vital that the perfect not be the enemy of the good. Embracing the approach of crowd-sourcing or continuous improvement, you basically start with what you have. You make sure you’re cognizant of the caveats and limitations, and you use that as a jumping off point to go deeper and fix some of the issues you find along the way.

In part, I think open data is really valuable in modernizing and leading innovation in government because it’s a new way of thinking. Transparency on the outside drives two things: it drives people to [take another look] at how they’re delivering services to constituents, and it leads to an embrace of a more data-driven approach around how to maximize those services and improve delivery. In the criminal justice system, it just made sense to start there, because that’s where the real need was. Then we can start moving from looking externally to internally: to how our lawyers and our cops can better use data to help further their mission.

As you alluded to, two bills were just passed, and there’s real opportunity for data collection and analysis around them. One bill is AB-71. Law enforcement agencies are collecting data on the use of force starting in 2016. They’ll report to that in 2017. The other [bill] you mentioned is AB-953, which is collecting all traffic stop and pedestrian stop data, which will come in over the next couple of years.

I think we’ve really learned that it’s important to embrace what I’m referring to as a vertically integrated data strategy. You need to really be clear on how the data’s going to be useful, when it’s being analyzed and informing public policy. You need to understand what that looks like, and what you then need to collect in order to make that process as useful as possible. We’re currently meeting with academics, researchers, policy makers, to understand where they see data will be the most useful, which will then determine what we should collect.

Then, just as a startup would do, we’re also trying to do it with user-centered design around data collection. So, meeting with those who have to fill out the data in the first place, law enforcement, to talk about how we see this going down. What are the sources of the data? We want to get their input, so we have a real sense of their burden around compliance, and that’s how we then develop the data fields we’ll collect. Then, it’s really important to integrate key technologies to make sure that we can collect it in the most efficient way possible and in the most streamlined way possible.

In that way, it’s important to integrate all of these elements together at one point in a vertically integrated process, in order to make sure that we’re getting the best use of our data.

You’ve been mentioning “digital services” and “service delivery” a bunch. I think that’s one of the most promising areas in this space.

We’ve talked in the past about Abhi Nemani’s critique of open data, where governments focus on data publication at the expense of [thinking about how to deliver value] to citizens. Do you think that’s valid? Also, in the area of digital services in criminal justice, where do you think the greatest opportunity lies for transformation?

First of all, I’d say open data is a great example of an important step [that governments can take], where we [also] need to make sure that we take the last mile to complete the work. That’s about making clear why transparency is important. To us, transparency isn’t just releasing raw CSV files, or even just publishing APIs, but also making the data really meaningful. That means [creating] clear data visualizations that tell a story, curating and lifting up the most important insights from the data so that people really understand what it’s saying. So it’s not just available to researchers or civic coders, but also reporters, the public, policy makers.

I think that’s an important part of really understanding how this data will be useful and inform public policy. Part of [providing] digital services is creating an ecosystem around the data. Digital services are going to be just as important as furthering these open data goals, because we’re going to need to improve data collection. We’re going to need better forms to collect the data. We’re going to need digital services to help transmit the data from one agency to another, and we’re going to need digital services to better analyze the data.

We’re going to need platforms where you don’t have to be a data scientist yourself to run really complex programs in Stata or Microsoft Access. How can we use digital services to let staff members do simpler analyses without having to rely on real experts? I think digital services will form the foundation to further all of those efforts.

I also thought it was interesting how you mentioned running the project like a startup. In my experience, that’s not something that is always in line with how government agencies normally operate. Stepping back from your data and technology strategy, what was your strategy for organizational change in a place which might not always be amenable to the startup mentality?

[Our strategy] created some really interesting moments. We had to break silos from a few different divisions across the Department of Justice. Pulling together different people to work together on a small team was already a novel concept. It involved working in iterative cycles with short timelines, which is again, not a thing often done in government. It involved getting real-time feedback from users and constantly adjusting, moving away from the classic waterfall approach of building something and shipping it, towards continually altering the product.

We have a lot more data and analysis that we could put out, and we’re comfortable with just putting out a little bit. We were comfortable releasing a certain level of analysis, knowing that there was more context that could always be added, and that we would try to continue to improve. I think embracing an approach where we would just take little steps and continue to build, rather than build one big thing, was a big step for us. It was something we embraced so we could move a lot faster.

I think a lot our audience is in a position where they see the opportunities [for organizational change]. Their task is then to be diplomats for that new way of doing business internally. They understand the landscape of the tools to some extent, but also sort of feel like they face an uphill battle. Do you think diplomats within the organization can be effective, or are there some systematic barriers that are holding back some of these changes in government?

This is at the core of any transformational change program. Starting with some new way to do something is a great way to [encourage] innovation, and it can really be an opportunity to have people rethink how they do things. I think the challenge is that the conditions need to be right. You need some combination of senior-level support, the right set of [colleagues] who can ask questions about why things are being done, and external pressures that encourage people to be open to new approaches (if not all of the above.) I also think it takes a willingness to know that you have to bring your colleagues and others along with you. You have to explain the value of the change, the need for the change, to show that there’s a way to doing things differently.

Part of what you might call diplomacy or collaboration is understanding that trying a new thing isn’t just about achieving that one outcome, but about changing the hearts and minds of others in the organization. Others might have used the phrase, “Delivery is the strategy.” Just being able to show that you can do something different is just as important.

In fact, when we launched our first version of OpenJustice, there were some who weren’t sure if we were going to be able to pull it off, and some who weren’t sure what exactly we were doing. But after launch, they started to get it, and got more excited. Then we released our second version, and more and more people started to rally around it. Ultimately, it’s about figuring out how you can get a few achievable wins to show momentum and the art of the possible, and get people to rally around a new sense of innovation.

Finally, what attracted you to the civic innovation space? What drew you into this community, and where do you see it going?

I think why the civic innovation / civic tech space is so exciting is that you predominately have a bunch of very impassioned, talented people who want to figure out how to do good, while also developing their careers and embracing entrepreneurial spirit. If disruption is often viewed as taking something that works effectively in one industry and bringing it into another, this is a classic example of how can we leverage the best of what has made our tech sector and many aspects of our private sector so great, and apply those same lessons to the public sector.

It’s valuable to take a lot of really fun, interesting technologies and applying it to a space where it’s most needed. The impact could not be higher. I think what will be critical for the civic tech space to grow is making sure that we continue to develop new products and services being developed, but also be thoughtful about identifying new ways of doing things in organizations where bureaucracies are set up to move slower than we like.

This is partially using tech as a driver for cultural change. We will really need to have civic tech leaders and forward-thinking government leaders come together to figure out how to show these wins. I think helping governments move away from the large IT complex to embracing more nimble projects will be critical for not only the growth of civic tech, but also for the success of government to continue to modernize in the 21st century.

Justin Erlich, thank you so much for joining us. Where can people find more about what you’re and your team are working on?

You can visit us at openjustice.doj.ca.gov, or you can find me on Twitter @JAErlich.

Thank you so much.

Thank you!

Joshua Goldstein is the CEO of The Department of Better Technology.

Want more articles like this? Subscribe to our newsletter.