how transparency works

It’s no fun to argue with the dead, but sometimes it can’t be helped. Or at least, I can’t help myself.

I’ve tried to limit how much I say about Aaron Swartz’s death, because I’m not sure I have much right to say anything. I only met the guy a couple of times, exchanged a couple of emails, replied to a few tweets. We were not friends, and I understand that I don’t deserve to grieve the loss of him the way his loved ones do. That would be distasteful and presumptuous of me.

But it is true that Aaron was part of an embarrassingly small pool of technologists that I revere — people toward whom I do not feel my usual, regrettable reflex toward rivalrous, Holden Caulfieldish pissiness. Instead, I was in awe of Aaron. I told the PACER story frequently, and I read his book reviews with disbelief. (I didn’t realize it at the time, but I suppose I must have begun my Goodreads book-reviewing habit as a way of emulating him.)

I didn’t know him well, but I sort of thought that someday I would, when (if!) I had done something that earned me the right. The fact that this will never happen is something that I suspect I haven’t fully grasped.

I’ve been reading a lot of his old stuff this week, and today that included the chapter he contributed to the book Open Government, which O’Reilly Media generously open-sourced as a memorial to Aaron. Entitled “When Is Transparency Useful?”, I think it could fairly count as one of the “high-profile, public denunciations” that Cory Doctorow mentioned in his remembrance of Aaron.

Aaron began work on a project supported by Sunlight, but he ultimately decided it was fruitless and abandoned it before completion. This was all before my time, but I know it resulted in some hurt feelings. I’m sorry things ended that way, because in this case I think Aaron got it wrong. He had good company, though: people like Lessig and Clay burned out on transparency, too.

I haven’t, and I don’t think I will. My understanding of how transparency works is different, I suspect (and probably different from that of my colleagues). It might be worth writing it down here, in advance of the next ally that loses faith.

So. Let’s say we’re running a school. The school has a cafeteria, which is largely unmonitored. In political theory we would call this a Hobbesian state of nature, but never mind that for now.

Within this cafeteria we have a bullying problem. Some kids make a regular practice of socking other kids in the gut and demanding their lunch money, which is then handed over. This is against the rules, of course, but enforcement is imperfect. Sometimes a teacher will be passing through and catch a bully in the act, or a victim will muster enough evidence and courage to tattle effectively. When that happens, detention is assigned to the bully. Along with anecdotal evidence, the number of detentions forms our sense both of the extent of the bullying problem and of how effectively we’re fighting it.

Eventually, repeated petitions from the members of the Audio-Visual Club convince us that this situation is intolerable, and we agree to install some surveillance cameras in the cafeteria. We can’t monitor every moment of footage, of course, but we make a concerted effort at the start, handing out tons of detention slips. Soon the bullies wise up, the on-camera punching stops, and the detention hall sits empty.

There are a lot of potential problems with this. Maybe the bullies have taken to waiting for the A.V. Club by the flagpole after school. Maybe they’re issuing verbal threats instead of physical ones. Maybe the invasion of student privacy that the cameras represent is a substantial harm that outweighs the reduction in bullying.

But it would be pretty silly to look at the empty detention hall and conclude that it means the cameras aren’t working. And it would be downright stupid if we gave in to disappointment because we find our lack of bullies to punish less emotionally satisfying than the pre-camera days.

Besides, even if the system isn’t perfect, it’s still probably harder for bullies to catch kids when they’re running past the flagpole instead of massed in the lunchroom. If the threats are now verbal, at least fewer people are getting punched.

You can argue that other reforms would be superior, and I might agree with you. Having the school sell lunch plans that eliminate the need for carrying cash, for instance, might make this entire problem disappear. But I do think that the cameras were a useful intervention, even if they fall short of “silver bullet” status.

Obviously it’s easy for me to concoct this story. But there’s evidence that this is how transparency actually works.

It’s deeply unsatisfying, I know! Believe me, I dream about our databases leading to corrupt politicians being marched off in handcuffs. It’d make for some great stories — I suspect grant officers would much rather hear about that stuff than about unmeasurable counterfactuals (“Why should we keep paying for these videotapes when they don’t show any bullying?”). If I’m honest, though, it’s probably a better idea to raise the costs of bad behavior so that it doesn’t happen as much, rather than trying to catch bad behavior after it happens.

Perhaps this isn’t enough to fix things. But it really can help.

7 Responses to “how transparency works”

  1. Eric Mill

    I’m very happy you wrote this. I expect I’ll be citing it frequently.

  2. Clay Johnson

    I don’t think it’s fair to say that I (or Lessig for that matter) “burned out” on transparency. I did write a book that said “hey, you should pay more attention to public data to inform your opinion.” Rather, I think that we ascribe too much value to it as a final solution. I think a lot of people in our community think that transparency is universally good, and I don’t believe that to be the case.

    To use your example, the cameras don’t make a bully decide to be ethical. They were already beating up little kids and taking their money. Why on earth would we presume that someone like this would look at a camera and go “you know, I’ve reconsidered my life, and decided not to punch out little kids.”

    This assertion, held on to too many in our field, is my primary complaint with the “transparency is a universal good” philosophy. What you’ve accomplished isn’t “making it so little kids don’t get bullied” but rather “making it so little kids can eat safely” which, itself is a good thing. The problem is, we tout that as a success, whilst our kids still come home with black eyes.

    I generally think transparency is a good thing. But I think we need to be realistic about the good it creates, and the problems we intend to solve with it, and be honest with ourselves that if we intend on solving a problem like bullying in schools, we must compare the cost of the cameras and monitoring to the cost of other methods of stopping the bullying.

    In the world of transparency, we claim victories like “The House released their office disbursement reports” without looking back and saying “and the good that that created was that Breithbart got to write stupid stories about bottled water.” Should that data be open to the public? Sure. But we should also look back to learn where returns on investment are, and actually get smarter about this stuff.

    I’m a man that’s phobic of religion, and I worry too often that the transparency community has a religious zealotry about it that heralds itself as infallible. That’s not how we solve old problems, but rather how we create new ones. I wish transparency thought of itself as a weapon in an arsenal to fight corruption, but not the only weapon.


  3. Thom Neale

    +1 for the phrase “Holden Caulfieldish.”

    I don’t view transparency as something inerrant or as an end in itself, but as something more basic, like measurement in basic research. Lab work can be costly and time consuming, and more than one administrator has probably asked herself whether it’s worth the cost sheets she signs off on every month. But there’s a sense in which observation is fundamental to knowledge. And if the impetus for change arises from knowledge of and dissatisfaction with current conditions, then without observation, knowledge–and therefore change–is impossible.

  4. Thom Neale

    Er, and by the way, I just generally like any lengthy phrase ending in “ish,” like “Lord of the Ringsish.” I wasn’t seconding the comparison of Tom Lee to Holden Caulfield. You’re way more of a MacGyver (minus the mullet).

  5. Tom

    You are too kind, Thom. I’m sticking with Holden :-)

  6. No transparency organization can get away with… | open up

    […] PS – thank you to Sunlight Foundation’s Tom Lee for pointing out that transparency also works by forestalling bad action, which is hard to second guess and/or measure. […]

  7. An Ethnographic Approach to Impact Evaluation: Stop Measuring Outputs, Start Understanding Experiences | Reboot

    […] the in-depth research needed to assess motivations and outcomes. And everyone will maintain that attribution is tough, as is developing a credible counterfactual—how to gauge what would have happened without this […]

Leave a Reply