I’m taking some grim satisfaction in the dissolution of the Haystack Project. If people have actually been hurt by this software, I’ll be removing the “satisfaction” part of that reaction.  But given the relentless overhyping of technological interventions in pro-democratic organizing, I’m hopeful that little actual damage has been done.  These technologies will no doubt soon become genuinely crucial to people living under repressive governments, but I’m optimistic that Heap’s work didn’t spread far enough to cause serious trouble.

A few key links:

• The EFF says to stop using Haystack
• The resignation letter of its main developer
• Ed Felten’s take
• Nancy Scola discusses the media’s inability to report responsibly on technology (and is very nice to me, though I hasten to say that I’m no security expert)
• The twitter stream of @ioerror, beginning at this tweet

I think Nancy’s piece raises the most important questions: why did the media — and more importantly, the federal government — fall for Austin Heap’s bullshit?  Here I disagree with Nancy: I don’t think it’s just a question of technical expertise.  I’ve been skeptical of Haystack since its announcement, but I couldn’t write a secure crypto proxy to save my life.  That stuff is really hard.

But here’s the thing: the technology is hard, but it’s also done.  Smart people have looked at these problems and have solved the ones that are solvable.  There’s work left to be done at the edges — better hash algorithms, that kind of thing — but I guarantee that’s not the sort of flaw that’s afflicting Haystack.

The thing to realize is that good security is a set of patterns.  Relatively few of them have to do with technology (although learning about the technology behind security systems can certainly help you understand the patterns).  The media’s naivete springs from the idea that the right app could liberalize Iran.  Ignorance of the app’s internals is a much smaller problem than the sort of wishful thinking that gets that story pitched and approved (and read).  Recognizing a snake oil salesman is just as important as recognizing an exploit.

Alarms should have gone off when Heap announced he was writing new software.  Why write software?  We have software.  Did Heap think the limitations of Tor and Freenet and Freegate and PGP and all the others exist because those tools’ authors are lazy or stupid?  That was never the problem.

I guess what I’m trying to say is that reporters should spend more time reading Bruce Schneier.  Even the repetitive entries. Especially the repetitive entries.

Compare and contrast.

(via Yglesias & Annie)

Excuse me for this fanboy moment.

I’m not in much of a blogging mood, but I wanted to get this down now, with a timestamp, so I can say “I told you so” in 9 months: app is going to become a dirty word.

At work I’ve been fielding a number of calls from tech companies who would like us to develop a version of one of our offerings customized to their upcoming platform — whether mobile, or web-based, or (most ridiculously) centered around a particularly power-efficient x86 processor.  This enthusiasm is a testament to the quality of the work my colleagues do and to the excitement currently surrounding the open gov space.  Both of those are wonderful things!  And to be clear, I intend to take advantage of some of these opportunities.  There is publicity to be had. In some cases I think that saying “yes” is the rational, self-interested thing to do.

But only insofar as it allows us to take advantage of an essentially irrational trend.  It is increasingly clear to me that, in the fall and winter, exclusive” app stores will begin proliferating at a pace that is unjustifiable, and which will likely lead, appropriately, to the concept of “apps” and “app stores” being denigrated and then stigmatized.

Companies have looked at the runaway success of Apple’s mobile application distribution model and found themselves slavering.  Nevermind that the mobile space, and Apple, and their first-mover advantage are all unique.  These copycats are going to try to recreate that success.  Everyone can have their own walled garden!  Every platform will be exclusive and revolutionary!  An no, of course we’re not trying to lock anybody in to anything. Why would you even think such a thing?

It’s heinous, it’s stupid, and it’s contrary to the norms that have made the internet as great and powerful as it is.

To get philosophical for a moment, all this is an iteration of one of a handful of archetypal technology debates — in this case, open versus closed.  The thing is, this is a very boring debate: we know that the answer is “almost always open”, and we’ve known it for a while.  I’d much prefer to go back to fat client versus thin client (aka “the cloud will change everything forever and I have the powerpoint to prove it”).  That debate is at least deservedly cyclical, driven by the ebb and flow of processing power, storage, technology’s social ubiquity and, more recently, battery life and wireless bandwidth.  We’ll probably arrive at an answer there, too, but not before we get a bit closer to the fundamental physical limits constraining our technologies and/or nervous systems.

Tim is one of the very few people I know of who can product posts like this one: posts which, when I’m finished reading them, seem so perfectly clear, cogent and direct that it’s hard to find a single word I’d care to quibble with.  Not that I agree with him all the time, of course.  But this time I do (actually, I’m probably willing to go somewhat further out on this limb than Tim).

This makes me wonder, though, about the compensation structure of top-tier professional writers.  I have a hard time believing that Charles Murray is feeling the financial pinch of the collapsing media industry, his complaints about Times op-ed rates notwithstanding.  It seems a lot more likely to me that his compensation has shifted away from writing-for-hire and toward various cushy sinecures.  Rich people tend to have friends who’ll help them stay rich, after all.  I have a feeling that the way things went down at the Chicago Tribune is fairly typical.  Or maybe I just took that last season of The Wire a bit too much to heart.

Anyway, rich get richer, proletariat squeezed, dog bites man, Fox News Edge at 11.

So!  At work we’ve been spending a couple of days working on off-the-wall projects — it’s a change of pace, a chance to work with folks not on our usual teams, an opportunity to try out new ideas, and a venue for some friendly competition.

One of the projects that my team considered but ultimately discarded was some sentiment analysis on on press statements made by legislators about the BP oil spill.  I figured we could pull some press releases, scan them for their level of aggression (or whatever) and compare the results to the level of oil industry support enjoyed by that legislator (thanks, Transparency Data!).  The result probably wouldn’t have set the world aflame, but if it turned out the way I expected it might’ve made for a fun and topical visualization.

As I said, we didn’t end up pursuing that idea.  But I did get far enough in researching sentiment analysis to realize that I’d like to use the ANEW dataset — a spatial model of various emotionally-charged words that would help me classify arbitrary texts.  Now, Emily says that she thinks sentiment analysis is “kind of bullshit”, and I’m not sure I disagree.  But I think it might still be interesting to run the numbers and see what comes up.

Unfortunately, the folks who created ANEW don’t want to give their data away.  Well, that’s not quite right: they’ll give it away, for free, if you’re a researcher.  A researcher who has a .edu email address.  And who isn’t a student.

This seems a little silly to me.  And it seems really silly when you consider that their widely-available 1999 paper introducing ANEW contains a complete data set.

You can probably see where this is going.

Here’s the data in CSV format.  Here’s the code used to generate it. Here’s a paper that shows how to use ANEW.

Given the age of the paper its widespread availability, I can’t imagine there’ll be any objections to transforming its contents slightly into a more useful format. If there are, I’d be happy to hear them in comments — and if any are made by the folks responsible for ANEW, I’ll be happy to remove the link to everything here that contains even a whiff of their copyright.

And of course this is pretty old data.  I’m sure that ANEW’s gotten better in the last decade (this page, for example, refers to ANEW as containing 2000 words; my copy has just over a thousand).  But it’s something to start with.  It’d be great if its creators decided to remove some of the hoops surrounding their list — there are lots of research efforts that exist outside of the .edu TLD.

• Interesting article from Foreign Policy.
• Haystack still hasn’t released any code or binaries.

Alright, a quick poll for those of you who love this album: what have you been listening to it on?  The last time I really checked these guys out was when they were being tweeted about endlessly during SXSW.  I headed to the Hype Machine and listened to what had been released or leaked.

It was physically unpleasant. I mean, not painful, exactly. That would be going too far.  But my old man ears really, really recoiled from it.

But that was all on headphones (sealed-back headphones at that). Today I’m giving the album a chance over speakers, and I have no complaints.  I actually like it!  And am adding it to my still-extremely-tenuous “2010 = summer of anthemic rock” thesis (Fang Island being the other major data point).

Maybe I’m an outlier here.  I still think the clipping is a gimmick, and one that really, really doesn’t work without a room full of air to mellow it out.  But I’m excited at the prospect of being less of a pain in the ass about this record.

(And yes, I realize that this post constitutes an inevitable but still detestable descent into explicit audiophile pain-in-the-assery.  But if you know me you’ve known for a while that this was only a matter of time.)

People seem to really be upset with Facebook this time!  Naturally, I think this is great.  I’m on record as a Facebook curmudgeon, having almost entirely displaced my anger over the flight of my friends from social blogging — a change that was probably inevitable thanks to the progression of age and career — onto the service that so many of them fled to.

But I feel some ambivalence, too.  I’m increasingly convinced that it’s fruitless to consider social networking products in terms of their absolute, instantaneous attributes.  A changed privacy policy is just one small force in a vast landscape of shifting demographics and trends.  Considering the situation as if the market is settling down, converging on some stable attractor — (“blogs and Twitter are the answer and always have been — now they can take their rightful place!”) — that’s a shallow way of thinking about it.

I’m convinced that online society has a rhythm.  A while ago, I proposed a lifecycle for social networks.  I’m pretty sure that that latter hypothesis will prove to be hopeless, that those considering the question won’t be able to draw any firmer conclusions about the fall of Friendster than historians have about the fall of the Roman Empire.  But there’s no doubt in my mind that these systems are fundamentally dynamic, and subject to entropic forces even beyond their maintainers’ sinister efforts at profit-maximization.

This Politico story finishes solidly, but man does it ever start off badly.  I suppose I shouldn’t expect anything different, but the newspapers’ obsession with an imagined era of dispassionate objectivity is now less charmingly eccentric than it is indicative of a distressing disconnection from the fundamental nature of reality.  Anyway, Ezra’s take is both smarter and, hopefully, will allow those truth-and-justice-believin-in newspapermen a way to mentally sidestep the dreaded question of partisanship.

In other MSM-bashing news: man, I had a hard time writing the morning roundup today.  The latest details from the UVA lacrosse player murder were clearly one of the day’s most prominent stories, and I found myself trying to thread the slim gap between condemning the coverage and failing to condemn the murderer.

To be clear: Huguely, having reportedly confessed to the crime, can’t be considered anything other than a monster.  But I can’t help but recoil at the way the Post’s editors handled this story.  I mean, take a look at this.  It’s cartoonish.  If you read the story it’s clear that the reporters were unable to turn up any really damning quotes about Huguely from anyone who actually knew him or his victim.  So instead they hang their case for his obvious villainy on a drunken run-in with the cops where he seems to have mouthed off, then engaged in some violent behavior before or in the course of getting tasered.  That’s not the mark of a model citizen by any measure, but it’s also not an outrageously unusual thing to find on a fraternity member and varsity athlete’s resume.

The story goes to even more ridiculous lengths, actually underscoring in the text the disparity between the victim’s lovely class photo and Huguely’s bedraggled mug shot.

I’m not at all anxious to rehabilitate the image of a murderer.  But I do think the Post is doing everyone a disservice by playing up this guy as some sort of transparent villain, skulking through the UVA campus until his dark and brutal nature finally, inevitably, claimed a victim.  That sort of laughable oversimplification doesn’t help anyone except those who enjoy clucking their tongues and patting themselves on the back — as if they don’t know anyone who could ever perpetrate or fall victim to domestic abuse.

Far more useful would  have been an effort to tell the story that seems to be implied by the scant quotes people who actually knew the couple: that all too often people fail to anticipate these tragedies before they occur. Unfortunately, even monstrous human beings rarely appear to be the sort of villainous caricatures that sell newspapers.  It’s worth taking that idea seriously more seriously than the Post did.

This is entirely correct.

Also, if any of you accidentally upgraded or have a newish device that couldn’t be jailbroken, you might be interest in this, which was released yesterday.  From there you might be interested in adding this repo to Cydia, then using it to enable tethering on your device — it works almost magically well.

Yesterday Ryan posted about the dread he feels at the prospect of one day having to take his daughter to an amusement park.

Specifically, I’m dreading the queues. Endless, winding queues, lasting hours, all to ride a roller coaster for two minutes.

He proposes the creation of a market for individual rides — whether through actual pricing or some sort of scrip — which would help shorten lines and, hopefully, do a better job of letting customers use the attractions they value most.

I don’t like this idea! I am a roller coaster enthusiast — though each visit to a theme park makes it clearer that adult bodies are physiologically unsuited for such entertainments — and I would hate to have to participate in a market for rides. I’m sure it would lead to more efficient outcomes in a narrow sense. But the increasing body of literature around the power of “free” as a price point indicates why this would still be a drag: freedom from calculation, judgment and decision-making has utility all its own. This would be no kind of way to run a society, but for entertainment it often makes a lot of sense to excuse people from the burden of weighing pros and cons. Folks really like all-you-can-eat buffets, even if they have to pay a little more. Besides, nobody wants to take their kid to the park and be asked why the rich families don’t have to wait in line.

Fortunately, I think technology can save us (surprise!). Disney’s got something called FASTPASS that lets park visitors claim an electronic reservation for a ride, then wander around the park until the appointed time. Limits on the each customers’ number of simultaneously-held reservations are enforced to prevent opportunistic oversubscription, and dynamic displays showing the expected wait times help regulate demand across the park’s attractions. You spend much less time in line listening to the loud, looping and probably-broken pre-ride video presentation, and the park has successfully put you back into the market for games, concessions and other add-ons. Everyone wins! Plus, with the ubiquity of mobile phones, the up-front cost to implementing such a system is driven way, way down — all you need to do is hand out some ziploc bags to make the system log flume-compatible.

So there. I think these systems will proliferate, solve Ryan’s worry, AND still provide enough interesting data for an econ paper or two. I think everyone can get excited about that.

Good stuff from Wired (via Julian).

Here, read this.  The author got his tweet removed via a DMCA takedown notice that read like this:

jp917, Apr 22 03:10 pm (PDT):
Hello,
The following material has been removed from your account in response to a DMCA take-down notice:
Tweet: http://twitter.com/jp917/statuses/12499491144 – New Post: Leaked: The National – High Violet http://jpsblog.net/2010/04/20/leaked-the-national-high-violet/

As he points out, this didn’t actually point to any copyright-violating files.  The link just discusses the leak.

I don’t think this is deliberate suppression of the discussion of piracy.  What I think is probably happening is this:

1. Copyright-holding Corporation A pays Technical Vendor B to monitor Twitter for leaked albums. They can do this cheaply by creating an automated process that looks for band names and the word “leaked”, or through some other simple heuristic.
2. Matching results are turned into DMCA takedown notices with a minimum (or no?) human intervention.
3. Twitter receives the notices, removes the user’s content and notifies the user with a minimum (or no?) human intervention.

The user is then free to make a counterclaim!  Unfortunately, there is not a button or script for that.  Before long he or she will have to pick up the phone, find a lawyer, and pay that lawyer to fight on their behalf.

This is flatly unacceptable.  The need for a DMCA-style simplification of the takedown process is understandable, but this level of automation of the process should not be tolerated.  As currently structured, it’s a surefire recipe for Type I errors.  We’ve known this about the DMCA takedown process all along, of course, but this really brings it into stark relief — there’s a huge power asymmetry introduced by the DMCA, so much so that the rights-holders it empowers can’t even be bothered to follow a link and read a paragraph.  Why would they?  There’s no incentive for them to.

There ought to be substantial sanctions — payable to the counterclaimant — that can be recovered when copyright holders suppress legal content through spurious claims.  Let’s get some of those much-maligned trial lawyers up in here.

My colleague Kerry assures me that I am insufferable when I talk about the interesting things I do in an increasingly desperate attempt to forestall… well, whatever [INSERT SUBJECT OF EXISTENTIAL DREAD HERE].

Still, with apologies, I will persist.  First, though: Emily tells me that research indicates that the utilitarian return on investment is better for experiential purchases — things like classes — than anything else.  I believe this!  I’ve been on kind of a class-taking kick lately: surfing, trapeze, microcontroller-programming and now welding.  It’s working out so far. Take some classes!

But okay, welding.  I’m taking a class at the Art League.  It’s very affordable, but apparently in high demand.  The subject is oxy-acetylene welding — there are opportunities to learn to use MIG and arc welding setups, but the focus is gas welding, which is both easier and less dangerous than those alternatives.

Welding is neat.  In the past, I’ve done quite a bit of soldering, a process in which a low-melting alloy is used to make electrical connections.  This background proved to be both helpful and confusing.

It was helpful when cutting metal.  This is probably the coolest thing I’ve done in the class — it’s a very different operation from welding.  The oxy-acetylene flame is used to heat the steel to a yellow-hot state, at which point you press a lever on the apparatus that projects a jet of pure oxygen onto the heated spot.  The steel then serves as the fuel, sending sparks everywhere.  The oxygen both fuels the reaction and physically pushes the molten steel away.  It’s an interesting trick, playing with metal right at the edge of liquefaction, trying to melt some but not too much of the metal in order to blow away an optimally precise slice of steel.  Soldering electrical components can definitely convey an understanding of how metal melts and flows.  I was pretty good at cutting right from the start.

I sucked at welding.  You learn to be tentative when soldering: you do have to work quickly, before the flux burns off, but in general you learn to be careful about applying heat.  Too much and you’ll burn the component; better to back off and resolder, if necessary.  This isn’t the case with welding. You need a lot of heat, and to patiently wait as the structure of the piece begins to collapse, forming an entirely new and unified hunk of metal.  It’s unnerving to watch this happen.

It gets even worse when you start to incorporate a feeder rod.  I’m assured that this makes for a stronger (abeit less attractive) weld than working without one, but it sure is frustrating.  When you feed solder into a joint, it dissolves, flows, incorporates.  The feeder rod really wants to stick.  Gently rocking it back and forth sort of works — I can see how this will eventually make sense — but it feels pretty unnatural.  Particularly given that your other hand is manipulating an insanely powerful jet of superheated gas.

Anyway!  Below is some evidence of my efforts at cutting, and my first and second weld (each of which is half with-rod and half without).  It’s all quite fascinating, I’m sure.

[flickr]photo:4527682609[/flickr]

[flickr]photo:4539402993[/flickr]

[flickr]photo:4545208967[/flickr]

[flickr]photo:4539403981[/flickr]

Now it can be said:

• Those of you who enjoyed the new Titus Andronicus record — which should be all of you, as it’s really good — would do well to reconsider the Bright Eyes records Digital Ash in a Digital Urn and I’m Wide Awake It’s Morning (and maybe the rest of the oeuvre; I couldn’t say). At the time of their release, I can’t recall any friends saying anything even vaguely nice about these albums, with two exceptions: 1) Lindsay expressed delight at the phrase “hoodie-clad trim”, which I suppose doesn’t really count, and b) Susan agreed with me that they were good and hey maybe all this NEXT DYLAN BUSINESS was maybe not so crazy, potentially? No: probably it was crazy! But pause for a moment and consider that, statistically speaking, Susan is almost certainly both smarter AND deadlier than you. Then reflect on whether you have given Conor Oberst a fair shake or whether you have discriminated against him just because he has so many feelings. Or because he was overhyped, or because his fanbase uses strange social networking sites that confuse and terrify you.Get past that. They’re good albums, is what I’m trying to say, with timbre, cadence and emotional content similar to the Titus album. Worth a shot.
• On Monday I listened, for the first time in at least half a decade, to the high points of the first big-hit Everclear LP. You know what? It holds up. This was not actually a surprise to me: I have been an apologist for Sparkle & Fade and So Much For the Afterglow for years now (the other albums: emphatically not). What was surprising: this is an alt-country album! Sort of, anyway. The guitar work is a bit too clean, and there’s nobody playing pedal steel while chain-smoking. But try listening to the phrasing of the songs on S&E with the words “Son Volt” in your mind; I think you might be surprised. And anyway albums about heroin always tend to punch above their weight. Reconsider.

I like Penny Arcade quite a lot, but this is a really terrible argument to make in response to Roger Ebert’s equally terrible argument that video games can never be art:

All you can really say about this is that neither side knows what the hell the other side is talking about.

Alright, enough hectoring.  Better news!  Tomorrow we’re having a fundraiser for work, and I hope you’ll consider coming.  It’s at Tabaq, it’s $20, there’s an hour-long open bar, there’ll be a raffle for various exciting prizes (coffeemaker! kindle! other stuff!), and the band that’ll be playing is actually pretty good (also, preposterously/awesomely, their guitar player is named Lisa Simpson).  You should come!

Sorry, I can’t help it. I’ll be quick:

• As she reported on Twitter, Emily got clipped by a car pulling out of a parking space this morning as she cycled in the bike lane, slowly, in Philadelphia (she’s fine).  A cop who saw the incident advised her to “follow the rules” to avoid this sort of situation (she had been).  It’s not always apparent, but there really is something pathological about the way we’re viewed by non-bikers.
• Less distressingly, a tip and request for motorists: I appreciate the sentiment, but you’re not actually doing me any favors by yielding to me in situations where you have the right of way — like, say, when the light has just changed and I’m trying to make a left.  It’s good to know your intentions, but I still have to assume the worst about every other driver’s plans — like those of the inattentive guy in the lane next to you.  By waving me through you’re basically asking me to either make an assumption about those other guys’ behavior and put myself at risk, or to inconvenience you by ignoring your generous offer to yield.  I don’t really want to do either.  Just treat me like a car the way the law tells you to.  But thank you for not deliberately trying to kill me!