My grandfather was no slouch: he was a Rhodes Scholar and had successful careers in the foreign service and pharmaceutical industry.  But that’s not what I remember him for.  Instead I think about his absent-minded professorial air; his tendency to try to repair everything with epoxy; his fondness for hollering out Aura Lee after too many sherries; his kindness, which was so profound that it — and it alone — could bridge the bitterness of his kids’ divorces.  Part of the reason these traits took the foreground was that he was by nature a humble guy.  But a big part was that he happily shared a stage with my grandmother.

She was an incredibly impressive woman — possessed of a forceful mind, a New Englander’s unshakable belief in liberal values, and a sharp sense of humor.  Recently my aunt sent me an interview that she’d just discovered, in which my grandmother spoke frankly about her life as a foreign service officer’s wife (very much a career in its own right); her experiences being stationed in Senegal, New Zealand, Australia, Jamaica, Iceland and Belgium; her sense of Europeans’ comparative drinking habits; and our family’s run-in with some of the worst of the McCarthy era.  I think it captures a lot of what made her so great.  Maybe it won’t be of interest to anyone else, but believe me: if you’d known her, you’d be rapt.

(more…)

IBM’s victory at Jeopardy has led to the usual woe-is-humans handwringing, including (charmingly) from Jennings himself.

But look, while I agree that the logic of machines crowding out off-the-shelf humans is compelling, I think there’s some reason for hope.  I, and probably most of you, already carry a substantial neural prosthesis around with me all the time, which confers superhuman trivia capabilities at a level even more impressive than the Watson system.  It’s just that a couple of the system buses are a bit slow.  The neural bus will remain relatively sluggish, but for now it’s the interface that’s the real problem.

I think there’s a lot of low-hanging fruit that can be grabbed on this front: voluntary cochlear implants, haptic interfaces, mobile or ubiquitous ambient displays — but at some point we’ll be talking about neural interfaces, and the problem’s going to require technology that’s currently only speculative.  That’s going to be a difficult problem to solve.  More difficult than creating artificially intelligent beings?  Honestly, I don’t think we know enough to guess.  But I suspect the answer could go a long way toward determining whether or not our descendants spend their days fighting Terminators.

Let me add: it occurs to me that one interesting/tragic side-effect of the steadily improving quality of these interfaces is likely to be the disenfranchisement of older workers who are biologically incapable of utilizing these self-enhancing tools.  I don’t just mean in an “old people can’t work the VCR” way (though I think that may be related); I mean that neuroplasticity declines with age, and it’s conceivable that some of these technologies will simply be impossible to install in people with less malleable nervous systems.  My understanding is that there’s already some reason to think this phenomenon affects users of cochlear implants (though I’d want to do a bit more reading about it to be sure).

How are you going to compete in the workforce against someone who can Google without a keyboard?

Over at Grist Tom Philpott has written a few posts about the dangers of diet soda, and I’ve been acting cranky about them on Twitter. To be honest, it’s not that the posts are that bad, or even that I disagree with their gist. I tend to think that soda isn’t very good for me in general, and I avoid artificial sweeteners because (in order of decreasing importance to me) I don’t like the taste, have mostly weaned myself off sweeteners, and put at least some stock in the health case against them. So I haven’t really got a dog in this fight, other than general reactionary impulses. I am also not anxious to get into a big huge debate with Tom, because frankly he knows much more about food science than I do. Besides, he seems like a nice guy.

Still, these posts got my goat. I think it’s because they’re emblematic of a number of rhetorical tics that people use when trying to write scary stories about chemicals. And I hate that shit. The audience for that kind of thing is useless at picking or prioritizing their battles; I tend to think it leads to more anxiety, fewer vaccinated kids, and absolutely no effect on policy or health outcomes (other than those related to communicable disease, of course).

Also, I’m confident that Tom’s a smart guy, so the posts struck me as deliberately engineered linkbait, which I also dislike.

Anyway, the Twitter can of worms has been opened, so let me explain here what bugged me about these posts and made me feel that they were representative of bad online coverage of environmental toxins in general.

Cherry-picking research

This one’s easy. Confirmation bias is real (look around and you’ll see it everywhere, the joke goes). I particularly winced when I read the phrase “Italian researchers”. Admittedly, I don’t know the state of Italian biomedical research. But I do follow some alternative energy stuff, and I can tell you with confidence that that country produces more cold fusion “breakthroughs” per capita than anywhere else on earth. Plus, people smarter than me seem to think the Italian academy is a joke. But hey, if the abstract says what we want it to say…

Succumbing to this temptation is understandable, and I’ve certainly been guilty of it myself.  You need to be shut up in a doctoral program for a very long time to begin to possess any kind of immunity.  Frankly, I doubt that any of us are equipped to truly resist it.  But it’s still irritating to be on the other side of it.

Nomenclature as indictment

Particularly infuriating. Lots of chemicals sound scary, and health writers never resist the opportunity to deploy a 5,4-methylpropylbenzepene or whatever into a laboriously recited ingredient list. But it’s meaningless. The nomenclature got a lot scarier when chemists changed how they name compounds. There were good reasons for doing this (reasons that open data evangelists would find particularly appealing!), but the change does seem to have some unfortunately profound side effects on the human mind.

Similarly, pointing to constituent elements or the chemical class to which a compound belongs is rarely illuminating. At one point in the third post Tom discusses how caramel coloring is made by denaturing starches with heat and “ammonia-based” compounds. A shout-out to something you keep under your sink next to the drain cleaner can’t help but cause alarm. But of course fertilizer is also “ammonia-based”; and nobody seems to worry much when it gets turned into protein and eaten.  If you are Derek Lowe or someone like him, you can start to develop rules of thumb about the biological effects of compounds based on their structure.  If you aren’t, you can’t and shouldn’t.

Tom would probably object to this characterization, as his account goes on to discuss a byproduct of the denaturing process that’s carcinogenic. I’d say that that falls under “cherry-picking research”. In truth, lots of cooking processes produce carcinogenic compounds (the process he describes is not dissimilar to how pretzels are made). Remember the scare over acrylamides in french fries and other oil-cooked carbohydrates from a few years back (or the one over the char on your steak)? Suddenly it became apparent that we were all consuming boatloads of cancer-causing chemicals!  But it turned out that the situation’s more complex than that, and a study suggesting that one component of a food has shown carcinogenic effects (often only in an animal model) isn’t nearly enough evidence to justify a freakout over the food in general.

As another example, consider coffee, a beverage that Tom points to approvingly at the end of the second post. The roasting process produces a huge array of chemicals, not nearly all of which have been characterized and studied, and many of which have demonstrated carcinogenic effects (here’s a Wikipedia link, though like me you’ve probably already heard this elsewhere). But that doesn’t matter, because coffee is categorized under our semi-arbitrary “good/natural” heuristic, albeit perhaps with various qualifying words attached to it in our laughable mental medical tag cloud (parkinsons/osteoporosis/miscarriage/anemia/whatever).

Ignoring scale

Less relevant to this series of posts, admittedly, but a habitual mistake of this kind of writing. Humans are awful at understanding differences of even several orders of magnitude, and authors are even worse at bothering to convey them.  For our purposes, this is a problem on two fronts: both chemical concentrations and the scale of measured health effects. All you have to do is say “there’s uranium in your drinking water!” and people will start going nuts. You know what? There almost certainly is uranium in my drinking water, and yours too. There’s nasty stuff everywhere; you can’t avoid it. Things diffuse! Hell, cosmic rays transmute atoms! But until you quantify the scale and demonstrate the effect, these facts are meaningless. Bad things will get into your body, but usually in very small amounts, and your liver and kidneys will probably have no problem doing their usual kickass job of dealing with them (if the bad thing bioaccumulates, you should admittedly be less sanguine about this; but that’s not applicable to the compounds under discussion).

But this is not how the human mind works. You only have to invoke a substance and a sort of animistic power takes over. It’s homeopathy at an angle. It’s why we have airport security theater. Numbers mean so very little to our idiot monkey brains.

Pretending epidemiology hasn’t been invented

Perhaps I will be accused of a kind of libertarian placidity for this one. That would probably be fair. Still, I think that writers often lead with “we are all unwitting participants in a huge and unethical natural experiment!” and then decline to ask whether any data’s come in from this bit of mad science. And the fact is that it has. “It’s a mixed bag” is the answer that then invariably follows, but that’s sort of telling in itself, right? People have been consuming aspartame for a while now — some of them even doing so while participating in vast longitudinal studies — and it’s just not the case that it makes you start sprouting tumors or hemorrhaging spinal fluid or growing hermaphroditic genitalia. I’m all for getting BPA out of the lining of tins of baby food, but it’s pretty hard to make the case that any of the worrying maybes in Tom’s posts are actually manifesting themselves as health crises. Yes, someone needs to be manning the watchtower; yes, industry has incentives for bad behavior. But in this particular case I think the decades’ worth of rhetorical and scientific stalemate over sweeteners is enough to merit some shrugging and eye-rolling.

Diet sodas are not a serious health problem. They’re just not. It may be that avoiding them is a good idea at the margin — and like I said, I do exactly that, because hell, I don’t enjoy ’em anyway. But it’s hard for me to take myself very seriously as I do so. Sure, I don’t like the idea of methanol in my body, but I dislike it in almost exactly the same way that Scientologists don’t like the idea of thetans in theirs. It’s just a neurotic tic; an echo of a sort of Jungian archetype about impurity; really, a hobby at best. I mean, I still drink alcohol, and exercise less than I should, and go out in the sun a lot when I’m on vacation. Despite this disregard for my health I am still probably among the healthiest people in history, thanks to having enough money and living in America in the era that I do.

But look, perhaps you are healthier still. That is definitely possible! If so, by all means, freak out about aspartame if you want. It just strikes me as unlikely to deliver a very good return on the worry you intend to invest in it.

Like Ezra, I find this explanation for Dropbox’s success intellectually attractive.  Minimalism!  Of course!  It’s so simple!  It’s exactly the kind of thing that software developers get off on. For one thing, it legitimizes our lust for Apple products — despite the fact that Apple UI betrays the idea more often than we admit — because most of us can’t or don’t want to distinguish between good aesthetics and simplicity.  It’s sort of zen.  It makes us feel wise.  But in this case I don’t buy it.

Here’s what I think happened.  Amazon launched its S3 service.  Suddenly you could buy Big Storage services at essentially marginal cost, immediately.  No physical capital expenses, and human capital expenses were dramatically reduced (now you need a programmer who can work with AWS, but not someone who can source, install and manage huge RAID arrays in a datacenter somewhere).

That happened, and then a bunch of people tried to resell this service with some paper-thin rebranding/UI work.  Kevin Rose and Leah Culver with Pownce, and some Flickr clone with too many O’s in its name, and whatever, a bunch of people.  Dropbox was the first — as far as I know — to come up with a funded business model that could provide a useful amount of synced storage for free.

Also — and this is something that the Quora answer completely underplays —  Dropbox is quite technically sophisticated.  It’s not just rsync on a minute cron, you know.  It’s hooking into filesystem interrupts to notice when stuff changes in the synced folder, and doing it natively on every major OS.  It’s got quiet but powerful ways of dealing with versioning conflicts.  It’s also doing all of this with a high degree of polish (I mean: Growl notifications, c’mon).  Plus it’s smart enough to do things like notice when it needs to sync within a LAN instead of over the net, avoiding complexities you might not have considered like NAT traversal.  It’s not that it’s so simple; it’s actually a very sophisticated execution.  It’s just that those parts aren’t necessarily visible (and no, many of its competitors were not as clever).

Now, this is minimalism, in a sense.  But it’s not the sort of minimalism pointed to in the Quora answer, which amounts to “Let’s offer fewer features than those other jerks and we’ll all get rich!”  It’s more about doing things that are sophisticated and difficult, and not wasting time on UI afterthoughts.

It was 2006 when S3 launched, but a few years isn’t THAT long for a specialized market to shake out.  Besides, S3 prices have been falling since its launch, so it could’ve either been a lack of investment in the synced-storage space or just the need to wait for a cheaper equilibrium point that delayed the rise of a winner in the space.  At some point a critical mass was reached and brand recognition took over.

Anyway, Amazon’s contribution to web infrastructure is the key here.  Its transformative field-leveling effect on the industry (and the web’s increasing reliance on it) is a story that ought to be explored more in the popular press.  AWS deserves a bit more of the concern that Google commonly attracts for its market power.

Why aren’t there more open data startups?

I’m sorry, but no. It’s a lousy polemic. Here’s its structure:

1. SEO-friendly statement of controversy
2. Presentation of opinion A. Assertion that people who hold it are rubes.
3. Presentation of opinion B. Invocation of authority.
4. History lesson! Discussion of old technology; no mention of enforcement of author’s preferred orthodoxy by newer technology (e.g. HTML rendering multiple spaces as one)
5. Rumination on beauty. Grecian urns, etc.

For now let’s ignore the ignore the bullying nature of this argument (it should be obvious to anyone that those of us who believe in two spaces are a minority that’s relentlessly and mercilessly persecuted by the bloodthirsty masses, both through jeremiads like Manjoo’s and through the technological eradication of our ability to express our beliefs). Which of the points in the above argument are rhetorically meaningful?

Only point 3 really carries any weight with me. I’ll take Manjoo’s word that all typographers like a single space between sentences. I’m actually pretty sympathetic to arguments from authority, being the big-state-loving paternalist that I am. But, with apologies to friends and colleagues of mine who care passionately about this stuff, I lost my patience with the typographically-obsessed community when they started trying to get me to pay attention to which sans-serif fonts were being used anachronistically on Mad Men.

I love you guys, but you’re crazy. On questions of aesthetic preference there’s no particular reason that normal people should listen to a bunch of geeky obsessives who spend orders of magnitude more time on these issues than average. It’s like how you probably shouldn’t listen to me when I tell you not to use .doc files or that you might want to consider a digital audio player with Ogg Vorbis support. I strongly believe those things, but even I know they’re pointless and arbitrary for everyone who doesn’t consider “Save As…” an opportunity for political action.

Nor should we assume that just because typographers believe earnestly in the single space that their belief is held entirely in good faith. They’re drunk on the awesome power of their proportional fonts, and sure of the cosmic import of the minuscule kerning decisions that it is their lonely duty to make. Of course they don’t want lowly typists exercising their opinions about letter spacing. Those people aren’t qualified to have opinions!

(For what it’s worth, I don’t think you rabble should be using Flash or Silverlight or anything other than plain text in your emails. You can’t be trusted with it! And, not that this motivates me or typographers at all of course (we just want what’s best for you), but when you do such things it makes my job slightly harder.)

Manjoo’s argument about beauty, like all such arguments, is easy enough to dismiss: I disagree. I find it easier to read paragraphs that are composed of sentences separated by two spaces. Perhaps this is because I, like most technologists, spend most of my time working with (quite lovely!) fixed-width fonts for practical reasons. But there’s also a deeper beauty to the two space rule — a sort of mathematical beauty. Let me explain.

Consider the typical structure of writing. Letters are assembled into words, which turn into phrases, which are arranged into sentences — at the same time being assigned to speakers, a neat trick — which are then combined into paragraphs.

It’s a chemical process, a perfect and infinitely flexible hierarchical system that should command our admiration. Being able to rationally examine, disassemble and interrogate the final product is a mark of the system’s beauty. Anything less is settling for a sort of holistic mysticism.

It’s disrespectful to let writing’s constituent elements bleed into one another through imprecise demarcations. If you see me “making mistakes with comma placement”, please rest assured that I’m doing it deliberately. In most cases the comma doesn’t belong to the phrase delimited by the quotation marks that enclose it. Placing an exclamation point or question mark to the left or right of a close-quote is a weighty decision! That we violate the atomic purity of quotations with injected commas is an outrage.

And though I don’t get quite as worked up about it, the same sort of thinking motivates my belief in the double space. Sentences deserve to be clearly delineated, but because of the complications of quotation, ellipses, interrogatives and exclamations (among others), there is no reliable punctuation that can be counted on as a terminator for sentences. Single spaces are already spoken for: they separate words. The double space is an elegant and subtle solution.

To operationalize it: I can split any of the paragraphs in this post (as composed, not as rendered) into its constituent sentences with a simple line of Python:

“And though I don’t get quite as worked up about it, the same sort of thinking motivates my belief in the double space.”
“Sentences deserve to be clearly delineated, but because of the complications of quotation, ellipses, interrogatives and exclamations (among others), there is no reliable punctuation that can be counted on as a terminator for sentences.”
“Single spaces are already spoken for: they separate words.”
“The double space is an elegant and subtle solution.”
[/cc]

Further disassembly is easy from there. I can’t do that with the degenerate text that Manjoo prefers. As a journalist who makes his living on consumers’ pageviews it’s perhaps understandable that he would deliberately complicate news consumption for his non-human audience. But I hope the rest of us can make our aesthetic decisions a little less selfishly.

It’s a little pointless to speculate about where the Arizona shooter got some of his super-crazy ideas: presumably he will eventually be medicated and/or interviewed enough to simply tell us. Still, the press’s sudden fascination with David Wynn Miller strikes me as a little over-eager. There isn’t really much evidence tying him to Loughner. Yes, both men seem to have strange ideas about the government using grammar as a means of mind control. It seems perfectly possible — maybe likely — that Miller’s the source of that. But for what it’s worth, I think we probably shouldn’t rule out simultaneous, uh, discovery.

It’s just that Miller’s ravings aren’t as novel as they might seem. Consider timecube.com (it’s a lot sadder to read than it was a couple of days ago). In this case the author is convinced the government is exerting control through a conspiracy centered around the 12 hour clock. There’s also plenty of railing against evil educators, biblical allusions and specious logic.

In Gene Ray’s case it’s the clock, not grammar. But it could easily have been grammar. This kind of sickness seems more about obsessive and broken thought than conceptual synthesis. Whatever bibliography we generate is going to be meaningless, and it seems a bit ludicrous to be playing name-your-influences as if Loughner had just released a debut album.

I’ll keep my ill-advised and emotional reactions to the horror in Arizona confined to Twitter for now (though I should say that I’m quite certain I have a unique and trenchant perspective on the matter, having had a brief and superficial conversation about gun control with a cab driver in Phoenix last month). Let’s talk about something totally different!

I just stumbled across a video that explains my favorite experimental neuroscience result pretty well:

Libet’s work isn’t the conclusive case against free will that it might seem at first blush. Neural evidence of motor planning can occur without resulting action; such planning may not occur in the same way for different types of decisions; and you can come up with various free-will-as-last-minute-veto accounts that strike me as pretty lame but seem to make some people feel better. You don’t have to accept conscious experience as an epiphenomenon. Still, I’ve always found the result a great means of attaining a mild but pervasive sense of existential worry, which I think is worthwhile on its own merits.

I was prompted (through a clear chain of causation!) to search for the video after reading this (PDF), via Trivium. It’s short and worth a look. It descends into improbable and grandiose flourishes after the first column (people like me would keep a Predictor on their keychain and then get on with their lives). But with all the excitement over Daryl Bem’s paper that’s arisen during the last few months, I’m feeling pretty good about my odds of being able to buy a Predictor-like gadget from DealExtreme in the next decade or two.

It was good to read this (via the great Justin Miller). In it, Patton Oswalt laments the cooption of geek culture by the masses. He lovingly describes his youth as a Northern Virginian Otaku, and laments that culture’s adoption by the mainstream.

I’m sympathetic, but Oswalt is totally wrong. It’s not a problem that jocks like Watchmen or Buffy or whatever else. It’s great that they do. If something has artistic value, it should be enjoyed by as many people as possible.

I get that this stuff feels like a life raft to teenagers in legitimate pain; that it provides a sense of identity for outcasts. I was one of those kids. But I’ve reluctantly concluded that it’s not healthy to create these islands of identity. Imbuing pop culture with value, then borrowing that value for yourself — it’s not a winning formula. If kids who can throw footballs know who Boba Fett is, and if that cuts the legs out from under some aspiring introverts… I guess I think that that’s a good thing, even if it’s painful for them in the moment.

Oswalt’s lamenting the destruction of the nerd ghetto, and proposing that we build a new one. That’s an understandable impulse, but it’s wrong. Undercutting the drive to self-victimize can appear cruel when considered in the context of young people’s deliberate and inevitable victimization of one another. But the alternative isn’t really mercy; it’s just assisted masochism.

(A note: I wrote this late at night, then pulled it down the next morning until I could reconsider it. Seems fine now (1/8/2011), so back up it goes.)

I’m a bit late to this, but for the record: it’s extremely disappointing to see the most important cyclist advocacy organization in the city concern trolling its own members.  As others have noted at length, the proposed pledge to more lawful behavior is both insulting and potentially unwise: complete and mechanical obedience to the rules of the road can be dangerous in its own right.  I’d say I’m pretty scrupulous about following the law — dorky hand signals and everything! — but if you think I’m going to hang out in a left turn lane facing a red light but no oncoming traffic, waiting for someone to plow into me from behind, you’re nuts.

I get pretty upset when I see someone salmoning up a one-way street or failing to yield the right of way, but the actual incidence of serious cyclist misbehavior seems to me to be relatively small.  In the same way that many motorists habitually execute rolling stops, or fail to signal, or go five miles over the speed limit, cyclists commit a lot of offenses that don’t represent serious safety problems.  No one spends much time bemoaning drivers’ peccadilloes because we’re used to them.  But first-hand experience of biking through the city is still pretty rare. It’s unfortunate, but I think that without such experience, drivers are unqualified to weigh in on what constitutes dangerous behavior, and consequently overstate the problem of cyclist recklessness to an astounding degree.  In perfect world everyone would make at least a few commutes using each mode (and in the presence of users of other modes) before engaging with this topic.

Motorists often mistake their own surprise for evidence of a safety problem.  This is understandable; we should all resolve to surprise each other as little as possible on the road.  But sometimes a driver’s surprise represents proof of a cyclist’s erratic behavior, and sometimes it’s just evidence of the driver’s lack of experience around bikes.

I think folks behind the wheel are used to having responsibility vested with them.  As is often repeated, they’re in control of deadly weapons.  A burden of responsibility is understandably placed on them, rather than pedestrians.  Unfortunately, the speed and vulnerability of cyclists combined with drivers’ inexperience around them makes this two-party arrangement untenable.  Cyclists have to adopt a patronizing attitude toward people in cars: to declare themselves the judges of how the street should be navigated. We’re the most vulnerable; no one else is willing to take charge of keeping us safe.  If we decline that responsibility, it’s our own asses.

This isn’t an ideal state of affairs.  The burden should be shared, and codified by law. And it really can be that way!  I rode in the Netherlands this fall; that’s how things work there. But that experience was both inspiring and depressing: believe me when I say that we are nowhere near ready to rely wholly on the rules of the road in the way that American drivers and all of the Dutch are able to (the only time I have been hit by a car as an adult, I was completely in compliance with the law; the driver simply didn’t expect a bike to be there).  The current situation will obtain in most American cities for the foreseeable future.

Besides, sharing the road is a pain in the ass.  That’s just the truth: it would be a lot more convenient for me if there weren’t any cars in the street, and it would be a lot more convenient for the drivers of those cars if I walked to work.  That’s just how it goes.  Some folks in cars are never going to be happy that I’m on the street, on a bike.

Safely navigating the city on a bike and keeping drivers happy are goals that are somewhat in tension.  Trying to meet drivers’ motivated and at times incoherent expectations is a losing proposition.  Changing those expectations is the only way forward, and the only way to do that is to get more bikes on the road.  I wish WABA’s staff was focused on that task instead of spending time scolding the people they’re supposed to be representing.

I had some interesting Twitter conversations this morning about American Airlines’ departure from Orbitz. Various sites seem to be pissed off at AA for this, presumably because they interpret any additional consumer inconvenience as the likely result of a greedy corporate cash grab. That’s not a terrible rule of thumb, but in this case the situation is a little more complicated.

On the one hand, you’ve got online ticket agents (OTAs) like Orbitz, Kayak and the like. On the other you’ve got the airlines. In between are the global distribution systems (GDSes). These are the centralized reservation databases that are used by airline staff when they have to book you a new flight on another airline; they’re the things that were used by travel agents back when travel agents still existed. The OTAs use them, too — though the situation here gets a little bit complicated, as some of the OTAs are in bed with and/or wholly owned by the GDSes. GDSes aren’t just price aggregator: they do various complicated things like hold tickets for you while you’re in the process of checking out through a travel site. They’re sort of like stock exchanges for airline tickets.

The key thing to realize is this: the GDSes are old systems, and represent something that is at least a bit cartel-like. They introduce friction; they charge fees. They’re middlemen who serve other middlemen. If we can manage it, eliminating our collective reliance on them would be a good idea. Disintermediation should benefit the consumer!

As I understand it, American Airlines is asking the travel search engines to talk to AA’s systems directly instead of going through a GDS. Perhaps this is a temporary tactic that’s being used to put pressure on a particular GDS as AA renegotiates the terms of their contract. Perhaps it’s an earnest effort to begin tearing down the overcomplicated system currently governing airline booking. I don’t know. What I do know is that AA is getting a pretty raw deal in the media. I find it astounding that any reporter could be so credulous as to write this story. Expedia’s offering moral support to Orbitz? Right. Sure.

So look, I don’t know how much goodwill AA deserves in this situation. But all that they’re asking Orbitz to do is talk to their systems directly. This would involve some technical work by Orbitz, but would allow for the elimination of a Ticketmaster-like entity that’s currently involved in the purchase of air travel. That could potentially mean lower prices. What they’re not doing is saying, “Sorry consumer, I don’t care about the extra hassle I’m causing you; you’re just going to have to search for and buy tickets straight from our own site so that I can avoid paying any helpful search engines a booking fee.” (This, incidentally, is exactly what Southwest, jetBlue and various other discount airlines used to/still do, but for some reason it didn’t stop anyone from loving them). If they were doing that, I’d be a lot more sympathetic to the hue and cry that’s being kicked up. As it is, this is just a business flexing its muscle to get out from under a gatekeeper’s thumb. I’m all for it.

I’ve been following the “Do Not Track” discussion with interest. I’m not sure I have any particularly strong feelings about the proposed implementations, but I’m curious to see what happens. And of course it’s been hilarious, as always, to see some of the knee-jerk pro-corporate responses that the policy proposal has prompted.

There’s an aspect of the debate that I briefly tweeted at Harlan Yu, and I guess it might be worth expanding on a bit. Consider an excerpt from this post over at TLF:

The FTC’s report calls for a “uniform and comprehensive” way for consumers to decide whether they want their activities tracked. The Commission points to a Do Not Track system consisting of browser settings that would be respected by web tracking services. A user could select one setting in Firefox, for example, to opt out of all tracking online. The FTC wrongly calls this “universal choice.”

Really, it’s a universal response. It’s a single response to an overly-simplified set of choices we encounter on the web. This single response means that tracking for the purpose of tailored advertising is either “on” or “off.” There is no middle setting. But it is the “middle” where we want consumers to be. The middle setting would represent an educated setting where consumers understand the tradeoffs of interest-based advertising – in return for tracking your preferences and using them to target ads to you, you get free content/services. But an on/off switch is too blunt and not, err, targeted enough. There is no incentive for consumers to learn about the positives, they’ll only fear the worst-case scenarios and will opt-out. In return they’ll also opt-out of the benefits. [more on the “middle” below].

(Aside: can we please stop with the transparent attempts to frame the debate in a self-serving way through use of the word “nuclear”? Everyone knows what you’re doing, dude.)

Let me quibble with the free content/services bit embedded here. Implicit in this assertion is the idea that advertisers’ ability to possess better information about consumer behavior allows ads to be more precisely targeted, making the ads more valuable to the businesses that place them. This increases the revenue available to ad-supported industries, many of which provide valuable content or services through what is a de facto tax on businesses.

I think the evidence for this dynamic is weaker than a lot of people suspect. As far as I can tell, it’s all based on Google. GOOG showed up and provided contextualized ads to consumers and a model that allowed advertisers to only pay for purchases that were “working”. This is pretty much the only way they make money, and they make a lot of it.

But Google’s a huge success in a landscape of failure. Online ads sell for pathetic rates relative to broadcast or print. This is because by all accounts online advertising doesn’t work very well. You can measure whether someone clicks on an ad, and often whether they buy something after that click. But it turns out they rarely do those things. So businesses aren’t willing to pay very much for ad space on websites.

Is it really a coincidence that the advertising medium with the best instrumentation also appears to be the least effective? I suspect it’s not. It may be that ads never worked as well as the industry had told us; or it may be that the eyeballs/clicks/conversions funnel is a naive conceptualization of how the system works. Either way, Google has succeeded by giving advertisers what they think they want, which is analytic tools that seem to reveal that the whole enterprise is horribly ineffective.

I think the push for better tools and more efficient ads is basically a race to the bottom. In fact, less perfect instrumentation might allow the ad industry to capture a bit more revenue from business thanks to decreased efficiency. If I’m right, those in the business of selling ads should be excited about initiatives like Do Not Track. I don’t think it’s conceivable that the market will cease pursuing greater efficiencies unless this kind of regulatory intervention occurs.

It might sound a little strange to hear me sign off on the government propping up an industry I detest. Honestly, advertising is a big reason I find myself on the liberal side of the corporate/state distrust spectrum. There aren’t many things I find more odious than cynical attempts to manipulate people’s thoughts and desires — especially when it’s done using quirks of biology and psychology. Our anti-propaganda norms make it quite clear that we don’t tolerate such manipulation from government; when examples of it are revealed, they’re harshly criticized and punished. Yet we accept as perfectly normal the idea of such manipulations being pursued by private entities, who are free to try to cajole us into doing things contrary to our own interests, and without any mechanisms for controlling them other than the adeliberate market and (viciously resisted) regulation.

It’s a bit weird. Distressingly weird! So I think preserving personal privacy is probably worth the introduction of this kind of inefficiency. And I think the preservation of the media creation business is desirable, and I don’t really believe in any models for accomplishing this except some sort of (probably convoluted) public subsidy. If I’m right, decreasing the efficiency of advertising will act as a de facto tax on businesses which choose to advertise. We’ll consequently pay slightly more for products sold by such businesses, and the difference will help pay for our magazines and newspapers and web videos and music. I think that’d be an okay deal.

I should have written this a while ago, but: those few of you who visit here but don’t visit sunlightlabs.com — please consider that latter decision. A lot of my writing winds up there these days, including stuff on some issues that readers of this site might actually care about.

Oh, also: @sunlightlabs. C’mon, you can afford to follow one more account.

Earlier tonight I tweeted something snide about “Big Data.”  Some people actually replied!  This was a useful reminder that Twitter is not just a black hole into which bile can be thrown.  It also made me realize that I should probably explain my thinking a bit more.

At this point, I consider software development to be a trade.  It’s easy to pretend otherwise, what with so many of us having degrees in computer “science” and the fact — often only apparent to practitioners — that a program’s implementation can reflect an aesthetic sensibility.  But in truth those of us who work in the industry perform tasks that are a lot more like the work of an electrician or plumber than that of a sculptor or theoretical physicist. We apply a relatively small number of solutions to an almost infinite landscape of problems.  There’s still plenty of room for variation and artistry in the execution — there’s more than one way to skin a cat — but it really is about deploying specialized tools to solve applied engineering problems.

Ted Dziuba once wrote something like “scalability is the problem that every software engineer wishes he had, and ten actually do.”  There’s something to this, but it’s a bit unfair to the people who wish they had the problem.   A run-of-the-mill web developer studying scalability is a bit like a construction carpenter studying Amish furniture making techniques.  Is it relevant to his work?  Well, strictly speaking, no.  He just needs to be able to tack up frames so the drywall guys can come by.  Still, if you’re choosing which carpenter to hire to help build a townhouse, it might not be a bad idea to pick the guy who’s interested in learning about dovetail joints and how to build a dresser without using any nails or screws.  To be clear: this is not where the guy’s talents are actually needed.  He could be rebuilding houses in Haiti, or New Orleans, or wherever.  There are plenty of problems that need his skills.  Still, there’s nothing wrong with a guy taking pride and interest in his craft.

And that’s about where we are with software developers interested in scalability, or Big Data, or whatever.  Interest in esoteric topics is healthy!  It shows that a person is engaged and anxious to learn.  A very few of those people actually will need to build dovetail joints someday.  And yeah, they could probably just learn once the need arises.  But it’s not a particularly grievous misallocation of resources for them to learn now instead of later.

But things get a little dodgier once you get beyond this circle of technologist self-betterment.  It is not particularly helpful, for instance, to start writing news stories about how the future of carpentry is the phase-out of screws, and how in the future no firm that hasn’t prepared itself and its employees for the screwless economy is going to be able to compete.  It’s just not true.  Some smart people are interested in this stuff, but that doesn’t mean it’s about to transform the industry.

So, look: Big Data.  Do some people genuinely deal with this class of problem?  Yes, some do.  Many more, myself included, wish they did: I get as excited as anybody else by the idea of marshaling huge quantities of information to reach interesting conclusions!  The problem here is really just the “big” part.  Computers are so powerful.  How big does a problem need to be before a single machine is an impractical solution?  The answer is: very, very, VERY big.  For instance, I’ve got the last ten years of federal spending data on the laptop on which I’m typing this.  That’s about 31 million rows of data.  That’s a lot!  But it’s still nowhere near the point where I’d need to pursue exotic optimizations for my queries.  Most startups do not do as much business as the federal government; and most well-designed systems don’t need to constantly query a huge working set of data.  I can’t help but wonder whether we’d be better off spending more time teaching each other how to run regressions, or just calculate a confidence interval (how big do you need that data to be before you can arrive at a decent answer, anyway?).

But this is just my own sense of the matter.  The fact is that I don’t do a ton of work in this area.  Perhaps I’m way off-base here!  But I kind of get the sense that engineers are getting excited about interesting things, and others are concluding that those interesting things are broadly important things.  It’s not always so.

Halloween is kind of a big deal for me. This year I was sad not to be able to throw an enormous party, but it did give me an opportunity to put some more thought into my costume than usual.

I always go as something comic-book-related; typically I go as a super-villain (I broke that tradition last year and sort of regretted doing so). In the past I’ve chosen characters based on my personal affinity for them and a balance between the feasibility and potential awesomeness of the costume. I kind of ignored that this time. This year I went as Dr. Manhattan. Like every other self-respecting geek, I love Watchmen, and there were some personal resonances that made me want to go as doc.

I’m supposed to be dressed the way Dr. Manhattan was when he made a TV appearance in the comic. There isn’t anything particularly important about that scene — it’s just the only one where he isn’t naked (I wasn’t that committed to the project).

I think people tend to have a confused reaction to Dr. Manhattan when reading the book. It’s very tempting to view him as a sympathetic character. For one thing, he’s omnipotent, and people naturally find themselves sucking up to all-powerful authority figures. Also, his backstory does make him something of a victim. Finally, he’s helpless — in a sense — and people mistake helplessness for innocence. But this is a bad judgment. In truth the guy’s actions are abhorrent both before and after his self-imposed exile. So I feel comfortable including him in the “villain” costume list.

At any rate, the costume was well-received, though most often as “blue man” or “blue man group” (on the way home a woman yelled out to me that her brother-in-law is an actual blue man group blue man, an admission that I think left us both embarrassed). Choosing a costume that’s so thoroughly makeup-based is definitely a pain in the ass — I think it’ll be a few years before I go down that road again. And to be honest, people mostly just liked the contact lenses: they worked great, thoroughly disturbed people, and seem not to have blinded me. And hey! At a mere $25, I think they’re well worth the investment. A+++, would stick into my eyes again.

It’s a very good movie. You should go see it.  I don’t have much more to say about it than that.

What I do want to observe, though, is that the reaction to the movie on Twitter has been fascinating.  A surprising number of people are leaping to Zuckerberg’s defense.  Many are linking to this account of the legal maneuvering dramatized by the film — an account which doesn’t actually exculpate Zuckerberg, but is written as if the author thinks it does.  Even more astounding are posts like this one, which lionize Zuckerberg as a sort of Nietzschean hero.

It’s all a little weird.  People clearly have reservoirs of affection for and identification with the founder of Facebook. Presumably this is because of what his invention means to them.  I suppose that’s fine.

But let me offer my predictable cranky technologist perspective.  Zuckerberg is wildly successful, and I don’t begrudge him that.  Nor do I find myself particularly outraged on behalf of the various people he seems to have betrayed, all of whom are now even more astoundingly wealthy than they started out being.

There’s a strange tendency in this country to declare everyone who’s extremely rich to be some sort of visionary genius, whose unprecedented perspective allowed a break from the chains of history.  This is ridiculous.  Mark Zuckerberg did not invent social networking.  His achievement at Harvard wasn’t even that remarkable: 400 unique visitors to a just-launched website is great, but hardly unprecedented.  Nor are his technological achievements all that impressive: assuming the movie’s depiction to be accurate — and I suspect it basically is, judging by the tasks he was performing — he seems like a strong web developer, but not any kind of wunderkind (Facebook’s ability to operate at its current scale using a hugely rewritten version of PHP is a genuinely impressive achievement, but the entry-level LAMP stuff Zuckerberg himself did is simple enough).

What Facebook did was deploy an extremely effective marketing scheme — one based on exclusivity — at exactly the right  moment.  From there, network effects took over and delivered success.  This is an impressive commercial achievement, but it is not a triumph of vision or technical skill.  Depending on your perspective, you might not even consider it to be particularly admirable.

In the absence of Zuckerberg, someone else would have almost certainly come along and stolen Myspace’s thunder — just as someone or something will surely supplant Facebook before its ludicrous valuation is realized.  Zuckerberg is not one of history’s great men — he’s just one of its richest.