Steph and I have been AirBnBing our guest bedroom for about a year and a half now, and it’s been mostly great. It’s not a ton of extra money, but it’s not an insignificant amount, either. We occasionally meet interesting people. We are regularly reminded that the world is full of people unlike ourselves, and that it’s possible to get along with them, at least when the interaction is supervised by an app’s influential reputation system.

And it seems like it’s probably good for us. I love my job dearly, but it does involve a lot of playing with computers and eating free snacks. Spending a few hours a month scrubbing toilets for money makes me feel like I might have at least a slim chance of surviving the revolution.

The aforementioned reputation system does a good job of driving nearly everyone within it toward neurotic approval-seeking. You are not just supplying a service for money; you are also hoping your guests will have a good time and compliment your home and be impressed with your lifestyle. They are not just paying for lodging; they are trying not to be a bother and giving you coffee from their home country and making implausible offers to put you up if you ever visit COUNTRY/CITY X. It’s all a little bit charming and let’s be honest more than a little pathetic, but the five-star ratings really do seem to weed out the sociopaths. It works pretty well.

Despite their business being built on overeager social fumbling, AirBnB is taking steps to make interactions more impersonal through the use of their Instant Book feature, by which hosts relinquish their ability to approve requests from prospective guests one by one. They have both good and bad reasons for this. Good, in that there is strong evidence of racial and no doubt other forms of iniquity in how hosts accept guests. Bad, in that they were pushing Instant Book well before the bias conversation, presumably because lowering coordination costs leads to more transactions and revenue.

I am in favor of systematizing away opportunities for human odiousness, but Steph and I are unlikely to enable Instant Book. In our experience, the worst AirBnB interactions occur when expectations are mismatched. A first-time user with poor English expects to get a full condo, rather than just a room, for eighty bucks a night. A high school kid from suburban Maryland expects to stay at our place with his girlfriend. A couple of friends expect a third to maybe join their weekend trip on a whim and crash on our living room couch. A new graduate’s parents have never been out of China before and expect to order room service. Or maybe Steph and I just have an inkling of plans for a week we haven’t thought to block off yet. Reviewing potential guests allowed us to avoid several of these catastrophes. Plus the silly letter the guest has to compose gets the stilted courtship described above off on the right foot.

Most guests are nice but boring folks. They typically hole up in the room they’ve let and do their thing without much bother. We have learned to identify risk factors for those who will spend a bunch of time at home (when they do this you have to pretend not to mind):

  • Those at the end of very long trips will be exhausted and will just want to surf the web rather than continuing to fake enthusiasm for seeing cultural treasures or whatever.
  • The young will be too poor to spend time outside of the house except for whatever their Big Planned Event is. Also their blase attitude toward their creeping mortality makes them feel like they have all the time in the world to visit the National Archives someday, but you know what, young man, you don’t.
  • The old, particularly those who have clearly been dragged along by their spouses, might just want to watch TV, which in our household often requires moderate-to-advanced skill at TCP/IP network configuration. We try our best but some of them don’t even know how to run a traceroute.

We have been charmed by several guests. There was the Parisian atmospheric scientist who flew planes to take air samples, who Steph still pines for. There was the bartender who was back to take the bar for the third time after missing the date for the one near his home in New York City. There was the Eastern European flight attendant who met his internet language-learning community girlfriend for the first time in our house on Valentine’s Day, followed her home to Southeastern Virginia, proposed, was not allowed to meet her family, flew home without a ring or engagement, and was still so polite as to respond to our thirsty AirBnB requests for details about how it all went. I think fondly of all of them.

But if I were to offer only one piece of advice about how to charm your AirBnB host it would have to be this: try not to leave too much hair behind. Honestly, this is all I care about. Maybe avoid putting wet garbage in the bin, too, so I can reuse the liner after I empty it. Otherwise, go nuts. My dream guests are an aging and profoundly bald gay couple. No stray hairs, in bed at a reasonable hour, and maybe they can tell us some good stories over a glass of wine. God bless them.

If you are considering AirBnB hosting, I have a longer list of suggestions:

  • Consider the cost of room turnover and the cost of guests letting their guard down and beginning to transgress boundaries. For us this means bookings of 3-5 nights, which is a narrow window that probably hurts earnings but minimizes stress.
  • Use AirBnB’s free photography service. It makes your place look great, though it can take a while to get an appointment and our listing makes it look like we have a Christmas tree up year-round.
  • Buy some makeup removal wipes and leave them in a prominent spot, or else you will eventually wind up with some destroyed towels. The first time you see a guest heading out for the evening in unexpectedly high heels you should immediately buy another towel set on Amazon.
  • Get some programmable door locks or a key lock box, obviously. Alternately, prepare yourself to get upset over guests not respecting your arrival policies. You will be irate but they will not understand the problem at all, like a dog getting yelled at for drinking out of the toilet. It’s water, isn’t it? I don’t see what the big deal is.
  • Ask existing hosts for testimonials to get over the “is this new listing a scam that will steal my kidneys” hump.
  • If your guest shower has a hose sprayer, great, bully for you. If not, get one of these things. It will make your bathroom cleaning so, so much faster. A little handheld vacuum is also a substantial time-saver.
  • It’s maybe overkill but we installed some fan timers and USB outlets to encourage non-moldy bathrooms and fewer questions about borrowing chargers.
  • We’ve found that guests leave reviews about how much they like having a good neighborhood guide, a comfy bed, immaculately clean bathrooms and an electric kettle in the room. They don’t seem to care about the cheap Trader Joe’s wine we gave away for a while. I guess we weren’t fooling them.

Sometimes we are attentive, sometimes we are busy, sometimes we are maybe even a little surly. The guests don’t seem to notice. The AirBnB review is by now a well-understood compositional form. Give them a fair deal and they will write you one, like a traveling bard composing a song. You’ll meet some strangers and it’ll basically be okay, sometimes a little worse but often better than you’d think.

black hat and def con

A week ago I was in Las Vegas, surrounded by thousands of people attending Black Hat and DEF CON, the computer security industry’s two most important US gatherings. I had never been, and I loved it. I loved it.

What could be more hyperbolic than an army of hackers building a hive within Las Vegas, a place that is already a cartoonish, predatory and vacuum-sealed daydream? We have all watched decades of hacker movies and shitty quasi-martial CBS procedurals where geeks have piercings and weird hair and (when in front of a keyboard) superpowers. My expectations had been set by portrayals I knew to be bad. Surely the reality would be different, which is a euphemism adults use for “disappointing”.

But it wasn’t. It was a wonderland. Those production designers did a better job than I thought. And the superpowers are real, albeit demanding of much more time and tedious scrutiny of IDA Pro output than can comfortably fit in three acts and 46 minutes. But I’m getting ahead of myself.

I did not understand the difference between Black Hat and DEF CON before going. Black Hat is more professional, I had heard. DEF CON is cheaper. A lot of the same people go to both. All of this is true, but it’s a shallow explanation.

Black Hat is an expertly-produced conference for security professionals, a category that includes researchers, software vendors and surely more than a few cybercriminals. Its escalatingly expensive tiers of access allow participation in the frenzied vendor hall; attendance at the “briefings” where new and sometimes dangerous research is presented; and “classes” where specific attacks are taught–to use a not entirely hypothetical example, a fun class exercise might involve bringing Iran’s IP address block to its knees for an hour or two.

I was surrounded by khaki pants and polo shirts and a pervasive air of menace. My most vivid memory of the conference is among my first: walking into the keynote, an impossibly vast space even by Vegas’s improbable standards. Far above me the ventilation system exhaled a low, cavernous breath. The only light came from the projection screens that relayed the distant speaker’s image, bright like windows in a spaceship orbiting a blue-white star. By that light I could make out the silhouettes of the audience, thousands and thousands of them sitting in shadow, waiting to learn something new and frightening. You don’t take photos of other people at Black Hat, but in a room like that it would be hard, anyway: they’re not even lit by their own screens. Bringing a computer into such a hostile network environment is considered by most not to be worth the risk. I’ve never seen fewer computers in a conference audience than at this computer security conference. Everyone there learned long ago that if you are in the business of having prey you are also in the business of being still and unnoticed.

It freaked me out. Black Hat’s lightning talk track is called “Arsenal”; on Twitter, observers complained about talks that didn’t come with working code. This is not an academic conference.

DEF CON begins as Black Hat ends, and it is fundamentally a social gathering. The Black Hat attendees slip into a less stony kind of crypsis, cheerfully blending in with people whose skills and interests are similar to theirs but who possess vastly less discipline. Things become a lot more fun. $240 — cash only — at the considerably seedier Bally’s got me a skull-shaped electronic entry badge with ciphertext printed across it, connection pins that would whisper more secrets if hooked to the right hardware, and absolutely no instructions. If you solve the puzzles built into it and the lanyards and the conference signs and the schedule booklet and a hidden subdirectory on the welcome CD-ROM and who knows what else, you win free entry for life.

This was just one of many badges. I stood in line to get a kit that let me build a different badge, this one associated with an annual cyberpunk role-playing game. I soldered it together at the hardware hacking village’s free workstations; afterward I could use its infrared LEDs to trade handles with other players, then send them radio messages. There were badges that could connect to your car’s diagnostic network, badges for the LGBTQ and women-focused subcons, and badges made by groups of friends where you just had to know a guy to get one. They blinked LEDs, and paired with each other, and had secret accessories.

The badges are only a small part of the con, though. In a little over 48 hours I learned about beating airport wifi portals, lockpicking, dumping and decompiling router firmware, messing with tamper-evident seals and hacking Italian parking meters. The vendor hall was full of booths devoted to hawking t-shirts rather than to collecting CISO emails; I bought gadgets to let me spy on Bluetooth network traffic and a USB thumbdrive that injects exploit code at a thousand characters per second. Around midnight on Friday I found myself in safety glasses, a nylon net cage, a 26th floor suite, and a state of considerable intoxication, all at once, as I prepared to pilot a drone against three other people vying to be the first to pop a balloon at the cage’s center. Across the hall you could get an RFID chip implanted in your hand for $60.

I was jealous of the kids whose parents had brought them. They were finding this place so early. There is a certain type of person with certain types of interests and this is where they occasionally gather, like migratory animals, for those rare experiences unmediated by LCD screens.

My ebullience at finding so many people of the same type as myself powered me through most of the weekend. But at some point I had to start asking myself what type of person that was, exactly.

On Saturday my badge was malfunctioning. It wouldn’t blink the way it was supposed to. I had no relevant tools to tinker with, and was eventually referred up to one group’s suite for help. There an exhausted man behind a mountain of soldering equipment and several empty Surge tallboys was explaining, to a semicircle of my fellow newbies, how he had designed and thought about the group’s badge. No, the group wouldn’t accept sponsorship, he said. They paid for it out of their own pockets. To get a badge you had to be cool, that was it. They never had as many to give away as they would like. He didn’t want to see them in the hands of certain people, people whose names I had never heard of but which he practically spat. No, this wasn’t his day job, he–he trailed off. “I don’t make a lot of money,” he said, breaking eye contact.

He was generous with me, and after attending to a long line of questions and requests for aid that had arrived ahead of me, he performed a simple diagnostic test and declared, reasonably, that the repairs I needed were beyond the tools he had present.

It was about five minutes into our interaction before I realized he was open-carrying a glock on his hip. That’s the kind that doesn’t have a safety. This made me pretty uncomfortable, but of course I didn’t say anything. I was his guest, in his and his friends’ suite. I thanked him earnestly and honestly and left to find someone else to help me with my badge.

That gun was jarring to me, but maybe it shouldn’t have been. What are all of these exploits, these network packet captures, these lockpicking sessions about, anyway, if not having power over others? All of us there wanted secret knowledge to make ourselves stronger. For some it’s because we can’t forget being weak before. For others it’s probably something uglier. But it’s hard for me to imagine any of it being born of an impulse that’s particularly noble.

At best, it can be enough to know when you have gained that strength, to limit its expression to sly winks and low-grade mischief shared with the similarly afflicted. That’s DEF CON. If you begin using that new strength for a living, or find yourself forced to face the people who do–Black Hat. Worse things, if you ask me.

Well, I might find its emotional foundation suspect, but DEF CON was a hell of a lot of fun, and I will be back. Eventually, I’d like to better understand why I want to go so badly. But my short-term goal is just to get faster with my new lockpicks.

black hat sessions I have attended, translated into their wizarding equivalents

introductory OpenStreetMap politics

None of the following will be of much interest to people already familiar with OSM, except perhaps as an opportunity to take offense. The internet has plenty of those; I encourage you to trust me when I say that I mean no disrespect. I offer this both to satisfy my own commitment to write more and because I’ve recently been thinking that saying things that seem obvious can be important. Sometimes they turn out not to be that obvious after all.

Mike Migurski has a characteristically thoughtful post about OpenStreetMap and how he feels the community must adapt to be more welcoming of automated edits (“robot mappers”) and communities focused on improving the map for emergency response (“crisis mappers”). He characterizes the interests of these communities as being at odds with the original participants in OpenStreetMap, which he names “craft mappers”.

In comments, several OSM doyens object to this characterization even while betraying some of its truth: Frederik Ramm cops to the project’s import as a social mechanism, and Richard Fairhurst (who I hasten to add seems like he might be the most reasonable person ever to use the internet) extolls the project as a vehicle for personal expression and empowerment.

One has to admit that this is a bit of stacked deck. Crisis mapping is enormously important but it’s also a rhetorical atom bomb. Saving lives is always going to be more important than preserving the hobby of a few map obsessives.

So let me complicate Mike’s typology by adding one more constituency: passive users of OpenStreetMap data. Naturally I am thinking of Mapbox customers, but also people using MapQuest and Mapzen and Carto and and countless other businesses. It’s not entirely clear to me how much the earliest mappers of OSM care about their efforts finding use, or whether they are content to build a beautiful scale model of the world. The project’s relative inattention to building consumer-friendly services has always seemed to me an intelligently-chosen strategy of allowing the competitive market to handle distributing the project’s achievements. But I suppose indifference is an equally plausible explanation.

Still, I don’t think there can be any doubt that an enormous number of people benefit from OSM as mediated by commercial entities like Mapbox. Anyone who owns a smartphone can understand the growing importance of geodata. Finding the nearest coffeeshop is a modest benefit compared to being pulled from a collapsed building by rescue workers, but it happens considerably more often. It’s reasonable to be wary of equating profit with good, but commerce generally does indicate that someone is having their needs or wants satisfied. And there’s plenty of money in maps.

This community has no meaningful franchise within OpenStreetMap, and is instead represented by the commercial actors who serve them (:waves:). Those actors command both the resources and resentment you might expect of profiteers in a volunteer community: valued for the contributions of effort, software and money that professionalization affords, but viewed with understandable suspicion as to motives. We can do a lot of things, but people are quick to assume that we are doing them for bad reasons.

It is not a very democratic circumstance, but no one claimed otherwise: OpenStreetMap is often referred to as a “do-ocracy”, implying that those who contribute work have outsize influence. Fair enough, but this means that when the work that must be done includes discussion, conflict that amounts to a veto is the overwhelmingly likely outcome.

And that means that stasis is the order of the day. A slowly growing map, best in places where people have enough time and money to support a particular type of eccentric hobbyist. Across from them, another group of professionals, this one anxious to build the map everywhere, and quickly, before the next earthquake or funding round.

Everyone involved is as earnest and passionate as you might imagine, but there are probably only a few hundred of us bothering to write heartfelt blog posts and send snarky tweets from conference sessions. Oblivious to all of this are the overwhelming majority of the tens of thousands of active mappers; the tens of millions of people using OSM data without knowing it; and the billions of people who could be safer, or richer, or freer if OpenStreetMap or a project like it became the understood commons where we map our shared world.

I think mapping is a great and interesting problem, but it’s my job, not my hobby. Perhaps this makes it too easy for me to follow my utilitarian open data beliefs and say that OSM should be built as quickly and unselfishly as possible. But there it is.

geocoding your way to a great vacation

The road had descended steeply into the valley, and if there was a river anywhere near, it seemed like we would find it soon. There were even a few signs for a dock, but by that time we were suspicious. The people of Skradin would have you believe that their great civic passtime is standing on street corners, wearing t-shirts emblazoned with the letter P and proclaiming the availability of free parking for ferry passengers. It seemed suspicious. Besides, Google told us the ferry was over one more mountain. Nice try, Skradinze.

Our rented VW diesel leapt up the incline, no doubt leaving unimaginable pollutants in its wake. At the bottom of the hill a construction worker made a sort of :no_good: gesture but maybe he was communicating with a colleague behind us? We proceeded onward toward the pin. The road became one way, not too unusual for this part of the world. Some trees were growing into the road; I rolled up my window. Soon the gap became so small that we worried about our mirrors. But the pin urged us on.

Eventually we reached the end of the alley. The walls had closed in. There were a couple of driveways headed up to estates on the hill to our left, so steep that a scooter might tip backward. Backing out looked terrifying: to our right, footpaths ramped down from, then parallel along the road. With no guardrails in evidence, a misplaced wheel could fall into several feet of empty air.

I was panicking, but Steph got out of the car and eventually guided me through a many-, many-point turn (restarted three times). The owner of an adjacent house looked on disapprovingly, convinced with good reason that an idiot was about to crash into her house at low speed.

Obviously you should not ask me for advice about driving when abroad. But thanks to work I can tell you exactly why this happened, and how you can spot the circumstances that might produce similar predicaments for you.

We had geocoded the ferry, plopping its address into an app’s text field and relying on the location to which it was matched. Results from such a service can be divided into two levels of quality: address point and interpolated.

Point geocoding works the way you might imagine. If you put in “150 Main Street”, the system finds the coordinates associated with the building at that address and returns them. Exactly what that spot represents can vary. Sometimes it represents the land parcel, sometimes the building’s rooftop, sometimes even the entryway.

Point geocoding results are the best kind. But point geocoding datasets are never comprehensive. People build new structures all the time, and it’s hard to drop pins for them all in a prompt manner, particularly since address numbers aren’t visible from satellite imagery.

So virtually all geocoders fall back to interpolation when they can’t find a point to match a query. An interpolated dataset includes a map of road network line segments. Each segment record has the road’s name, a start and end number, and whether the odd-numbered addresses go on its right or left side. This lets the system make an informed guess about an address location when no point is available. If a segment named “Main Street” starts at 100 and ends at 200, then a query for “150 Main Street” will be placed at its middle.

This works well a lot of the time. But of course addresses are not spaced perfectly evenly along roads. Some buildings are wider or thinner than their neighbors. Some blocks have several buildings clustered at one end, followed by empty space.

Geocoding data like this is often collected by state or municipal authorities and stitched together into national and international datasets. Those authorities collect it for different reasons: taxes or emergency services or road maintenance, for instance. So it’s common to have the level of quality vary by location even within a single country or region.

And quality tends to be worse in rural areas. There are fewer government agencies managing and maybe-digitizing those places. Businesses can’t make as much profit from those areas, so there’s less money sloshing around to fix problems. And everything is spread out: a so-so interpolation result can be off by a few hundred feet on a city block, or several miles along a farm road.

The interpolation data for our Croatian ferry terminal just wasn’t very good. And there probably wasn’t any data at all for the small houses that I nearly drove into.

In the abstract, this might not be of much interest to people who don’t work on geocoders all day. But it’s worth keeping in mind for your next vacation: if you get a result that doesn’t land on a rooftop, it might pay to be suspicious of the next results, too. At the very least, I recommend paying close attention to any non-obscene gestures that you see construction workers making.


I rode my bicycle a hundred miles yesterday, and so yes, this is mostly an extended humble-brag. But it is also a mark of how much has changed since my first, ecstatic road bike experience.

A down pedalstroke no longer seems to make the ground leap past me, and my legs no longer feel like springs ready to launch. In retrospect, I got what I wished for in that post, a melding of body and bicycle: a fall two years ago means my collarbone is now built with lightweight, precisely-machined titanium (alas, my bike frame is still mere aluminum). Having wishes granted is fraught.

But I still love being on a bike, and I understand it a bit better than I did. Certainly I understand my limitations better: I am not built to be a great cyclist. Too many college days in the gym, adding muscle to protect myself for an awkward but already-concluded adolescence. These days too much mass, period. The great cyclists are hugely tall and nearly skeletal, save for the enormous engines of muscle between their knees and hips. I’m not built like that.

More interesting than that is the revelation of a body’s limits. This is Charles’ insight, really. But a bicycle makes it easy to understand yourself as a machine, to deplete your metabolism before breaking your body in a away that’s hard to achieve when running or doing most other athletic things. The air cools you and there’s little stress on your joints. You can adjust gearing to manipulate the difficulty of the task. Really, you can just keep going until you run out of fuel. If you forget to feed yourself every 20 miles or so, you will fall to pieces, but the brief delay of digestion makes it hard to remember this. Which is why I bonked around mile 60, shortly after we had climbed Sugarloaf Mountain.

Well, no matter. Matt and Charles were patient with me as we stopped in the shade near a park, and a few miles later we sat down to lunch at a restaurant in Poolesville. I consumed a digusting amount of Coca-Cola at a pace exceeded only by my heart rate, which refused to slow down. I ate my club sandwich so fast that my throat still hurts today. Afterward I felt a lot better.

The last 40 miles were not pleasant, exactly–the human butt is not meant to sit on a saddle for that length of time, and a pinched nerve in my neck meant that the skin at the top of my back was numb until this morning. But I took better, steadier care of myself after lunch, and arrived home with an odometer showing a distressing 94.5 miles.

Steph had gotten me some century-related birthday gifts two years before, but schedules and injury had made it impossible. This was a long-deferred goal. Matt had promised an 85 mile ride, which we had (accidentally) exceeded. And Charles had headed down to Hanes point when we reached 15th Street to put in the necessary laps to hit three digits.

So I trundled up and down the Met Branch Trail a few times, watched my bike computer font get the smallest it’ll ever be, then laid down on my living room floor for an hour. Today I feel fine, though I can’t tell if it’s good or bad that the impact on my knees, butt and neck seem to be the same for a 40 mile ride and a 100 mile trip.

fortress: shed

I am already failing badly at Iron Blogger. Should I hit publish on that post in drafts that I wrote months ago but which might upset my family? Is that valuable self-expression or just narcissism?

Well, we’re going to ignore that question for another week, because it turns out I haven’t introduced you to my shed.

Some of you may not realize this, but when you get close to completing the process of purchasing a house your realtor will prompt you to issue some ridiculous demands to the seller, as if you have encountered an unusually boring genie. Typically your request will be for a novelty appliance the seller secretly regrets purchasing, or maybe some furniture. It’s considered impolite to ask for pets, and requests for children should only be attempted when buying from particularly large families.

Well, Steph and I asked to build a shed. The place came with a parking spot, but we don’t have a car or much desire to haul our bikes up a flight of stairs every day. The building is a duplex, so we had to get the neighbors’ okay to build a hulking bike receptable. They are kind people and, better still, maybe did not quite realize what they were agreeing to.

We built a mighty shed, thanks in large part to help from Charles, Kriston and Ben. And yes, we used a kit. But that was just the beginning.

First, the physical security layer. Some steel pipe, threadlock, wood screws and creative reuse of the “SOLD” sign our realtor repeatedly failed to pick up produced a serviceable indoor bike rack.

bike rack

But obviously my heart lies with the electronics.

led strips and infrared sensor

LED strips are affordable and already have resistors and adhesive backing in place. All you need to do is supply twelve volts. That happens to be the voltage of most automotive and marine electrical systems, and consequently also a lot of battery technology. But more on that in a second.

The white dome zip-tied to the cross bar is a passive infrared motion sensor. For $3, it works surprisingly well. The cable running up from there goes to a photoresistor. The apex of the shed’s roof is made of translucent plastic to provide illumination during the day. The photoresistor and motion sensor ensure that the lighting system only activates when someone is in the shed and it’s dark out.

arduino/solar controller

This gadgetry runs down to an Arduino and associated hardware like a MOSFET, a comparator, a trim pot. The Arduino spends most of its life asleep, consuming as little power as possible. But it wakes up a few times a second to see if there’s any business it needs to attend to.

Above it is the solar charge controller. That’s the gadget that sits between the solar panel and battery and the things you’re powering with both. This is a particularly cheap and crappy controller, but it seems to work fine. It’s connected to a sealed lead acid battery on the floor and, on the roof:

shed roof

This guy. My eBay history says I paid $50 for a 20 watt panel back in 2014. Honestly, it’s hard to find units rated for so little power these days. But this is about enough to keep the battery topped up. The arrangement works well, though on the coldest days of winter I’ve learned I need to take the battery inside to keep it from freezing and suffering permanent damage.

The final ingredient is an underwhelming electronic door lock. It wouldn’t survive an intruder’s boot heel for more than a stomp or two, but it might make enough ear-splitting noise before then to alert us. I have an old fire alarm bell from Community Forklift and an RFID module waiting to be installed. But not every project can be a shed project.

If only it could! But they aren’t all winners. Shed telemetry, in particular, has proven tricky:

temp sensor

The obvious need for real-time shed temperature readings, published to the internet, can no longer be ignored. But doing this without draining the battery turns out to be tricky. I’ve got some new ESP8266 Arduino clones in hand. If shed telemetry proves to be viable, you’ll be among the first to hear about it.

universal basic income

UBI is getting a ton of attention these days, and on the whole I’m glad. Smart people I know who care deeply about social welfare have been quietly nursing this dream for a long time, and I respect the depth of their thought and commitment. One friend is even writing a book about UBI! I would like to see a greater share of society’s wealth go to the poor, and this seems like a mechanism for achieving that goal that is worth investigating.

This is not to say that I’m a believer. Untested ideas for social improvement generally look pretty good compared to the ones we’ve actually implemented, which have somehow all turned out to be complicated and horrible. And I’ve known enough people with serious problems to call myself a paternalist without discomfort. The dignity of financial self-determination sounds great on paper, and saying that some people can’t handle it does not. Alas, personal experience has made the latter conclusion inescapable to me.

But I don’t want to talk about that today. Really, I want to talk about the deeply silly people from my own culture who are driving this policy moment. The nerds have discovered UBI.

There’s no better spot to observe this phenomenon than Hacker News, a site that shares Reddit’s basic worldview but is more brainy, shy, and — thank goodness — ashamed of its sexuality. HN is great because threads about, say, astrophotometry fill up with people whose PhD theses were about a _similar_ kind of laser, and they think the way these researchers cooled the dielectric here is really quite clever. It’s terrible because basically everything else is about Soylent or Bitcoin or buying Soylent with Bitcoin.

(Okay, that’s a cheap shot. These days it’s about how you could write Ethereum blockchain contracts to distribute Soylent.)

HN is populated with smart people who work at software startups, most of whom live in the Bay area. Many of them are very excited about UBI. There are nineteen pages of results for the first relevant phrase I tried. It has become a subject of sufficient fascination in the community that Y Combinator, the incubator that birthed and maintains Hacker News, is investing in a series of experiments to evaluate UBI’s viability. They just announced their first UBI director and pilot program, in fact.

This is all to the good. Wealthy people are going to give some money to poorer people to see if it helps them. I bet it will! Good for them.

But at the risk of ruining a good thing, I can’t help wondering why my fellow software developers find this idea so interesting. From that post:

One reason we think it may work is that technological improvements should generate an abundance of resources. Although basic income seems fiscally challenging today, in a world where technology replaces existing jobs and basic income becomes necessary, technological improvements should generate an abundance of resources and the cost of living should fall dramatically.

When Sam says “technological improvements”, I don’t think he means better cookstoves in rural China. I suspect he means the kind of stuff that Y Combinator is funding. Software stuff, mostly — probably a bunch of machine learning projects that promise to finally invent a machine that does more than rotate, plus maybe one or two discount-rate materials science startups (I hear you can make quantum dots in room temperature water these days).

He might be right! But even if he’s not, what glorious hubris.

Imagine meeting a child running a lemonade stand. She’s proud of her lemonade, and why not?

“In fact,” she says, “I think there’s a pretty good chance that this lemonade is going to be the only thing that people drink from now on. I mean try it.”

You do. You have to admit it’s pretty good.

“On the whole I’m excited and humbled to have finally solved the beverage problem. But it won’t be an easy transition for everyone! I mean, for me it will be, I’ll be fabulously wealthy, ha ha.”

It seems polite to join in the laughter so you do.

“So I’ve been thinking,” she continues, “That the responsible thing to do is to invest a portion of my profits into researching how to remediate those negative effects. If I don’t, I think it’s pretty likely that the children of soda manufacturers, for example, will wind up dying in the streets. And before they die, they could riot. It would be hugely disruptive.”

You remark that that sounds like it would be bad for business. She locks eyes with you with a sudden intensity, in way that inescapably says: I knew you would understand.

Imagine thinking you and your buddies are so smart that your efforts are going to make most other human endeavor pointless. It must be sort of overwhelming. I’d probably feel compelled to hire a postdoc to do something about it, too.

Obviously this is not the only reason people support UBI. It’s not even the only reason people in the software industry support UBI! But, knowing human nature and my industry’s hilarious track record at introspection, I do think it’s possible — juuuust possible — that some programmers have been driven by self-regard into a historical materialist analysis under which their superior intelligence transforms society and creates a permanent, pitiable underclass. UBI is both a compassionate response to this sad calculus and, coincidentally enough, a mechanism by which the vast majority of our fortunes and lifestyles can escape disruption.

Well, I wish us all luck. At the moment there is no reason to believe that any of this is happening whatsoever. But it does seem like the robot car thing might work out.

Still, if you’re worried about the technology industry further immiserating the poor, I would pay less attention to Uber ordering LIDAR units and more to them inviting their drivers to finance their vehicles instead of spending capital on a fleet. I agree that autonomous applications driven by deep learning models written in TensorFlow are much more exciting, but if you really want to avoid screwing over the little guy, Excel spreadsheets are probably the first place to look.

iron blogger

My friend and former colleague Paultag has challenged me to participate in a scheme he calls Iron Blogger, which nudges people to blog by fining them $5 per week if they don’t. More posts than usual should follow! And no, this one doesn’t count.

be gentle to their servers and mean to their lawyers

I didn’t like this article about ethical screen scraping very much, and said so on Twitter.

Well, you asked for it.

Screen scraping is the automated collection of information from the web. For our purposes, let’s assume it’s public information. Stuff you can load in your web browser, using an incognito window and a pasted URL. Stuff meant for ungated human consumption.

When might it be unethical to systematically collect this information, which is being published freely? I can think of a few scenarios that might qualify. If your use of the resource makes it unavailable to others, that might be unethical. This could happen if you hammer the server, but it could also happen if you mirror and resell a database that someone has spent money amassing and maintaining, undercutting them and destroying the model that sustains the resource for others.

What if the owner simply doesn’t like the way in which you use their information? Some people think this is a workable way of limiting how information is used. For example, they feel that public tweets shouldn’t be quoted by journalists if the tweets weren’t written with widespread distribution in mind. In a screen scraping context, a realtor site might be fine with you shopping for a home but less excited about your collecting price data to power an analysis of gentrification.

I think that these kinds of implicit rules about how information is used are at best impractical. Well, okay, that’s my diplomatic framing. I really think the sentiment is prudish, illiberal and ludicrous. The transfer of knowledge is not zero-sum and we should err on the side of preserving that miraculous quality. But some people do think along opposing lines.

And although there is very little legal support for their idea that such limitations should be the default way that our society works, it’s certainly possible to impose arbitrary limits on what people do with information you give to them if you can get them to agree to a contract.

This brings us to screen scraping.

You have a right to use information you’ve been told

Liberal society works because facts belong to everyone. Unless you have a very good reason not to, you need to believe in your right to use published information. You need to believe in your right to think and speak freely about the things you have perceived. This is how our civilization works; it’s how our minds work; it’s how reality works. Don’t give up this belief without a fight.

Fuck their contracts

Nearly every website has a terms of service document. These are typically contracts of adhesion that say you can’t use the site at all unless you agree to a ton of fine print, which will often include a prohibition on automated data collection and probably other things you will do in the normal course of using the web and sustaining belief in a modicum of personal rights. They’re also sometimes called “clickwrap” licenses, a rough category of legal agreements that people mindlessly agree to through implicit action when they use software (or when they broke a seal during unwrapping, back when software came in boxes).

This is fundamentally outrageous. You do not enter a contract when you walk into a store or open a book. But the law around websites was born in a later and worse age, when we let them get away with this kind of shenanigan. To a point.

Just because a company puts something in a TOS doesn’t mean it’s legally enforceable. Google probably can’t require you to murder a stranger as a condition of accessing your email, for example. They may not even be able to force Gmail users to permanently forego their right to sue if a self-driving car runs over Fluffy (though Google’s lawyers will certainly try). When the fight lands in front of a judge, who will determine which outrageous overreaches are allowable, Google’s case will be stronger if its lawyers can prove you read and understood the contract terms prior to violating them. I am not a lawyer, and you should consult a real one rather than relying on my advice. But reading the TOS may not do you any favors.

The CFAA is bullshit

Outside of any contracts you mistakenly agree to, the Computer Fraud and Abuse Act is the primary vehicle by which scraping might get you into trouble. Let’s let the EFF explain:

The CFAA is the federal anti-hacking law. Among other things, this law makes it illegal to intentionally access a computer without authorization or in excess of authorization; however, the law does not explain what “without authorization” actually means. The statute does attempt to define “exceeds authorized access,” but the meaning of that phrase has been subject to considerable dispute. While the CFAA is primarily a criminal law intended to reduce the instances of malicious hacking, a 1994 amendment to the bill allows for civil actions to be brought under the staute.

This is a stupid, arbitrary law, and you are potentially violating it every time you use the internet. You should be aware that it exists in the same way that you are aware sharks exist. But you shouldn’t let them stop you from going in the ocean.

Not asking questions is a great way to avoid dumb answers

This is tricky, I know. If a site operator might be excited about your project, getting their permission might unlock better data, save you time, and avoid subsequent fights. But if they’re antagonistic or even just *surprised*, they will instead ask their lawyers how they should respond to your request and their lawyers will (eventually) tell them to say “no”. Then you will have no plausible way to claim that you didn’t know you shouldn’t collect the information. Worse, the publishers will be on their guard.

If you think the site operator might want to work with you, you should ask for their help. If you’re not sure, you should instead ask yourself if you have an ethical claim to the data. The site operator is not necessarily the appropriate arbiter of that question. At Sunlight we encountered endless situations where the site operator was not the information’s rightful owner. Government sites, hosting public information, with robots.txt files forbidding automated collection? To hell with that. It’s wrong.

The stakes matter

I say all of the above blithely and confidently, and I think it’s good advice for the audience to which the original talk was aimed: journalists. It is decidedly not how I approach these questions in my professional life, at least not these days. I work for a private, for-profit enterprise. We’re trying to make money. We have the resources to be careful, to buy licenses, to read contracts, and to be worth suing.

And while I’m proud of how much work our company does to add to the public good, we are not investigative reporters or nonprofit activists. Perhaps more to the point: if we callously take someone else’s information and they come after us with a decent argument about it, no one will shed tears for us.

If you are acting on behalf of a corporation, talk to your counsel, then talk to their counsel, then work out an agreement. Take it from Gawker, getting deposed isn’t as fun as it sounds.

If you are a journalist, a hobbyist, an activist, or really anyone seeking knowledge rather than wealth: scrape that site. Teach us something. Try not to be a jerk about it. It will probably be fine.