I went about casting a net on my Twitter account, and I already have two potential occupiers lined up. But, from that post I also received encouragement to scale this up and accept donations.

I’ve never done this before, but I signed up for an account on WePay. I am now accepting donations if anyone wants to add their support sending people to sit with the #OWS movement.

This blog post serves as a way of providing the social proof of who is behind the donation page. If you donate and you don’t like how your money got spent, I am the guy you’re going to come talk to.

So, if you want to help have your voice, as a part of the 99%, to be heard but you don’t have the time to spare, please consider donating.

At one point in my life, I was a volunteer Firefighter/EMT. During my time working in paramedicine, I learned a valuable concept: the pertinent negative.

Let’s say you’re examining someone who fell from a height. In your report you node that yes, they had a laceration on their hand. But it is also important to note that you examined them and found no evidence of any other broken limbs. Not noting your investigation into broken limbs could mean one of two things: either you didn’t check their limbs, or you did and found nothing of concern.

By noting the pertinent negative, you clear up that ambiguity.

I find this concept incredibly useful in software. Frequently, I need a pertinent negative to do my job well. (Did that script complete without errors, or did it not even run? In a report, a 0 entry tells me that the item did fit my search criteria and it had no requests. Etc.)

I have to say that this concept of pertinent negatives keenly reminds me of how important an ‘invention’ zero was.

I recently went to my second South by Southwest Interactive (SxSW or SxSWi) in Austin, TX. If you’re not familiar with the conference, it is a 5-day internet and technology subsection of the greater SXSW conference which also has Music and Film components.

SxSWi has a long history of being a great place for cutting-edge technology makers and creators to mingle and share ideas. I personally saw, felt, and benefited from this great tradition. However, people who have been attending the conference for more than 3 years say that it has changed substantially as its influence has been recognized, and there are continually growing populations of the ‘business-y’ (read ‘douchebag’) hangers-on: marketers, salespeople, and those I think of as ‘traders in idea derivatives’. (Luckily, for my part, I was able to find a tribe of people who are actually creative and engaged in what they do, so I was able to minimize my exposure to the unsavory, financially-motivated elements.)

But not fully. There are several blog posts I’m sure I could write about the interactions between ‘the Makers’ and ‘the Sellers’, but one aspect of our differences really calls out to me for comment: business cards.

Some people can be really, really intense about business cards. (Check out this guy.) In some business cultures, not treating the business card exchange with the proper respect can end up costing you potential partners.

My impression is that at SxSW, people think of business card dispensing as a quantity-over-quality proposition. Some members (with whom I am friends) of the “How Not to Be a Douchebag at SXSW” panel (2010, 2011) panel even give advanced warnings that there are people who approach business-card delivery in this manner.

The whole business card thing is so ripe for mockery that @scottyiseri and I created a fake ‘Angel Investment’ firm, just so we could give out the fake business cards you see at the top of the post. (I ended up dispensing these cards out to friends, so they could in turn use them as a way to avoid giving their actual information to someone who was being egregiously douche-y.)

But it wasn’t the blatant card-distributors that got to me, it was the nuanced ones. After one session, I ended up with a card from a recruiter from North Carolina, and another from a small-business efficiency consultant from New Jersey. These cards were handed to me while I was having actual, pertinent conversation about the session with the card-givers. Twenty minutes later in my hotel room I was tearing the cards up and throwing them away. I had zero need for these cards.

On the plane ride home I got to thinking about those cards. I realized that every single unwanted card that ended up in my possession was a symbol of laziness: my laziness in protecting my integrity. (And I’m pretty into my integrity.) If I have absolutely no interest in your business card, and have no intent of ever trying to contact you ever again, why accept the business card? (You may say ‘politeness’, but I don’t buy it, because any ‘good’ marketer will politely bully their way into your wallet if you give them the chance.)

Going forward I have a new resolution. I will no longer passively accept business cards, I will either actively accept (less likely) or actively reject (more likely) any business card that is offered to me.

Will you join me? (Why not?)

Both of the the Bit.ly referrers (like “forexmarket”) just resolves to this URL “www.plus500.com/SiteNotAvailable.aspx”.

It’s just a stupid banner page with a big GIF built to look like an actual web page.

What the fuck is the point? Yah, I was interested enough to investigate it, but only forensically… There is no way I would ever do any business with this place.

The other thing that frustrates me is that Bit.ly let the redirect through, even though they know it’s a spammy thing. (A couple of weeks ago when I first found these types of referrers, I tried to follow the link, but Bit.ly warned me about not-above-board behavior of the link. Why no longer?)

Also, the landing page is using Google Analytics. Does this mean all the good tools that those of us who aren’t devil-spawn are being used by they outright shitheads? I guess I know the answer, even if it does hit me right in my idealism.

For the past year or two I’ve had an ongoing side project where I post to the Twitter account @bman_countdown. It started out as just a simple countdown, but I realized I could use it as an opportunity to provide more information about the event, so I started to tack on links to relevant content on the BurningMan.com website.

Over time I’ve increased the functionality from being a static set of links, to mixing in the most recent blog post from the Burning Blog. (Actually, this functionality just went live today.)

This has been a great ‘toy’ project that I use to experiment with different technologies.

The Environment

For the first couple of iterations it was a Ruby script that I ran off of my ‘media server’ (a Mac Mini which I had set up as recommended by MacMiniColo). To emulate cron functionality, I used launchd to run the script once a day.

Predictably, this setup caused some hiccups in the continuity of the service: when my ISP service went out; when my router borked; when the hard drive failed; when I moved; etc. (Yay for providing non-mission-critical projects!)

Around the time of the 2010 Burning Man event, Twitter removed the ability to use Basic Auth, which is what the library I used relied upon. I decided the service could take a break right after the event while I figured out what I was going to do with it.

I had done some minor playing with Google AppEngine in Java, so I was familiar with the basic services GAE provides. (I love AppEngine’s first-class cron support. And the TaskQueue concept ecourages truly asynchronous and scalable design.) But, since this project is what I use to experiment with stuff, I figured I’d try out Python.

Meeting Python

First, I had to get up to speed on Python itself. So, I got the book Programming in Python 3. I studied several chapters of this book and did a couple of the ‘programming assignments’ (which I thought was a great idea), until I felt I was familiar enough with the language to take a poke at upgrading the countdown.

I came to realize that the process of developing in Python is so easy! (Even more so in GAE, as the development server environment that you get is great, and shit just works: just save the changes to the file and you can hit the page, and you get instant feedback.)

I know I’m a latecomer and I am treading dangerously close to Programming Language War territory here, but I have to say that I am entirely fed up with the edit/build/deploy cycle that leaks time and energy in Java projects. Do you realize that in big Java shops, there are engineers whose entire function is to iterate on the build process? Remember that whole “Google App Engine in Maven + IntelliJ” blog post that I made? In Python, the build-cycle-via-build-tool process doesn’t exist. Imagine a future where you don’t need to wrestle through Maven’s bullshit to get stuff done. I can see this future in Python, and it gives me tingles in my naughty bits!

Sure, I’ve got rose-colored glasses on here. I am a native speaker of high-protocol Java (my kingdom for a native “?” operator!), who is currently a tourist in a strange land speaking a stilted pidgin version of Python. Admittedly, I have an incomplete grasp on what the costs and benefits are when moving from a compiled language to an interpreted one. But it sure seems easier so far.

Future of Java

I was originally pulling for the Java ecosystem, whether that comes from a selfish (it’s where my bread gets buttered) or a philanthropic (because I’ve seen so much good Open Source stuff come out of it) point of view, I’m not sure. I was happy when some very smart people talked about ways that at least the JVM could stay relevant by embracing more dynamic languages built on top of it.

But then there was last weeks announcement that Mac OS X would deprecate support for Java. This generated a lot of excellent discussion. My favorite post both described why this is a death knell for Java, while also outlining a strategy for ensuring Java could continue to function (by breaking up it’s fairly monolithic API into discrete modules).

Alas, because of a very specific interaction I had, I fear Chris Adamson’s cogent post will probably be ignored by Oracle. Back when the news came out that Oracle was buying Sun and therefore Java and MySQL, I spoke to a friend of mine who worked for Oracle. I told him I didn’t care if Oracle owned Java and MySQL as long as they didn’t fuck it up. His response: “Don’t worry. We’ll fuck it up.”

The mere fact that I heard this kind of pessimism coming from an employee within Oracle makes me think Java will be viewed more and more as a legacy system in the months and years to come.

Not that I have a whole lot to hide nor do I wear a tinfoil hat, but every once in a while I like to do a personal audit of where my privacy could be (or is) getting invaded. So, I’ve been collecting informal data points about my experience with the ClipperCard, which is what leads to this post.

What They Expect

Here is my daily paper trail (all times are approximate):

• 0800: “tag on” an inbound #14 MUNI near Duboce & Mission
• 0810: “tag on” an outbound #47 MUNI near 11th & Mission
• 0840: “tag on” at the CalTrain station at 4th & King
• CalTrain conductor scans my ClipperCard on the 0844 Southbound train
• 0926: “tag off” in Palo Alto on the Southbound side of the tracks
• 1700: “tag on” at the CalTrain station in Palo Alto on the Northbound side of the tracks
• CalTrain conductor scans my ClipperCard on the 1706 Northbound train
• 1744: “tag off” at the CalTrain station at 4th & King
• 1746: “tag on” to an inbound #47 MUNI at 4th & King
• 1800: “tag on” to an outbound #14 MUNI at 11th & Mission

Ways to Mess With the System

1. I have a monthly pass for the CalTrain. I’ve realized that my pass doesn’t ever require that I tag on and tag off, as long as it responds appropriately to the on-train scanning by the conductors.
2. The only times I’ve gotten away without being scanned by a CalTrain conductor is when the train is packed to standing-room-only because of a baseball game in the city. You might be able to sneak past the conductors, but I wouldn’t count on it.
3. Similarly to CalTrain, I don’t really need to tag on to the MUNI busses, as long as my monthly pass resolves itself to a Fare Inspector. Most of the drivers will either look at you funny or explicitly require you to least flash your card by the reader. However, you can get a beep out of the readers by placing your card on the reader, but then removing it quickly. The reader realizes there’s some kind of card in front of it, but I’m assuming it can’t figure out exactly whose card is there, so it gives an error-sounding beep. This usually satisfies the drivers. (Only once in a month of doing this did a driver shame me into fully tagging the reader.)

What They Can (Still) Learn

If I didn’t take the pains to mess with my own data, you could have a pretty good idea that I live somewhere near Duboce & Mission and that I work somewhere in Palo Alto. With further analysis (and further data that I didn’t provide here), you could also notice that on alternating Wednesdays I don’t go to Palo Alto, but I seem to take the 14 to work somewhere in downtown, probably near 3rd and Mission.

I was able to mostly obfuscate my whereabouts with regard to the MUNI surface busses, except for the occasional pedantic driver. Plus, there’s a built-in obfuscation because they don’t require a tag off. However, this obfuscation doesn’t hold if you are trying to get into one of the underground stations downtown (or BART, for that matter).

For CalTrain, I was able to obfuscate the specific stops where I get on and off, but since my monthly pass is specifically identified as a Zone 1 (SF to San Bruno) to Zone 3 (Atherton to Sunnyvale) and I use “Limited” or “Express” lines (which only service certain stations), they can still get a general idea as to where I’m located.

Additionally, if you look at the records of the conductor scans, you’ll find that on my return trip I am repeatedly found within 2 or 3 scans of specific other customers. You might be able to deduce that since I am likely coming back from work, these other people are highly likely to be my coworkers. And if these people explicitly tagged on, there’s a high probability that you now know where I got on, even though I didn’t tag on myself.

Other (Potential) Problems With the System

To use the website to add cash in addition to the loaded passes, you had to do some configuration. I didn’t recognize it as a problem at the time, but it required that you have a valid credit/bank card. A friend of mine (who had to cut up her credit cards to pursue a debt-free lifestyle) said she was basically left by the wayside with regards to the ClipperCard and the benefits of its use. (Which is becoming, more an more, mandatory for monthly pass holders.) I do not know if this is still the case, but it sounds like something the ClipperCard implementers would do.

Another thing to keep in mind is that all my travel is also hooked up to my Credit Card via the ClipperCard website. You’d have to talk to other black-hat-ier people than I to learn how vulnerable the ClipperCard site is. (As a none-too-impressed end user of the website, I am sure the implementation has plenty of security holes.)

One last thing to keep in mind is who has access to your ride records. A co-worker said he was able to get a CSV file of all his ClipperCard tag ins, just by calling the Customer Service line and the rep running a quick query to get all his activity. This means the reps probably have access to everyone’s records at their whim. I have dealt with Customer Service at ClipperCard, and I’m sure it’d be pretty easy to socially engineer one of them into giving up a whole bunch of information.

Do I Care?

What does this all mean? Well, with a little bit of effort, you could figure out where I am likely to be at several points during the week. Why do I care about this? Imagine a universe where I am a vocal proponent of a measure that takes aim at MUNI drivers’ pay, and people within the union want to find me at some point to change my mind for me… Is my data easily obtainable, especially to those within the public transit system? I think so.

Conclusion

In this exercise, I’ve tried to show just how vulnerable your personal location data can be, and how it could be used to gain some further bits of knowledge about you.

In reality, the biggest risk to the average person in the ClipperCard system are the potential for a third-party (hacker) to get your credit card and personally-identifying information. A far smaller secondary risk is having your historical transit records and patterns made vulnerable, specifically because access to this information is a lot less regulated than access to your personal financial information.

At first, I just developed the prototype in the default GAE/J Eclipse environment until I could deliver a functional POC. After finding the GAE/J capabilities more than adequate for what we wanted to do, I was challenged to bring the project into our standard IntelliJ + Maven development environment. For the rest of this post, I’ll share a couple of tips and tricks for getting your GAE/J project to operate in this environment.

Basic POM File

There’s some funny business and frustration around the Maven community’s adoption of GAE/J, but I’ll skip that part of the story for right now. What I found is that the maven-gae-plugin project is the best place to go to for help Mavenizing a GAE/J build.

I have to say that it’s not ‘use the archetype’ easy (their archetype failed for me), but with a bit of elbow-grease and rummaging through their documentation I was able to get a decent and functional POM file built. Here it is (with some of our proprietary information scrubbed to protect innocent servers):

 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>com.widgetbox</groupId>
   <artifactId>image-proxy-webapp</artifactId>
   <version>1.0-SNAPSHOT</version>
   <name>Widgetbox :: Image-Proxy :: Webapp</name>
   <packaging>war</packaging>
   <properties>
     <gae.version>1.3.0</gae.version>
     <gae.app.name>qa-image-proxy</gae.app.name>
   </properties>
   <dependencies>
     <dependency>
       <groupId>javax.jdo</groupId>
       <artifactId>jdo2-api</artifactId>
       <version>2.3-eb</version>
       <exclusions>
         <exclusion>
           <groupId>javax.transaction</groupId>
           <artifactId>transaction-api</artifactId>
         </exclusion>
       </exclusions>
     </dependency>
     <dependency>
       <groupId>javax.transaction</groupId>
       <artifactId>jta</artifactId>
       <version>1.1</version>
     </dependency>
     <dependency>
       <groupId>com.google.appengine.orm</groupId>
       <artifactId>datanucleus-appengine</artifactId>
       <version>1.0.4.1</version>
     </dependency>
     <dependency>
       <groupId>org.datanucleus</groupId>
       <artifactId>datanucleus-core</artifactId>
       <version>1.1.5</version>
       <exclusions>
         <exclusion>
           <groupId>javax.transaction</groupId>
           <artifactId>transaction-api</artifactId>
         </exclusion>
       </exclusions>
     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>
       <artifactId>datanucleus-jpa</artifactId>
       <version>1.1.5</version>
       <scope>runtime</scope>
     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>
       <artifactId>geronimo-jpa_3.0_spec</artifactId>
       <version>1.1.1</version>
       <scope>runtime</scope>
     </dependency>
     <dependency>
       <groupId>com.google.appengine</groupId>
       <artifactId>appengine-api-1.0-sdk</artifactId>
       <version>${gae.version}</version>
     </dependency>
   </dependencies>
   <build>
     <plugins>
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-compiler-plugin</artifactId>
         <version>2.0.2</version>
         <configuration>
           <source>1.6</source>
           <target>1.6</target>
         </configuration>
       </plugin>
       <plugin>
         <groupId>net.kindleit</groupId>
         <artifactId>maven-gae-plugin</artifactId>
         <version>0.5.3</version>
       </plugin>
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-war-plugin</artifactId>
         <version>2.1-beta-1</version>
         <configuration>
           <filters>
             <filter>${project.build.directory}/version.properties</filter>
           </filters>
           <webResources>
             <resource>
               <directory>src/main/external</directory>
               <targetPath>WEB-INF</targetPath>
               <filtering>true</filtering>
             </resource>
           </webResources>
         </configuration>
       </plugin>
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-antrun-plugin</artifactId>
         <version>1.3</version>
         <executions>
           <execution>
             <phase>compile</phase>
             <configuration>
               <tasks>
                 <echo file="${project.build.directory}/version.properties">
                     friendlyversion=${project.version}
                 </echo>
                 <replace file="${project.build.directory}/version.properties" token="." value="-"/>
                 <replace file="${project.build.directory}/version.properties" token="SNAPSHOT" value="snapshot"/>
               </tasks>
             </configuration>
             <goals>
               <goal>run</goal>
             </goals>
           </execution>
         </executions>
       </plugin>
     </plugins>
   </build>
   <repositories>
     <repository>
       <id>maven-gae-plugin-repo</id>
       <name>maven-gae-plugin repository</name>
       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
     </repository>
   </repositories>
   <pluginRepositories>
     <pluginRepository>
       <id>maven-gae-plugin-repo</id>
       <name>maven-gae-plugin repository</name>
       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
     </pluginRepository>
   </pluginRepositories>
 </project>

(FYI, we’re not actively using any datastore functionality just yet, so if you are going to use this template please forgive me if those dependencies are a little bit wonky.)

Since Google hasn’t (yet) decided to publish their development environment in a Maven-friendly way, there’s a bit of dependency wonkiness involved in getting the maven-gae-plugin to work. I included the repository information required by the plugin (lines 121 – 134), but if you use a repository manager (like Nexus), you’ll want to remove those lines from the POM and add a proxy for the maven-gae-plugin’s repository.

To get the development environment working the plugin also requires access to the unzipped SDK as packaged by Google. The plugin tries to help you set this up (“gae:unpack”) but that failed for me. I was able to get stuff working by manually unzipping the SDK artifact downloaded directly from Google to the following directory:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0

Incremental Improvments

Initially, I had kept the appengine-web.xml within the WEB-INF directory, but I realized I could make our Release Manager’s life a bit easier if I added a bit of build-time substitution.

Here’s our appengine-web.xml:

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
	<application>${gae.app.name}</application>
	<version>${friendlyversion}</version>
	<system-properties>
		<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
	</system-properties>
</appengine-web-app>

And you’ll see that I put it into a new source directory called ‘external’:

At build time, I use the AntRun plugin (lines 97-118) to create a small file under the target directory that holds a ‘sanitized’ version of the standard Maven version. (I.e. “1.0-SNAPSHOT” becomes GAE-friendly “1-0-snapshot”.) I then use the Maven filter functionality available in the WAR plugin (lines 80-96) to copy the appengine-web.xml into its proper directory with the version substituted in.

You’ll also notice in our appengine-web.xml that we substitute in our application name. By default this comes from the properties section of the pom.xml file (line 11). I did this because we’ve actually got 2 different applications up on GAE’s servers, the QA version and the Production version. By default we build using the QA server’s application name, but when our Release Manager is building to upload to Production, all that is needed is an additional command-line argument of “-Dgae.app.name=<prod-name>”.

Running, Debugging, and Deploying

The two most valuable targets that maven-gae-plugin provide are “gae:run” and “gae:debug”. These will assemble your code in the standard Maven webapp target directories and run your app. (Note: “gae:debug” didn’t actually work for me until the 0.5.3 version of the plugin.)

There is also a “gae:deploy” target that is supposed to invoke the Google-supplied shell script that will upload your application to the Google servers, but it failed for me several time. Since then, I’ve defaulted to using the shell script directly to deploy my app once it has been built:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0/bin/appcfg.sh \
    update \
    ./target/myApp-1.0-SNAPSHOT

Results

So, this is how we got up and running with GAE/J in our standard development environment. Hopefully this post ends up helping people out to reduce their bootstrap time when evaluating/investigating GAE/J for their own uses.

X-hacker: If you're reading this, you should visit automattic.com/jobs and apply to join the fun, mention this header.

I think this is a pretty cute and subtle way of looking for technically adept potential employees.

They gave me an electronic copy of the book to read, and asked for a 300 – 350 word review about the book. I’ve gotten it mostly to the point of ‘completion’, and any more changes would just fall under the category of ‘tinkering’.

Maven. I hate the word, as I hate hell, all Montagues, and MAKE.

Maven is a powerful Open Source build system which is becoming a de facto standard in Java development circles. Like any powerful tool, Maven has its own proponents and detractors, benefits and drawbacks. For example, one of the consistent issues that plagues the Maven ecosystem is a dearth of quality documentation.

When I heard about “Apache Maven 2 Effective Implementation”, I had hoped that it would be the One Book to rule them all, the One Book to find them, the One Book to bring them all and in the darkness bind them. Unfortunately, this is not that One Book.

“Apache Maven 2 Effective Implementation” is yet another effort by a Maven contributor to provide some clarity to Maven’s user community about how exactly to use Maven correctly. But much like the rest of the Maven documentation ecosystem, this book provides the type of documentation that only its author could love.

The author is obviously knowledgeable, however I can’t recommend this book as anything more than an incremental addition to the Maven user’s arsenal. Having been a Maven ‘believer’ since the late 1.x days, I would say that ALL the documentation I’ve seen on the subject has been no better than incremental. In most cases, like this one, the authors falls into a classical trap of technical documentation by explaining what to do, but rarely explaining why. (Nor, more importantly, explaining how to figure these things out for yourself).

I did find a few helpful tidbits of new information in the book. However, I’m not sure I would have been able to quickly find them again in a reference situation because the logical flow was all over the place. Example: Chapter 4 had ten pages of fundamental reporting information, which is great information to have; but maybe Chapter 5, which was titled “Reporting and Checks”, is where that information rightly should have resided.

Unlike some of the best technical books I’ve read, my ability to stay awake while reading this book was challenged because it is so very dry. I had no delusions of it being as compelling as the “Encyclopedia Galactica”, but the prose was completely wooden. This blandness and the previously mentioned problems render the book altogether forgettable.

I actually had a couple of people help me with the editing and/or proofreading of my review, and I’d like to thank Jennie-Sue, Eli, and Starchy for their offers of help.

If you haven’t noticed, I have ZERO advertising on my site. It only costs me a couple of bucks a year to keep this site going, and I didn’t want to spoil the relationship with my readers by trying to commodify their viewing. So, when I was approached to do the review, I was very specific that I would not pull any punches just because they gave me a free book. Packt also tried to sign me up with an affiliate link to the book, which you’ll notice is nowhere in this (or any) post.

As you can tell, I really didn’t use the kid gloves with this review. I’m not very happy with the general quality of Maven documentation (as I’ve noted before), and I wasn’t going to give this book high marks for being just as bad as the rest.

I did manage to sneak in (or are they clumsily manhandled?) 3 different literary fiction references, which I hope spices up the review and adds a bit of character. Let me know if you can name them.

I started out my trip in Bordeaux, where I found a pretty nice hotel to be my ‘base of operations’ whilst I tried to work out my motorcycle rental. I managed to find the website and then the physical location of Bordeaux Scooters, and this is where I rented the motorcycle for one whole week.

For the first day, I just rode around the city of Bordeaux a very little bit, trying to accustom myself to the Yamaha Fazer 650 (since I normally ride a Kawasaki Ninja 500).

On my first full day on the bike, I got out of Bordeaux and hit a bunch of secondary (C and D) roads. I hit places like Biscarosse(-en-Plage), Mimizan, and Léon, before I stopped for the night in Hasparren where I stayed at a cute Hôtes de Chambre (Bed & Breakfast). I was feeling pretty good and confident, both in my motorcycling ability and my language ability to operate in the hinterlands of France.

For my second day of riding, I figured I’d take some more scenic roads, and even cross over the Spanish border for a bit. I made it through the border just fine, and was cheerfully enjoying the nice big sweeping roads on the way to Elizondo, and then turned on the road towards Erratzu to make my way back into France near Saint-Etienne-de-Baïgorry.

It was just after a hairpin turn, not more than 1km from the French border and the summit of the hill. I noticed the old stone barn on the left side of the road, then I looked uphill and contemplated that I should be careful because of all the mountain mist that was probably making the road slippery. I brought my attention back to the road, and was alarmed to find that I had drifted close to the side of the road. I started to panic. I was so worried about the side of the road that I couldn’t look away, and motorcyclists know that’s exactly where I ended up.

Now that I was on the edge of the road, I made a snap decision to bite the bullet and take the bike off the road and try to stop before anything untoward happened. Something untoward happened. The mist I had been worrying about mere milliseconds before was coating the grass I was on and the front tire slipped sideways, dumping me and the bike onto our left sides.

I am lucky because I had just recently come out of a tight turn and I wasn’t going all that fast. I landed face down with my torso on the pavement, and did only about 1 1/2 horizontal pirouettes before I came to a rest. While the helmet I was wearing was scraping across the pavement, I remember thinking “Boy, a helmet is a really good idea!”

I was up in an instant, just in time to see that the hard case (holding most of my possessions) rolling down the hill. In my adrenaline-addled mind, I decided that I really needed to get my stuff back. So, I went careening down the side of the mountain in my full gear. I found the case 1 1/2 switchbacks down. I brought it up to the road (1 full switchback below the motorcycle) and only then did I decide to take off some of my gear and inspect for damage. Other than a little bit of scuffed up skin, I was actually fine.

As I was walking back up to inspect the bike, a younger French guy in a car stopped to give me a hand. He helped me turn the bike over, and point it away from downhill. He told me he expected insurance (assurance in French) could be found in any town, and that they’d have me set back up in no time. I thanked him for his help and advice, and sent him on his way, while I figured out what I needed to do.

After I got all my stuff assembled, and took a couple of pictures, I sat down and tried to call the rental shop. I ended up calling Ryan (in the U.K) for some research help because I didn’t know what the country-code is for France. After we got that figured out, I realized the rental shop was on their stated lunch break until 2PM. (It had just turned about 12:30PM.) I sat there for a bit trying to keep myself calm and counting my lucky stars. After I changed into some less-sweaty and warmer clothes, I decided that I should try to get to ‘civilization’, preferably on the French side of the border since the motorcycle was rented from France.

Eventually a nice (German?) couple on motorcycles stopped to give me a hand. We were formulating an elaborate me-plus-my-stuff-on-their-bikes plan, when I realized it was a bit soon for me to be riding on a motorcycle again. I got them to help me flag down a passing carpenter, who agreed to take me and my stuff in his truck to the mechanic’s in Saint-Etienne-de-Baïgorry. The local mechanic was of no help, other than trying to help me call the rental shop again, which I knew was still closed. After he basically told me to go away because he didn’t deal with motorcycles and he was on his lunch break, I walked towards town center and sat on the steps of a hotel/brasserie that was also closed for lunch. I decided to wait out the remaining hour or so until the rental shop opened up by chilling out and reading my Kindle on those steps.

Once I did get the rental shop on the phone, they told me that I needed to get the bike transported back to Bordeaux (250km) by whatever means necessary. The shop owner offered to see if he could find someone to do it, and I said I’d look around at my end. After our conversation, I had no idea if I would be stuck in the town of Saint-Etienne-de-Baïgorry overnight, but I figured I’d check into the hotel until things settled, and if I had to leave early, I could just lump one night’s charge.

As I was checking into Full-screenHotel Juantorena, I mentioned (or whined) to Mélanie (who I later found out was one of the owners) that I had just been in a motorcycle accident and needed to find a truck to bring the broken motorcycle back to Bordeaux. She said that her husband might be able to help me out. Bixente came out from the kitchen, and called around to some truck services in Biarritz, and found me a quote of €1,200. I thanked him and said I would call back to Bordeaux to see if the rental place had found me a better quote. The rental place said to call back in a couple of hours because the friend they were going to ask was out at the moment and it would be a while before they got an answer.

It turns out that Bixente is also a motorcyclist, and offered to take me (after he finished setting up his kitchen for the evening) to pick up the motorcycle on his motorcycle trailer to bring it down (the 10 km) to his hotel parking lot. I was kind of amazed at this, but just went with it after the day I was having. About a 1/2 hour later, after I had washed off some of my road rash, he called me down to the front of the hotel where he, his truck, and his motorcycle trailer were waiting for me. He drove me up to the crash spot where we loaded up the wreck, and brought it back to the hotel.

The ride up and back could have been horrible. But after figuring out my French skills needed just a bit of annunciation and a slower pace, Bixente actually took the time to get to know me. We had a really nice conversation, and I’ve come to look back upon that conversation as one of the nicest moments during my time in France.

After getting back to the hotel, I called the rental shop to tell them that the bike was now in the parking lot. In addition to the location of the bike, we had some (mis-)communication where I thought they said to wait several more hours to verify that the truck from Bordeaux would be interested in the job. So, I took a little nap, then read some more whilst having a beer in the bar. I was really surprised when Bixente told me the truck driver was at the other end of town and would be arriving soon. (Evidently, when I thought the rental shop was telling me that I was waiting for confirmation, they were actually telling me that I was waiting for the arrival of the truck.)

The truck driver, myself, and Bixente loaded up the bike into the truck. It was now about 8 or 9PM, and the driver hadn’t eaten, and neither had I, so we had dinner at the restaurant. The driver was kind of gruff, and not all that friendly, but I managed to get some small-talk in. After dinner, using Bixente as a patient go-between, we established that it was best if I went with the truck driver back to Bordeaux with the bike. (I didn’t realize there was more for me to do there.) So, I checked out of the hotel. I tried to convince Mélanie to keep my room charge, but she wouldn’t hear of it. She insisted that if I weren’t staying overnight they wouldn’t even think about keeping my money.

The driver brought be back to Bordeaux, and dropped me off at midnight at a hotel right near the rental shop. The next day, I went to the rental shop to find out that we were waiting for a verdict from the motorcycle repair shop. When I remembered that I hadn’t yet paid the driver from the evening before, the rental shop owner called him for the price. Since I was paying in cash for the truck ride, it only came out to €500, which is a great deal compared to the other quotes I had received.

Later that afternoon, the verdict came down that the bike’s frame was bent and therefore “totaled”. I went back over to the rental shop to sign the credit card slip for the €1800 security deposit that I had now forfeited. And thus ends my motorcycle adventure in France.

Looking Back

Now, people say all sorts of horrible things about the French. In general, I can tell you that much of this is wrong. In specific, I can tell you that Mélanie and Bixente are some of the warmest and most caring people I’ve ever run into in my travels. They took the time to patch up and help this stupid American with bad French and even worse motorcycle skills, and not ask for anything in exchange, other than an email greeting at some time in the future. I hugged Mélanie and Bixente fiercely, and I left their hotel with tears in my eyes because I felt really fortunate to have met these beautiful people.

I realize that I directly benefited from what I refer to as the Motorcyclist (a.k.a. Motard) Fraternity. Without knowing me, but because I spend time on two wheels like they spend time on two wheels, both the German couple and Bixente went way out of their ways to make sure I was taken care of. I guess it’s a similar thread amongst motorcyclists to help out your fellow motorcyclists.

Doing Them A Solid

In addition to adding Mélanie and Bixente to my Xmas card list, I have decided to send as many people as I can to them. So, if any of you readers are ever interested in a quaint village in the beautiful Basque region of France, might I suggest Hotel Juantorena? Also, spreading this info to motorcyclists far & wide would be much appreciated.

(I will try to scan in their pamphlet, but in the mean time, here’s their contact info:) t

Hotel Restaurant Juantorena

64430, Baigorry

Pyrénées-Atlantiques, Aquitaine, France

Lat/Long: 43.17405, -1.34875

Phone: 05 59 37 40 78

Email: hotel.juantorena@orange.fr

Their pamphlet shows that they have a website (http://www.hotelrestaurantjuantorena.fr) but it doesn’t seem to be working?

I was blown away today when I got an email from Brittany saying that not only will Widgetbox management support me in this endeavor, but they are going to add a 1 day per year allowance to our policies for all the employees to pursue similar efforts. Wow!

This just cements for me that I am totally working with the right set of people in a company that is just the right size for me. I feel appreciated, listened to, and respected, even when I bring forth crazy ideas like sending me off on our investors’ dime to save the world.

Yeah, my life: it doesn’t suck!

I do not like Facebook. I do not like it one bit. I want nothing to do with Facebook at all, and I’ve been very adamant about not signing up for an account despite the urging of everyone and their brother.

So, when I recently gained an understanding of how Facebook Connect works, I got a bit upset. Let me show you why with an example.

Let’s say I want to create an account at MapMyRun.com which implements Facebook Connect, I will use my primary email address of “nelz@example.xxx”. After my account gets created MapMyRun then encrypts my email address with a one-way hash, which for the sake of argument looks like “BIGBADHASH”, and sends this hash to Facebook. “So what?” you think. Sure, they can’t figure out anything about me from a single one-way-hash of my email address, right?

Well, then let’s say I register for an account at io9, again with my primary email address “nelz@example.xxx”. Since io9 also implements Facebook Connect, they also hash my email address and send it to Facebook. Guess what? The resulting string, “BIGBADHASH”, is the exact same as what MapMyRun just sent them.

Now, Facebook knows that someone with an email address that hashes to “BIGBADHASH” has an account on both MapMyRun and io9. Again you think “So what?” That one-way-hash protects my identity from being associated with any of these specific behaviors, right? Au contraire, mon frère.

Now one of my well-meaning friends, who doesn’t know how much I loathe Facebook, tells Facebook that they want to connect with someone at the address “nelz@example.com”.

This was the last piece of the puzzle that Facebook needed. Facebook need only apply the one-way-hash to the plaintext email that my friend just provided them with, and they get “BIGBADHASH”. Then, they look up in their database where they’ve seen that hash before, and now they know that a user with an email address of “nelz@example.com” has an account on both MapMyRun and io9.

Now I, who has never ever logged into Facebook, am getting tracked by Facebook whether I like it or not.

External Remediation

Okay. So what can we do about this?

I looked on Facebook.com, and I found a Privacy Policy. But everything on there requires that you have an account on Facebook. I guess I could create an account, then delete it, but I don’t want an account on Facebook! What Facebook really needs is some web form for non-members that says “Forget anything you have, or will, ever collect about email XYZ”. But, that’s not likely to happen.

What about the end-points, the sites that are implementing Facebook Connect? Personally, during their sign-up processes, I’d like to see a check box that says “Don’t sell me out to Facebook.” Again, that’s not likely to happen.

Personal Remediation

Now, don’t get me wrong, I’m not a n00b. I realize there are all sorts of sites tracking my footprints across the digital landscape every day. (Quantcast, DoubleClick, etc…) For most of them, if I were paranoid and technical enough I could purge their cookies or set up a proxy that doesn’t let my browser connect to those URLs.

But this Facebook thing is different, because it’s highly likely that the communication back to Facebook is not coming from my browser, but from the servers of sites like io9 and MapMyRun. And I have no way to stop it.

How about using a different email address for each site. Sure, that could work. If I had my own mail server, I could create “io9@nelzserver.xxx” for io9 and “mapmyrun@nelzserver.xxx” for MapMyRun, but how many people will have access or technical ability for something like that?

Gmail has a feature that could help out here. If I have an email address like “example@gmail.com”, I will still get the mail if it is sent to “example+io9@gmail.com” or “example+mapmyrun@gmail.com”. I found two challenges with this pattern: 1. it can break the “Forgot My Password” functionality if you forget what email you signed up with, and 2. emails with “+” in them don’t always pass the (incorrect) regex‘s sites use to validate emails.

Conclusion

As I have said before, I am living life pretty openly on the internet. And like I said above, I know there are countless other companies doing much more specific tracking and profiling of me.

But Facebook’s hegemonic desires to run the internet frustrate me, and I don’t want to be a part of it. And it pisses me off that there’s nothing I can do to extract myself.

Update: Found this interesting blog post – “Dark Stalking on Facebook“

So, here is the short version:

1. Have Rocking Goals
2. Be Ambitious
3. Invest In Yourself
4. Healthy Lifestyle
5. Be Responsible
6. Patience Is The key
7. Never Give Up
8. Learn From Failures
9. Remain Positive
10. Celebrate Success

Normally at this time of year, I would be in full-on packing mode for Burning Man. Alas, my best friend from college decided to get married on the day the Man burns (in the U.K.). Then, to top it all off, my brother decided to get married on the 18th of September in Boston. This left me with little to no obligations from the 7th to the 17th of September, so I decided that I was going to rent a motorcycle and tour around the south of France.

What does a techie do when they plan a trip? They buy tech gadgets of course!

My first purchase was a netbook. I got an Asus EeePC 901 (Linux). I opted for the white model, because there was a $300 (100%) ‘black tax’. I opted to get some scull vinyl stickers ($13) instead. I dumped the default Xandros OS and installed Ubuntu.

My original thought was that I didn’t want to be tempted to work on work or side projects while I was traipsing around Europe, so I got a solid netbook that could provide me with email/Twitter/basic web surfing, but would lend itself to opening up an IDE. After a while, I realized that I should reserve my vacation for things I *want* to do, even if it includes working on a software
project. So, I’ve decided to forget the netbook and take my home laptop, a 15″ MacBook. (If you are reading this and want the netbook I described above, I have one for a decent price if you’d like it.)

The second bit of tech that I purchased was a Kindle (2nd gen, non-DX). I do enjoy ‘personal’ reading, but rarely take the personal time for it (except on vacations) because I’m usually caught up in some technical book. My usage of it has waxed and waned since my purchase, but it definitely addresses my criteria of reducing the amount of ‘stuff’ that I am carrying whilst on-motorcycle in France. I do plan to put some entertaining books on my Kindle before I leave. (“Necronomicon” anyone?)

So, I’m down to my Kindle and my MacBook. What about my G1 mobile phone? Well, the surprising thing is that I’ll probably just stick to my current provider (T-Mobile). I was aghast when I heard their prices (something like: $0.99/min of phone usage, $0.35 per sent text message, free received text message, $0.35 ‘connection charge’ [for every time the phone checks for messages?], $15.00 per Megabyte of data), but in my subsequent research that’s about par for the course for prepaid SIM cards. I don’t like it, and I’m dubious that this is as good as it gets, but I don’t have the time or the inclination to fight
it too hard. I just have to accept that my Twitter addiction may cost me some. One of the positives though, is that when I told T-Mobile that I was considering using GSM cards purchased abroad they offered to help me unlock my phone. That should be a boon for me when I decide to replace it in the future and want to sell it.

I’m not planning on taking a camera, other than my G1 and my laptop. I am kind of hoping to take some videos using my MacBook’s built-in camera, but we’ll see if that even happens, because for most of my life I’ve been a forgetful and shitty photodocumentarian, and I don’t see that magically changing for this trip.

During my ‘where the wind blows’ tour of France, I’ll probably use CouchSurfing.org to meet people, or possibly even find shelter if I can plan that far in advance, but I’m purposly keeping my schedule unplanned so I don’t know if I’ll want to subject CouchSurfing hosts to my slapdash planning. The housing may just end up being, in order of preference: couches, hostels, B&B’s, or hotels. Like I said, I going *very* unplanned. I think I should be okay, because the annual French
holiday season is mostly done by the beginning of September. If not, it’ll be an exercise in creativity, right?

I’ve got 3 projects to work on while travelling. Firstly, I have visions of a Simple-Spring-Memcached v2, and I’ve already made a bit of progress starting that. Next, there is my as-yet-unannounced Ruby-on-Rails, Twitter based web service. And thirdly, I’ve decided that I want to do a presentation at Silicon Valley Code Camp in October about Simple-Spring-Memcached, and I can work on that presentation for a bit if I’m bored.

So, that’s my story. If I do get around to photodocumenting (or videodocumenting) any of my trip, I’ll be sure to post a link here.

#1 (“Its not a relational database.”) is very, very important. You’ve really got to stretch your brain a bit to get out of the RDBMS headspace to see how it can be beneficial. Everything is stored as a string, so if you want to any kind of sorting with regards to numbers or dates, you have to format the values to be sorted lexicographically. (This reminds me of when I was working with Lucene.)

In addition to not having any kind of auto-increment (as mentioned in that blogs #5 point), there is no concept of an “increment” operation on a field. At first I was tempted to try for a Read-Increment-Update pattern, but if you think of how that would interact with Amazon’s concept of Eventual Consistency (which, for some inexplicable reason, I find absolutely hilarious), you will realize this is a sub-optimal pattern for data integrity. (The pattern I ended up using lends itself only to the specific interaction patterns I am designing for, so I won’t bore you with the details.)

Though there is no auto-increment, many blog posts have been published recommending UUID’s, which seems to be working for me as well.

As for that blogs #8 point, there is now a offered called Javascript Scratchpad for Amazon SimpleDB, which is just a small set of HTML/JS pages that you save locally and run in a browser. I think it’s actually a pretty cool lightweight and simple GUI tool to offer.

Looking at the price list, you might think that SimpleDB could get expensive fast. But, if you read carefully you’ll see the following: “Data transferred between Amazon SimpleDB and other Amazon Web Services within the same region is free of charge (i.e., $0.00 per GB).” That’s pretty cool. As long as whatever is hitting your SimpleDB instance resides on the Amazon cloud (in the same region), you don’t pay for data transfer.

I also gotta comment on the Java client provided by Amazon. Yeah, it works, but working with it is fairly “meh”. You get the source an everything for it, but it’s not Open Source and they definitely don’t give the impression that they’re looking for any contributions. I would’ve submitted a couple of lines of documentation for some of the methods at the very least.

I’ve heard some good things about the typica library, specifically that the interaction patterns with the library are nicer than the Amazon-provided library. There only seems to be one (maybe two?) developer working on it, but at least it’s hosted on Google Code so, unlike Amazon, maybe the developer would welcome contributions.

I haven’t used typica myself yet, but since I’m not too far down the other path, maybe I’ll check it out.

Background

I started this blog way back in the beginning of 2007. At the time, I wanted more than just a blog host, I wanted a host where I could deploy other Java webapps if I felt the need. (For other side projects, y’know.) Following the lead of Matt Raible, I chose KGBInternet to host a Roller (which wasn’t then, but is now an Apache project) blog instance. Keith, the proprietor of KGBInternet, is incredibly responsive and I’d have to say that his service was a great $20/month host where you get full command-line access and control over your JVM/Tomcat instances.

A couple of months ago, I decided to critically look how I use my different online providers.

I had been maintaining a cheap, $5/month GeekISP account for personal data SVN hosting/backup. Since most of my projects are Open Source (and usually hosted on Google Code), I realized I wasn’t really using my account, so I dropped it.

And when I critically looked at my usage of KGBInternet, I realized that I was only using it for maintaining my blog. That’s $240 (Candian) bucks per year for blog hosting, which could be done for much less by another provider. So, I decided to drop that account.

Getting Data Out Of Roller

The biggest problem I had was trying to get my blog content out of Roller. There is no “Export” functionality, which frustrated me to no end. I found a posts with a tutorial on how to export directly from the database, but that wouldn’t work from me because 1) I didn’t want to go through a PHP export process, and 2) my db entries are all in JSPWiki markup (via the Roller JSPWiki plugin). I found another post showing how to get my data out using Roller templates, but that wasn’t going to specifically work for me, since I wasn’t going to be hosting my own WordPress install. However, the concept of using the built-in Roller template capabilities had a bunch of potential.

At this point, I wasn’t yet sure if I was going to end up using WordPress.com or Blogger. My investigations proved that Blogger only likes to import/export in their own format, for which I could find no documentation. WordPress however has several import capabilities, so I figured I’d target WordPress, and maybe use some of the community WordPress -> Blogger converters if I needed to move my stuff over to there.

WordPress has its own WordPress eXtended RSS (WXR) syntax, but it’s not easy to get a definitive format information. WordPress will import from a MoveableType export file, which is actually very well-documented. This is what basically decided me on moving to WordPress, at least for the time being…

By referring to both the MoveableType documentation and the Roller templating guide, I was able to create the following template:

 1  #set($pager = $model.getWeblogEntriesPager())
 2  #set($map = $pager.getEntries())
 3  #foreach( $day in $map.keySet())
 4  #set($entries = $map.get($day))
 5  #foreach( $entry in $entries )
 6  TITLE: $entry.title
 7  AUTHOR: nelz9999
 8  DATE: $utils.formatDate($entry.pubTime, "MM/dd/yyyy HH:mm:ss")
 9  CATEGORY: $entry.category.name
 10  -----
 11  BODY:
 12  $entry.transformedText
 13  #foreach( $comment in $entry.comments )
 14  -----
 15  COMMENT:
 16  #if ("$comment.name" != "")
 17  AUTHOR: $comment.name
 18  #end
 19  DATE: $utils.formatDate($comment.postTime, "MM/dd/yyyy HH:mm:ss")
 20  #if ("$comment.email" != "")
 21  EMAIL: $comment.email
 22  #end
 23  #if ("$comment.url" != "")
 24  URL: $comment.url
 25  #end
 26  $comment.content
 27  -----
 28  #end        
 29  --------
 30  #end
 31  #end

One of the keys in the above template is Line #12. Instead of using “$entry.text”, I use “$entry.transformedText”, which applies all the JSPWiki formatting. (And, if you’re copy/pasting this template, be sure to update the “AUTHOR:” tag on Line #7 to something appropriate for you…) Keep in mind that this gets you one page of (published, not draft) content at a time, as defined by the “Number of entries to display on weblog” parameter in the Preferences -> Settings tab. You get to later pages of data by appending “?page=X” (0 being the default) to the URL: “http://nelz.net/roller/nelz/page/moveabletype.tmpl?page=1″.

It is then a simple matter of doing a “wget” for each page: “wget http://nelz.net/roller/nelz/page/moveabletype.tmpl?page=1 -O page1.txt”. If you have lots of posts, I’d recommend upping your entries/page setting, so you have less individual files to manage, but also beware of your file sizes getting too big… I left my entries/page setting at 10, and I ended up managing 14 files. Had I set it to 50 entries/page, it would have been only 3 files.

But, now you’ve got most of your content (not images and stuff tho) in a portable format!

Getting Data Into WordPress

After creating my WordPress.com account, it was pretty simple to get the data in, via the Tools -> Import menu. Now, I had 138 posts in Draft status. I tried using the bulk edit -> publish tool, but for some reason that wasn’t working for me. I published a couple by hand, but then I remembered back to something Neil Ford said at a conference I attended: “our computers get together and laugh at us at night because we keep doing their (repetitive task) jobs for them…” Well, I’ll be damned if I let my computer laugh at me!!

So, I popped open SeleniumIDE, and created a quick little automation to approve all the drafts in my queue, a single iteration of which looks like this:

<tr>
 <td>open</td>
 <td>/wp-admin/edit.php?post_status=draft</td>
 <td></td>
</tr>
<tr>
 <td>click</td>
 <td>link=Quick Edit</td>
 <td></td>
</tr>
<tr>
 <td>select</td>
 <td>_status</td>
 <td>label=Published</td>
</tr>
<tr>
 <td>click</td>
 <td>link=Update Post</td>
 <td></td>
</tr>
<tr>
 <td>waitForText</td>
 <td>//img[@alt='More stats']</td>
 <td></td>
</tr>

This worked swimmingly, and in 30 minutes or so, I published nearly 140 posts “by hand”.

Getting to know Wordrpress

Things that I like about WordPress:

• It is a much better interface than Roller
• “Pages”. I also use my blog as a place to keep an online version of my resume, and Pages allow me to exactly do that without having to use a ‘regular’ blog post that I update over time.
• WordPress automagically figures out how to map the old Roller URLs to the new ones. I was getting quite a few links to some of my recent posts, and I really happy that the links floating around the web-o-sphere will still work. (E.g. “http://nelz.net/roller/nelz/entry/velocity_conference_recap” ends up on the page “http://nelz.net/2009/06/25/velocity-conference-recap/“.)
• Setting up my own domain was absolutely painless.

Things that I haven’t yet decided on:

• You can’t use Google Analytics on a WordPress.com blog, but they do provide some of their own stats (and there are several other options as well). I haven’t yet decided if it’s better to have the stats easily available in my blog admin, or if I’d have preferred to keep all my analytics stuff together in Google.
• I haven’t yet purchase the $30/yr no ads upgrade. I haven’t seen any ads on my site as it is, so I’m not sure if I will actually need it?
• No specific control over robots.txt. Whereas I want 99% of my blog public, I want to prohibit Google from picking up my resume, ‘cuz I kinda hate cold-calls from recruiters (but maybe I’m just being silly).

To Do

• Some of the formatting (especially on the preformatted code sections) got lost in the migration… I’ll try to go back to the most used ones to reformat.
• I gotta update that resume of mine.
• Once I’ve been up on WordPress for a week or two, I’ll actually cancel my account with KGBInternet.

General Themes

• I had never heard of it before, but almost every single presenter referenced Ganglia as a de-facto monitoring system.
• It got presented a bunch of different ways, but basically all the big sites that presented stuff today all use on and off (or dial-able) configurations for features. This is not just for release-time of new features, but this can also help them manage their capacity if something is going wrong.
• Many of these talks are available online: http://velocityconference.blip.tv/

“Image Weight Loss Clinic” at Ignite Velocity

• Stop using GIFs. Use PNGs.
• Use data strippers/filters on JPGs. There is a lot of ‘extra’ data included in JPG that aren’t necessary.
• There are bunches of PNG optimizers out there. We should use at least one, if not all of them. (The suggestion was to build a serial pipeline for them.)
• Using the Velocity page as an example, the presenter was able to reduce the page weight by 30% following these suggestions.

The User and Business Impact of Server Delays, Additional Bytes, and HTTP Chunking in Web Search

• A lot of people loved the empirical data showing that slower sites cost you users, even for differences as small as 200ms. Brady Forrest wrote up a great digestion of this talk: “Bing and Google Agree: Slow Pages Lose Users“
• The technique that I pulled outta the whole deal is to use HTTP 1.1 Chunked data. This enables a site to deliver the easy-to-compute stuff first (static header?), and the harder-to-compute stuff later.

Fixing Twitter: Improving the Performance and Scalability of the World’s Most Popular Micro-blogging Site

• Uses NTT America managed hosting
• Put Google Analytics on 503 (Fail Whale) and 500 (Robot) pages. Use Google Analytics for failure metrics.
• Configuration Management: Do it ASAP, early & often, ‘cuz you’re gonna need it eventually
• Even their Ops stuff is checked into SVN, and they require code reviews on all their stuff, enforced by SVN pre-commit hooks and using Review Board
• Send emails (or ANYTHING ELSE POSSIBLE) asynchronously
• They recommend using “mkill“, which monitors for long-running queries and kills them, before the queries kill your site.
• Instrument EVERYTHING for timing/performance.

2 Years Later, Loving and Hating the Cloud

• Presented by an engineer from Picnik. They run a hybrid (part-cloud, part managed) site.
• Queues scale nicely in AWS. (I.e. if you are falling behind processing your queue, it is nigh trivial to just bring up another box to deal with the queue.
• In the cloud, you can plan for your average usage, and scale up/down as needed easily. (You don’t need to keep the 6th box at 1% utilization up, do you?)
• Buy hardware in batches, it gives you flexibility. No scrambling if you need a new box when there are extras around. Also waiting for good deals on price fluctuations on hardware.
• If you don’t have a good deletion plan on S3, it can end up costing you $$
• Being in the cloud enables you to ignore the S3 space problem, operationally at least, until it is too expensive (leaving you opportunity to work on other low-hanging fruit)
• Whereas you can get some nice SLA’s when dealing within your own network, latency should be treated as a complete unknown in the cloud.
• Be prepared for some difficult and juicy debugging when using the cloud.

Page Speed

• Twitter is a fantastic feedback mechanism, more so than Google Groups / wikis / forums (me: lower barrier for commentary?)
• Browser Tool like Firebug
• Someone (on Twitter?) made a very apt comment that it’s kinda sad to see Google (Page Speed) and Yahoo! (YSlow) shepherding similar projects, without trying to combine them.

10+ Deploys Per Day: Dev and Ops Cooperation at Flickr

• Websites pretty much always ship trunk. Having versions and point releases are vestigial remnants from old shrink-wrapped product lifecycles.
• “Dark launches”, where you use the on/off/variable conditionals to exercise the new backend before it becomes mission/feature-critical.
• Have all deployments notify IRC/IM/Twitter (to internal teams only) so EVERYONE knows what’s going on. Also, keep it around w/timestamps, and make it searchable
• Give ALL developers (at least read-only) access to the prod machines. It helps them help you (Ops) better.
• If there is an outage, EVERYONE stops working on new work. Even they aren’t directly responsible, JR engineers should be working to understand why something is broken. This is a good time for them to learn these diagnostic skills.
• AUTOMATE your INFRASTRUCTURE!!

Scaling for the Expected and Unexpected

• ‘Planned Degradation’ – switch off functionality, this can lighten the load on the back end
• If you hit high (un)expected load it is usually on a single/few page(s), route to a static copy of that page, regenerate every X minutes.
• The simple act of using a proxy server between your appServer and the outside world, even if it is not caching (like Squid/Varnish), is that the appServer is just delivering to a network neighbor, reducing its thread pool contention. This simple fact can have a great positive effect on your server performance.
• Watch out for cache stampedes.
• 3rd Party Resources – Load last, place at bottom of page, in an iframe. If sales doesn’t like it, tell them to go to hell. (Me: whoa.)

Infrastructure in the Cloud Era

• (Me: There were some great slides here, I hope they post them publicly.)
• With provisioning becoming so quick (minutes), we need a quicker way to get these provisioned machines up and running quickly to realize those benefits.
• The real benefit of the cloud is not $$, it is TIME (which you can turn into $$).
• Definition – meatcloud: the humans that run your cloud presence. Noticeably difficult and slow to provision a new resource in your meatcloud.
• A bit of operational philosophy – once you get your provisioning/setup all automated and quick, if a service is misbehaving have a bias towards killing it and recreating an instance, rather than trying to ‘recover’ the problem box.
• When you’ve got Command & Control systems in place, they also need an on/off switch, because sometimes you *do* need to do some manual stuff.

Ajax Performance

• Modify your nodes before you attach them to the DOM. Modifying them after can trigger cascading re-parsing by the browser.
• While most languages have Optimizers, JavaScript doesn’t. You should remove your own common subexpressions / loop invariants / etc.
• Prefer “[array, of, strings].join()” over “array + of + strings” because the “+” operator builds lots of spurious interstitial objects

Building OpenDNS Stats

• This talk was about a High-Write environment, which isn’t as applicable to Gallery, but is applicable to our Metrics app
• Don’t use auto-increment in a high-write environment, as it does a table_lock

Load Balancing Roundup

• This presenter is a committer for Perlbal and was very up-front that this talk would be heavy on the praise for it.
• Graphs that look at load every 30 seconds or more DON’T give you enough info about load on your server. Presenter suggests you watch “top -d 0.5″ for a while to get an idea of your server’s load.
• Presenter and audience agreed that HAProxy doesn’t work with “keep-alive”

I submitted the talk "Adopting SF Prankster Culture – One geek’s 10-year journey to find an outlet for his social creativity." I have posted the slides, but unfortunately those slides don’t convey a lot of information without my running commentary. I would have posted a video, but there was some problem with the video recording at the event, so I don’t think I’ll have access .

It was a really fun time. I did get a bit nervous ahead of time, but I just had to remind myself that I had just jumped out of a plane two days earlier. With that perspective, I was able to take the stage pretty confidently.

I did have a bit of a false start though, because the slides that were on the monitors right when I started were for another presenter. I did my best to bullshit my way through a slide or two until the organizers figured out the glitch. I think this actually softened up the audience with a bit of humor, which got them ready for my talk which I gave one deck later.

Doing the talk had some unexpected benefits too… For the whole next day that I was at the Velocity conference sessions, people would come up to me in the hallways and congratulate me for my presentation. I felt kinda like a geek rock star!

So if you have a chance, I would totally recommend doing an Ignite talk. I’m glad I got to ‘cut my teeth’ doing a talk on a social concept as practice in case I end up making a presentation on any of my other more technical projects.

It is in times like these when we should take time to appreciate what we have. I feel very fortunate to be gainfully and happily employed at Widgetbox. Will and Giles and our Board of Directors have done a great job of managing our funding, so much so that I know I have job for at least two more years. I daily experience a bit of cognitive dissonance when I realize that my position in a startup is more secure than many people in established companies.

I have been running around wrapped in my cozy little security blanket of a job. Sure, I’ve acknowledged that the economy is tough for a lot of people. I’ve even been trying to do my part by spending in local businesses. But the severity of the economic situation didn’t really rock me to my core until last night.

I was having a drink with a friend who does fine finish carpentry, and we started discussing how difficult things are out there. It turns out he is actively thinking about re-enlisting in the Navy after nigh 20 years away. He feels this is one of the few choices left to him to support his family. This really shook me up. Tonight, I called some of my best friends to quiz them on how they are set for maintaining their mortgage over the coming shaky financial times, because I needed to know that they are likely to be alright.

But, this is not just a time to buckle down, count my pennies, and continue living in the relative comfort that I have achieved. It is a time for action.

I feel a need to bring my blessings to bear on this economic crisis. I look about me to take stock of what I have. Whereas I am financially stable, I don’t have the financial wherewithal to employ anyone else, nor to pay for their housing. But, I do have quite a bit of space in this great apartment of mine, and I know that my privacy needs are flexible enough to deal with a shared-living-space arrangement…

Here is my proposal: If you are in my extended circle of community (Burners, techies, Twitterati, etc.) and you have suffered a financial setback (such as a layoff) and you could use some rent-relief to get back on your feet, you should get in touch with me. I am open to the opportunity of sharing my space with a fellow community member for low (or maybe even no) rent.

I don’t know what’s going to happen from this. Maybe no one wants to share a space with me? Maybe the economy makes a sharp upswing? Maybe, maybe, maybe… This is what I have to offer, and I’ll be glad if I can help someone out.

PS. I gotta give a shout-out to Toyota in this economic climate. If I were buying a vehicle, I’d prefer to support a company that operates in accordance with the very American ideals of education and recognition of service, rather than American-based companies that wallow in mismanagement and short-sightedness.

I have been in the thick of it at Widgetbox, and other than a bit of adjustment time in the other parts of my life I have been rockin’ and rollin’!

My biggest accomplishment has been implementing MemCacheD as our distributed cache. I’ve even submitted a couple of patches to the Spy MemCacheD Java Client. (Dustin, who wrote an maintains that client is a pretty cool cat with loads of talent!)

So.. Yeah… I’m not apologizing. I guess maybe I’m gloating?!?