Nelz's Blog

Mah blogginess

Google App Engine in Maven + IntelliJ

At Widgetbox, I sometimes get to play around with interesting technologies that are outside of our regular stack. A couple of weeks ago, I was asked to use Google App Engine’s Java environment (GAE/J) to prototype a resizing image proxy.

At first, I just developed the prototype in the default GAE/J Eclipse environment until I could deliver a functional POC. After finding the GAE/J capabilities more than adequate for what we wanted to do, I was challenged to bring the project into our standard IntelliJ + Maven development environment. For the rest of this post, I’ll share a couple of tips and tricks for getting your GAE/J project to operate in this environment.

Basic POM File

There’s some funny business and frustration around the Maven community’s adoption of GAE/J, but I’ll skip that part of the story for right now. What I found is that the maven-gae-plugin project is the best place to go to for help Mavenizing a GAE/J build.

I have to say that it’s not ‘use the archetype’ easy (their archetype failed for me), but with a bit of elbow-grease and rummaging through their documentation I was able to get a decent and functional POM file built. Here it is (with some of our proprietary information scrubbed to protect innocent servers):

  1 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  2          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  3   <modelVersion>4.0.0</modelVersion>
  4   <groupId>com.widgetbox</groupId>
  5   <artifactId>image-proxy-webapp</artifactId>
  6   <version>1.0-SNAPSHOT</version>
  7   <name>Widgetbox :: Image-Proxy :: Webapp</name>
  8   <packaging>war</packaging>
  9   <properties>
 10     <gae.version>1.3.0</gae.version>
 11     <gae.app.name>qa-image-proxy</gae.app.name>
 12   </properties>
 13   <dependencies>
 14     <dependency>
 15       <groupId>javax.jdo</groupId>
 16       <artifactId>jdo2-api</artifactId>
 17       <version>2.3-eb</version>
 18       <exclusions>
 19         <exclusion>
 20           <groupId>javax.transaction</groupId>
 21           <artifactId>transaction-api</artifactId>
 22         </exclusion>
 23       </exclusions>
 24     </dependency>
 25     <dependency>
 26       <groupId>javax.transaction</groupId>
 27       <artifactId>jta</artifactId>
 28       <version>1.1</version>
 29     </dependency>
 30     <dependency>
 31       <groupId>com.google.appengine.orm</groupId>
 32       <artifactId>datanucleus-appengine</artifactId>
 33       <version>1.0.4.1</version>
 34     </dependency>
 35     <dependency>
 36       <groupId>org.datanucleus</groupId>
 37       <artifactId>datanucleus-core</artifactId>
 38       <version>1.1.5</version>
 39       <exclusions>
 40         <exclusion>
 41           <groupId>javax.transaction</groupId>
 42           <artifactId>transaction-api</artifactId>
 43         </exclusion>
 44       </exclusions>
 45     </dependency>
 46     <dependency>
 47       <groupId>com.google.appengine</groupId>
 48       <artifactId>datanucleus-jpa</artifactId>
 49       <version>1.1.5</version>
 50       <scope>runtime</scope>
 51     </dependency>
 52     <dependency>
 53       <groupId>com.google.appengine</groupId>
 54       <artifactId>geronimo-jpa_3.0_spec</artifactId>
 55       <version>1.1.1</version>
 56       <scope>runtime</scope>
 57     </dependency>
 58     <dependency>
 59       <groupId>com.google.appengine</groupId>
 60       <artifactId>appengine-api-1.0-sdk</artifactId>
 61       <version>${gae.version}</version>
 62     </dependency>
 63   </dependencies>
 64   <build>
 65     <plugins>
 66       <plugin>
 67         <groupId>org.apache.maven.plugins</groupId>
 68         <artifactId>maven-compiler-plugin</artifactId>
 69         <version>2.0.2</version>
 70         <configuration>
 71           <source>1.6</source>
 72           <target>1.6</target>
 73         </configuration>
 74       </plugin>
 75       <plugin>
 76         <groupId>net.kindleit</groupId>
 77         <artifactId>maven-gae-plugin</artifactId>
 78         <version>0.5.3</version>
 79       </plugin>
 80       <plugin>
 81         <groupId>org.apache.maven.plugins</groupId>
 82         <artifactId>maven-war-plugin</artifactId>
 83         <version>2.1-beta-1</version>
 84         <configuration>
 85           <filters>
 86             <filter>${project.build.directory}/version.properties</filter>
 87           </filters>
 88           <webResources>
 89             <resource>
 90               <directory>src/main/external</directory>
 91               <targetPath>WEB-INF</targetPath>
 92               <filtering>true</filtering>
 93             </resource>
 94           </webResources>
 95         </configuration>
 96       </plugin>
 97       <plugin>
 98         <groupId>org.apache.maven.plugins</groupId>
 99         <artifactId>maven-antrun-plugin</artifactId>
100         <version>1.3</version>
101         <executions>
102           <execution>
103             <phase>compile</phase>
104             <configuration>
105               <tasks>
106                 <echo file="${project.build.directory}/version.properties">
107                     friendlyversion=${project.version}
108                 </echo>
109                 <replace file="${project.build.directory}/version.properties" token="." value="-"/>
110                 <replace file="${project.build.directory}/version.properties" token="SNAPSHOT" value="snapshot"/>
111               </tasks>
112             </configuration>
113             <goals>
114               <goal>run</goal>
115             </goals>
116           </execution>
117         </executions>
118       </plugin>
119     </plugins>
120   </build>
121   <repositories>
122     <repository>
123       <id>maven-gae-plugin-repo</id>
124       <name>maven-gae-plugin repository</name>
125       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
126     </repository>
127   </repositories>
128   <pluginRepositories>
129     <pluginRepository>
130       <id>maven-gae-plugin-repo</id>
131       <name>maven-gae-plugin repository</name>
132       <url>http://maven-gae-plugin.googlecode.com/svn/repository</url>
133     </pluginRepository>
134   </pluginRepositories>
135 </project>

(FYI, we’re not actively using any datastore functionality just yet, so if you are going to use this template please forgive me if those dependencies are a little bit wonky.)

Since Google hasn’t (yet) decided to publish their development environment in a Maven-friendly way, there’s a bit of dependency wonkiness involved in getting the maven-gae-plugin to work. I included the repository information required by the plugin (lines 121 – 134), but if you use a repository manager (like Nexus), you’ll want to remove those lines from the POM and add a proxy for the maven-gae-plugin’s repository.

To get the development environment working the plugin also requires access to the unzipped SDK as packaged by Google. The plugin tries to help you set this up (“gae:unpack”) but that failed for me. I was able to get stuff working by manually unzipping the SDK artifact downloaded directly from Google to the following directory:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0

Incremental Improvments

Initially, I had kept the appengine-web.xml within the WEB-INF directory, but I realized I could make our Release Manager’s life a bit easier if I added a bit of build-time substitution.

Here’s our appengine-web.xml:

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <application>${gae.app.name}</application>
    <version>${friendlyversion}</version>
    <system-properties>
        <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
    </system-properties>
</appengine-web-app>

Directory Structure

And you’ll see that I put it into a new source directory called ‘external’:

At build time, I use the AntRun plugin (lines 97-118) to create a small file under the target directory that holds a ‘sanitized’ version of the standard Maven version. (I.e. “1.0-SNAPSHOT” becomes GAE-friendly “1-0-snapshot”.) I then use the Maven filter functionality available in the WAR plugin (lines 80-96) to copy the appengine-web.xml into its proper directory with the version substituted in.

You’ll also notice in our appengine-web.xml that we substitute in our application name. By default this comes from the properties section of the pom.xml file (line 11). I did this because we’ve actually got 2 different applications up on GAE’s servers, the QA version and the Production version. By default we build using the QA server’s application name, but when our Release Manager is building to upload to Production, all that is needed is an additional command-line argument of “-Dgae.app.name=<prod-name>”.

Running, Debugging, and Deploying

The two most valuable targets that maven-gae-plugin provide are “gae:run” and “gae:debug”. These will assemble your code in the standard Maven webapp target directories and run your app. (Note: “gae:debug” didn’t actually work for me until the 0.5.3 version of the plugin.)

There is also a “gae:deploy” target that is supposed to invoke the Google-supplied shell script that will upload your application to the Google servers, but it failed for me several time. Since then, I’ve defaulted to using the shell script directly to deploy my app once it has been built:

~/.m2/repository/com/google/appengine/appengine-java-sdk/1.3.0/appengine-java-sdk-1.3.0/bin/appcfg.sh \
    update \
    ./target/myApp-1.0-SNAPSHOT

Results

So, this is how we got up and running with GAE/J in our standard development environment. Hopefully this post ends up helping people out to reduce their bootstrap time when evaluating/investigating GAE/J for their own uses.