Today, I set up a Maven module to run SeleniumRC (Java) tests via TestNG. It wasn’t a straightforward or easy task, so I thought I’d share some of what I found with you.
We opt to have all of our modules stay in lock-step version-wise, so we keep a pretty large structure in our SVN trunk. We include/exclude modules for compilation by using Maven profiles, so we don’t have to build the whole structure if we are only changing one small component. Our top-level pom.xml file looks similar to this:
<project>
<groupId>...
<artifactId>...
<version>...
<packaging>pom</packaging>
<modules>
<module>baseModule</module>
</modules>
<profiles>
<profile>
<id>all</id>
<modules>
<module>otherModule</module>
...
<module>web-test</module>
</modules>
</profile>
<profile>
<id>web-test</id>
<modules>
<module>web-test</module>
</modules>
</profile>
</profiles>
</project>
As you can see, running “mvn clean install” at this top level would only end up building the baseModule. To get all the modules, we run “mvn clean install -Pall” which builds baseModule along with otherModule and web-test and anything else as represented by the ellipse. The web-test module is the one we will be interested in.
For our heavy-weight tests (non-unit tests, like functional or web testing), we like to have the module compile, but not run unless we specifically trigger the tests. To achieve this, we create a pom.xml (for the web-test module) that looks something like this:
<project>
<parent>...
<artifactId>web-test</artifactId>
<packaging>pom</packaging>
<build>
<testSourceDirectory>src/it/java</testSourceDirectory>
<testResources>
<testResource>
<directory>src/it/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
<executions>
<execution>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<testSourceDirectory>src/it/java</testSourceDirectory>
<excludedGroups>web-test</excludedGroups>
</configuration>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
This pom has several features:
- It sets up src/it/java as the location holding the source code for the tests
- It sets the surefire plugin (which runs your tests) to only run at the integration-test part of the Maven lifecycle
- It says to ignore any tests in the TestNG “web-test” group
- Note the packaging says this module is of the type “pom”. I found that if this is set to “jar”, the tests run twice for some unknown reason
Now, if you had tests in src/it/java (that were all annotated as “web-test”) and you ran “mvn clean install”, you’d see that zero tests ran. This is what we want! Now, we have to get the tests running if we pass a specific profile in. To achieve this, I added the following profile definition to the web-test pom.xml.
<profiles>
<profile>
<id>run-web-test</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>src/it/resources/testng.xml</suiteXmlFile>
</suiteXmlFiles>
<!--<groups>web-test</groups>-->
<!--<forkMode>always</forkMode>-->
</configuration>
<executions>
<execution>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
This profile is invoked when you run “mvn clean install -Prun-web-test”. There are several things I’d like to point out here:
- At runtime, this overrides the previously-declared surefire plugin configuration, but only when invoking the profile.
- Originally, I was mostly getting stuff to work without the <suiteXmlFiles /> section, but with the <groups /> and <forkMode /> sections… But it was running every single test class as it’s own TestSuite.
- To fix the multiple TestSuite (and @Before*, @After* problems that I’ll discuss later), I ended creating my own testng.xml file. Having this file and using the <suiteXmlFiles /> element made things a whole lot easier for configuring my TestNG executions than trying to configure the tests through the clumsy interference of the surefire plugin.
- As much as I like Maven, I have to say that the lack of documentation and general obfuscation that happens when using their plugins can really drive a person BATTY!
Now, I need to start integrating the stuff I need for using SeleniumRC. I found some decent help on how to start up and shut down the selenium-server, however I disagree with using a post-integration-test phase to shut down the server, because if the tests fail in the integration-test, Maven won’t progress to the next phase and you’ll be left with a spuriously running selenium-server. The client API’s do have access to the command that the was being run to shut the server down, so it is just a matter of having some TestNG @AfterSuite or @AfterGroup method issue the Selenium.shutDownSeleniumServer() command.
This brings me to some of the TestNG wonkiness I saw. I decided to use a Singleton Data Provider (named SeleniumFactory) to deliver the Selenium objects to the individual tests. (Selenium objects are time-expensive to create, so I wanted to only create 1 for the duration of the entire test suite.) I wanted to initialize the SeleniumFactory object in a @BeforeGroup method, deliver the Selenium object as a Data Provider, and then tear the SeleniumFactory down in a @AfterGroup method. (At first, I was fighting Maven’s handling of each test as its own test suite which was giving me sequence issues, which is why I ended up defining my own testng.xml file, as I stated above.) Let me start with some code to show you what I mean:
public class SequenceEndsTest {
@BeforeGroups(groups = {"web-test"})
public void startupSelenium() {
System.out.println("-- > Called startup.");
}
@AfterGroups(groups = {"web-test"}, alwaysRun = true)
public void cleanupSelenium() {
System.out.println("-- > Called cleanup.");
}
}
public class SeleniumFactory {
@DataProvider(name = "selenium")
public static Object[][] getSelenium() {
System.out.println("-- > Called data provider.");
return new Object[][] { new Object[] {null}};
}
}
public class SequenceTest {
@Test(groups={"web-test"},
dataProvider = "selenium", dataProviderClass = SeleniumFactory.class)
public void testSomething(final Selenium selenium) {
System.out.println("-- > Called test.");
}
}
This is what I expected to see after running this code:
-- > Called startup.
-- > Called data provider.
-- > Called test.
-- > Called cleanup.
But this is what I *actually* saw:
-- > Called data provider.
-- > Called startup.
-- > Called test.
-- > Called cleanup.
I would have assumed that an @BeforeGroup method got called before a data provider method, but I guess my assumption is wrong, eh? After I figured out this order-of-method-calls business, I was able to adapt my SeleniumFactory accordingly.
… And that is my story. To summarize: each of these tools could be better documented, but at the end of the day I was able to get a Maven module up and running SeleniumRC tests via TestNG. Now, on to the test-writing!
I was recently tasked with investigating a new Java 
