LD.

Music, software, life… and stuff.

[ Twitter ] [ GitHub ] [ Linked In ]

Annotation Driven Extensions With Spock

The following is an introduction to writing extensions with everybody’s favourite testing tool Spock. Like all things in Spock, extensions are easy to use and require little ceremony. We are going to build an extension that times various parts of test execution and blindly writes the times to standard out.

Please note that the API we are going to go through should not be considered stable/final.

We are building against Spock 0.5-SNAPSHOT and the following APIs may change on the road to 1.0.

Annotation Driven

Our extension is going to be annotation driven. We are going to provide an annotation that users can place on various parts of their specs. At the core of this is the IAnnotationDrivenExtension contract…

package org.spockframework.runtime.extension;

import java.lang.annotation.Annotation;
import org.spockframework.runtime.model.*;

public interface IAnnotationDrivenExtension<T extends Annotation> {
  void visitSpecAnnotation(T annotation, SpecInfo spec);
  void visitFeatureAnnotation(T annotation, FeatureInfo feature);
  void visitFixtureAnnotation(T annotation, MethodInfo fixtureMethod);
  void visitFieldAnnotation(T annotation, FieldInfo field);
  void visitSpec(SpecInfo spec);
}

You’ll notice that (almost) all of Spock’s internals are implemented in Java. In this exercise we will implement our extension bits in Groovy.

Annotation driven extensions implement the Visitor Pattern, and Spock will invoke your visitation method for each annotation of your type present on the spec (We’ll discuss the visitSpec() method later).

• visitSpecAnnotation() - is called when the spec class is annotated.
• visitFeatureAnnotation() - is called for each “feature” (i.e test method).
• visitFixtureAnnotation() - is called for each of the setupSpec(), setup(), cleanup() and cleanupSpec() methods that are annotated.
• visitFieldAnnotation() - is called for each spec instance variable.
• visitSpec() - is always called after all of the visitation methods. This allows you to collect a model while visiting, then decorate the spec in some manner.

These methods all take the annotation instance and an info object which describes the annotated thing. For more detail on each of these info types above what we will go through in this exercise, check them out in the Spock source. They are pretty straightforward.

The Annotation

We need to wire our annotation up to our extension. We declare our annotation like normal, but annotate it with the IAnnotationDrivenExtension implementation that backs it.

import java.lang.annotation.*
import org.spockframework.runtime.extension.ExtensionAnnotation

@Retention(RetentionPolicy.RUNTIME)
@Target([ElementType.TYPE, ElementType.METHOD])

@ExtensionAnnotation(TimingExtension)

public @interface Time {}

Here we have declared a new annotation called @Time that takes no parameters and can be placed on types (in our context, the spec class) and methods. The @ExtensionAnnotation meta annotation is how tell Spock who deals with the @Time annotation.

The Extension

Now we need to actual implement our extension. As you would expect, Spock provides a AbstractAnnotationDrivenExtension super class that we can use. This allows us to only implement the methods we need to. In our case, we don’t need the visitFieldAnnotation() method.

import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension
import org.spockframework.runtime.model.SpecInfo
import org.spockframework.runtime.model.FeatureInfo
import org.spockframework.runtime.model.MethodInfo

class TimingExtension extends AbstractAnnotationDrivenExtension<Time> {
  def timeSpec = false
  def timedFeatures = []

  void visitSpecAnnotation(Time annotation, SpecInfo spec) {
    timeSpec = true
  }

  void visitFeatureAnnotation(Time annotation, FeatureInfo feature) {
    timedFeatures << feature.name
  }

  void visitSpec(SpecInfo spec) {
    spec.addListener(new TimingRunListener(timeSpec, timedFeatures)
  }
}

Here we simply collect information on which things we are timing, then add a run listener (explained below) that will do the timing.

The implementation of the visitation methods in AbstractAnnotationDrivenExtension always throw an org.spockframework.runtime.InvalidSpecException. Our annotation can legally be placed on any fixture methods according to it’s definition, but that doesn’t make sense in the context of our extension. Because we have not overridden the visitFixtureAnnotation() method in our extension, if the user annotates a fixture method with @Time, they will get a nice error message from Spock that tells them that that is invalid.

It’s important to consider that there is guaranteed to only ever be one instance of an extension for each spec. Spock will create an instance of your extension the first time it encounters your annotation, and then reuse that instance when it encounters later annotations (in the same spec).

Listening In

Spock has it’s own execution listening API specified by IRunListener.

package org.spockframework.runtime;

import org.spockframework.runtime.extension.builtin.StepwiseExtension;
import org.spockframework.runtime.model.*;

public interface IRunListener {
  void beforeSpec(SpecInfo spec);
  void beforeFeature(FeatureInfo feature);
  void beforeIteration(IterationInfo iteration);
  void afterIteration(IterationInfo iteration);
  void afterFeature(FeatureInfo feature);
  void afterSpec(SpecInfo spec);
  void error(ErrorInfo error);
  void specSkipped(SpecInfo spec);
  void featureSkipped(FeatureInfo feature);
}

This is all pretty straightforward, but you may be wondering about beforeIteration() and afterIteration(). This are for parameterised (or data-driven) feature methods and are called for each set of parameters.

Again, we have convenient super class to work with.

import org.spockframework.runtime.AbstractRunListener;
import org.spockframework.runtime.model.*;

class TimingRunListener extends AbstractRunListener {

  def timeSpec = false
  def specStartTime
  def featureStartTimes = [:]

  TimingRunListener(timeSpec, timedFeatures, timedFixtures) {
    this.timeSpec = timeSpec
    timedFeatures.each { featureTimes[it] = null }
  }

  private now() {
    System.currentTimeMillis()
  }

  private start(name) {
    println "starting $name …"
  }

  private done(name, started, finished = now()) {
    println "$name took ${finished - started} milliseconds"
  }

  void beforeSpec(SpecInfo spec) {
    if (timeSpec) {
      specStartTime = now()
      start("spec '$spec.name'")
    }
  }

  void beforeFeature(FeatureInfo feature) {
    if (featureStartTimes.containsKey(feature.name)) {
      featureStartTimes[feature.name] = now()
      start("feature '$feature.name'")
    }
  }

  void afterFeature(FeatureInfo feature) {
    def startedAt = featureStartTimes[feature.name]
    if (startedAt) {
      echo("feature '$feature.name'", startedAt)
    }
  }

  void afterSpec(SpecInfo spec) {
    if (specStartTime) {
      echo("spec '$spec.name'", specStartTime)
    }
  }
}

Our extension is now complete… let’s try it out.

In Action

Now all we have to do is annotate our spec…

import spock.lang.*

@Time
class DoingStuff extends Specification {

  @Time
  def "a b c"() {
    when:
    Thread.sleep(1000)
    then:
    true
  }

}

If we were to run this spec we would see something like…

starting spec 'DoingStuff' …
starting feature 'a b c' …
feature 'a b c' took 1000 milliseconds
spec 'DoingStuff' took 1500 milliseconds

Obviously the times are made up but you get the point.

The Real World

Spock ships with a bunch of extensions (as of today in 0.5-SNAPSHOT):

• @AutoCleanup - allows a named cleaning up/closing method to be automatically called by Spock on a field
• @FailsWith - makes a feature pass only if it raises a certain exception and it’s uncaught
• @Ignore - prevents the feature or spec from running
• @IgnoreRest - prevents every other feature from running
• @RevertMetaClass - undoes any changes to the specified to meta classes of the specified classes during the feature (or spec)
• @Stepwise - causes each feature method to be executed in declaration order and any after the first failure to be ignored (i.e. story mode)
• @Timeout - causes a method to fail if it takes to long (and will interrupt it if it does take too long)

For more info on any of these extensions, check out their javadoc comments.

You could very easily make your own extensions specific to your projects to make your life easier. You could load data fixtures, establish connections… all kinds of things.

Summary

So that’s “Annotation Driven Extensions” with the Spock Framework.

Go forth and create some… and may the force be with you (wait… what?)

Posted: Aug 18th, 2010 @ 10:58 pm

Tags: #groovy spock  

Archive · RSS · Theme by Autumn