Here at Palantir, a lot of our automatic tests are full-chain tests. A backend server is fired up, client code runs against it, and everything runs much like a production environment. This makes intuitive sense because it’s a faithful approximation of how the system will run in the field.

However, there are some disadvantages to this:

• Full-pass tests don’t always localize the problem. Tests on a client class might fail even if it was the service that behaved incorrectly.
• These full-pass tests are relatively slow. Client code is running against an actual remote service. If a client is being tested, the server code still has to do work — sometimes a lot of work — even if that isn’t the focus of the test.
• The constraints of the test are loose. Full-chain tests can mostly only see whether the operation finished correctly. It’s much harder to figure out whether the operation was done efficiently and without making unnecessary service calls.
• They’re very little setup flexibility. If you want an RPC to return a specific value, you have little choice but to have your test get the service into a state where it can return that value. This is easy in some cases, but prohibitively difficult in others.
• Client tests are forced to share any non-determinism leaked from the service. For example, under real conditions, a request to call A might respond before call B, and sometimes the other way around. This can result in flaky tests or tests that don’t always simulate the conditions you want to exercise.

What’s to be done? Fortunately, there’s an option that handles these cases elegantly. We also test with jMock, a library that dynamically generates mock objects from arbitrary interfaces. These mock objects can be configured to check that particular methods are called with particular inputs a particular number of times, and then give prescribed responses.

Hit the link to see a concrete example of jMock in action.

jMock in action

Let’s say I want to test my object viewer page in Palantir Web, but I don’t want to fire up a dispatch server at all. First, I create my mock service object.

Mockery context = new Mockery();
final PalantirService service = context.mock(PalantirService.class);

Then, I set the expectations of my mock object. In this case, I want to tell my mock object to expect a call to PalantirService.getObject() and PalantirService.getDataSources(). getObject() will return a specific object. Any call made to the service apart from these will make the test fail.

context.checking(new Expectations() {{
        oneOf(service).getObject(realm.getId(), myObject.getId());
        will(returnValue(myObject));
        oneOf(service).getDataSources(myObject.getDataSources());
}});

Now, I create the object I want to test and inject the service.

ObjectViewController controller = new ObjectViewController();
controller.setService(service);

And then we fire away.

ModelMap model = new ModelMap();
controller.doGet(myObject.getId(), model);

Now that the controller (the class we’re exercising) has gone off and populated the model, we check to see that the model is populated correctly. Just like we would in any other test.

assertEquals(myObject.getName(), model.get("objectName"));
assertEquals(myObject, model.get("object"));

But in addition, we also assert that the expectations specified above were satisfied.

context.assertIsSatisfied();

Not only can we be sure that the right calls were made with the right parameters, but we can also be sure that no calls besides the expected calls were made. So the next time you want more speed or control over your tests, take a look at jMock or another framework like it. It’s a powerful tool in the effort to test your best!

Here at Palantir we use test-driven development (or TDD for short). Integrated tools like Eclipse and JUnit simplify writing and running unit tests. However, once you need to test a broader swath of functionality, it’s time to write functional, integration, and system tests. While technically not ‘unit testing’, the testing framework that JUnit provides is basically the same infrastructure that you want to leverage for writing these more involved types of testing.

When you’re developing enterprise software, functional testing often means getting your clients to talk to your servers. For the main Palantir Government product, we integrate the process of bringing the server up and down with the Ant scripts that run our automated unit tests: our testing tasks bring up the server, run the test suite, and then kill the server. This works great and produces nice results.

When I started working on our authentication server, the pattern that we had used before didn’t work for me. While the Palantir Government tests ran with a single, static configuration file, I needed to run the authentication server with multiple configurations in the course of running through the all the different functional tests. I determined that I needed a way to programmatically bring the server up and down for testing. In JUnit parlance, I needed a way to programmatically launch the server component as part of my setup() function for my unit tests and stop it in my teardown().

With my itch-to-scratch firmly in hand (or some other mixed metaphor), I set out to figure out how to invoke new Java processes from inside a unit test. The solution I came up with (with source code and examples) after the jump.

The Six Ingredients

So there are six ingredients that go into spawning a new VM:

• The classpath to use for the new VM
• The name of the class to run
• The directory to be used as the current directory for the process
• The command line arguments to pass to the process
• The set of Java system properties to use for this process
• The environment to pass to the process

Let’s look at each item individually.

Classpath

The classpath will tell the spawned VM where to load classes from. In JavaInvoke, we use the existing classpath (from the spawning VM) as a starting point and then prepend any new entries to allow overriding the classpath for the spawned VM.

This takes a lot of the tedium out of having to figuring out what to put in the classpath. Most likely, you want something similar to what you already have, if not completely identical.

We get the classpath from System.getProperty("java.class.path") and can add new entries by prepending the new entry, using the value of File.pathSeparatorChar as the entry delimiter. Using File.pathSeparatorChar makes the code cross-platform friendly (since the path separator is ‘;’ on Windows and ‘:’ on Unix (Linux, Solaris, OS/X, etc.).

Caveat: if you change the working directory and your original classpath was constructed using relative paths, you’ll probably have trouble getting anything to run (since your classpath will no longer point to right locations).

Class name

Pretty simple: what do you want to run in the spawned VM? The class must have a static void main(String args[]) defined, and it must be available for loading via the classpath.

Working Directory

If it should be different from the current working directory (CWD) of the running process, then set it and JavaInvoke will change it in the environment.

Command line arguments

If the process needs any command line arguments, including VM options, specify them in a string array. Note that not all of these arguments will necessarily make it to your main method, since the VM executable will parse it first and remove the VM arguments, passing through the program arguments.

Java System Properties

System properties can be used to control many aspects of how a VM runs. You can set them programmatically in your code or you can set set them on the command line by passing -Dkey=value. Our JavaInvoke implementation will take a Map of properties as a convenience argument; all it does is rewrite the map into the command line.

Process environment

This is an operating-system level construct. This is the set of environment variables, also in a Map that you would like merged with the current environment. This would be the place that you set things like LD_LIBRARY_PATH on Unix.

Dealing with input and output

So you might ask the question, “where does the output from the process go?” Or more troubling, “How do I send the process some input?” The Java Process object has methods to deal with this, allowing you to get streams that give you access to the input, output, and error streams of spawned process. That API is straight-forward to deal with, just like any other use of the java.io streams.

However, we want to make the typical case really easy: pulling the output from the spawned process back to the parent that spawned it. To that end, we add into the mix a class called OutputPiper. It fires up a thread that pulls all input from the spawned process, tags it with an identifier, and then outputs to the spawner’s stdout/stderr.

OutputPiper

(as extracted from ProcessSpawner.java)

	public static class OutputPiper extends Thread  {
		InputStream in;
		PrintStream out;
		String tag = null;

		public OutputPiper(String tag, InputStream in,PrintStream out) {
			this.in = in;
			this.out = out;
			this.tag = tag;
			// make sure that we don't keep the VM alive
			this.setDaemon(true);
			this.setName("OutputPiper-" + tag);
			out.println("Starting output piper for tag: " + tag);
			this.start();
		}

		@Override
		public void run() {
			try {
				BufferedReader reader = new BufferedReader(new InputStreamReader(in));
				String line = null;
				do {
					line = reader.readLine();
					if(line != null) {
						out.println(tag + ": " + line);
					}
				}while(line != null);
			}
			catch (Exception e) {
				//
			}
			out.println("Output piper exiting for tag: " + tag);
		}

		public static OutputPiper createOutputPiper(String tag, InputStream in, PrintStream out) {
			OutputPiper rc = new OutputPiper(tag, in,out);
			return rc;
		}
	}

Outpiper extends Thread so that all the output will arrive back to the controlling process in a timely manner. For each given process, we spawn off two OutputPipers, one for stdout and one for stderr, corresponding to the Process.getInputStream() and the Process.getErrorStream().

ProcessSpawner & JavaInvoke

There are two key classes in the example:

• ProcessSpawner.java – Essentially a wrapper around ProcessBuilder, a generic process spawner that makes it simple to invoke processes that that use OutputPipers to forward their output back to their parent. This class allows you to specify the working directory, process environment, and command line for the process to be invoked.
• JavaInvoke.java – a specialized subclass of ProcessSpawner, this class makes spawning new VMs a piece of cake, doing the necessary translation for Java system properties, setting the proper classpath environment variable with potential overrides, and fills in the fully qualified class name to run.

The Example & Source Code

I’ve put together a running example that implements a trivial client and server in JUnit test. The setup() method spawns the server and then the tests run the client code against the server, tearing it down after each test. It’s available in the PalantirVMSpawnerExample.zip zip file. Unzip it, run the run.sh or run.bat script as appropriate. It should generate output that looks like this:

-----------------------------------------------------
Starting test testAck
INFO [main] JavaInvoke - CLASSPATH=./lib/devblog-vmspawner.jar
INFO [main] ProcessSpawner - Build process spawner for the following command line:
INFO [main] ProcessSpawner - /home/pteng/java/i586/jdk1.5.0_14/jre/bin/java com.palantir.blog.processspawner.Server
Starting output piper for tag: server-stdout
Starting output piper for tag: server-stderr
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler2]: Got message: some message
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler3]: Got message: SHUTDOWN
Output piper exiting for tag: server-stdout
Output piper exiting for tag: server-stderr
Finished test testAck
-----------------------------------------------------
-----------------------------------------------------
Starting test testShutdown
INFO [main] JavaInvoke - CLASSPATH=./lib/devblog-vmspawner.jar
INFO [main] ProcessSpawner - Build process spawner for the following command line:
INFO [main] ProcessSpawner - /home/pteng/java/i586/jdk1.5.0_14/jre/bin/java com.palantir.blog.processspawner.Server
Starting output piper for tag: server-stdout
Starting output piper for tag: server-stderr
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler3]: Got message: SHUTDOWN
Output piper exiting for tag: server-stdout
Output piper exiting for tag: server-stderr
Took 3 ms to send shutdown.
Took 335 ms for process to die.
Finished test testShutdown
-----------------------------------------------------
SUCCESS: all 2 tests passed

The source is included in the zip file, but if you wanted to look at it or link to it on the web, here are the classes involved:

• Client.java
• Example.java
• JavaInvoke.java
• ProcessSpawner.java
• Server.java
• ServerSpawningTest.java

And as an added bonus, there’s an Ant build.xml that will let you tweak and rebuild the demo yourself.

Comments and questions welcome. Enjoy.

The solution was to select a single event firing system. We wanted something that was easy-to-use yet powerful enough to express all the changes that might be made to a data model. Java’s Property Change Support (PCS) was a good fit because it can support arbitrary events in a very lightweight fashion.

Read on for details of our implementation…

Property Change Support

Java’s PropertyChangeSupport class (PCS) basically allows an object to easily fire events consisting of 4 pieces of information:

• source object – the thing that fired the event
• property name – allows the listener to tell events for different things apart
• old value – the old value for the property
• new value – the new value for the property

PCS handles all the bookkeeping for adding and removing listeners and firing events. It is very useful for creating listenable models, but we wanted to make it just a little bit easier by having an abstract class that exposed the add/remove listener calls and took care of initializing PCS:

public abstract class AbstractListenableModel implements Serializable {

    private static final long serialVersionUID = 1L;

    private transient PropertyChangeSupport pcs;

    protected AbstractListenableModel() {
        this.init();
    }

    /**
    * Adds a property change listener to the model.
    */
    public final void addPropertyChangeListener(PropertyChangeListener listener) {
        this.pcs.addPropertyChangeListener(listener);
    }

    /**
    * Removes a property change listener from the model.
    */
    public final void removePropertyChangeListener(PropertyChangeListener listener) {
        this.pcs.removePropertyChangeListener(listener);
    }

    /**
    * Fires a property change event to listeners of the model.
    */
    protected final void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
        this.pcs.firePropertyChange(propertyName, oldValue, newValue);
    }

    /**
    * Initializes transient fields during deserialization.
    */
    protected Object readResolve() {
        this.init();
        return this;
    }

    /**
    * Initializes transient fields.
    */
    private void init() {
        this.pcs = new PropertyChangeSupport(this);
    }
}

AbstractListenableModel is basically just a simple wrapper for exposing the functionality of PCS. By extending this abstract class, it’s very easy to create a listenable model:

public final class MyModel extends AbstractListenableModel {

    public static final String PROP_FOO = MyClass.class.getName() + ".Foo";

    private int foo;

    public int getFoo() {
        return this.foo;
    }

    public void setFoo(int foo) {
        //
        // The semantics of the following line are a little hard to unpack,
        // but it does exactly what it needs to do, and the tradeoff
        // for conciseness over immediate readability is worth it for
        // large models with lots of properties.
        //
        // First, the JVM starts to create a stack frame for the call
        // into firePropertyChange().  It begins binding parameter values
        // from the left to the right.  The pointer to the String contained
        // in PROP_FOO is passed in first, then the current value of
        // this.foo is passed in, then the expression
        //        this.foo = foo
        // is evaluated (setting this.foo to the new value of foo), which
        // returns the new value of foo.  All the parameters are then
        // passed down into firePropertyChange(), which checks whether
        // the oldValue is equal to the newValue.  If they're not equal,
        // it fires the event.  If they are equal, it ignores the event.
        //
        this.firePropertyChange(PROP_FOO, this.foo, this.foo = foo);
    }
}

In this example, MyModel contains a single property called foo. When the value of foo is changed, a property change event will be fired to listeners of the model.

You may notice that the value of PROP_FOO is prefixed by the name of the class. This ensure that naming collisions do not occur for scenarios in which the same listener is used to listen to multiple models which happen to use the same property name. This scenario becomes much more likely in the case of event bubbling, which I’ll talk about next.

Event Bubbling

Imagine a scenario in which we have a nested model:

Normally, if a listener needs to receive events from both models A and B, it will need to add itself as a listener to each individual model. While this solution would work, it’s a little cumbersome, especially when model B can get swapped out for model B’—the listener then has to keep itself synched to the internal state of model A. It would be nice if model A could just automatically forward all the events from model B (or B’) via its PCS support so that a listener only needs to attach itself to one model instead of multiple models. With a bit more code in AbstractListenableModel, this is possible:

public abstract class AbstractListenableModel implements Serializable {

    private transient PropertyChangeListener childModelListener;

    ...

    /**
    * Registers a child model to this model.
    */
    protected void registerChildModel(ListenableModel childModel, String propertyName) {
        childModel.addPropertyChangeListener(this.childModelListener);
    }

    /**
    * Initializes transient fields.
    */
    private void init() {
        ...
        this.childModelListener = new ChildModelListener();
    }

    /**
    * Listener for property change events fired from child models.
    */
    private final class ChildModelListener implements PropertyChangeListener {
        public void propertyChange(PropertyChangeEvent event) {
            // This is where the bubbling happens
            pcs.firePropertyChange(event);
        }
    }
}

Now, whenever model B fires a property change event, this event will also be fired by model A. This makes it much easier for the listener to listen to events arbitrarily deep in the model hierarchy, because each event fired by a child model gets re-fired (bubbled) by all its ancestors. All you have to do is attach a listener to the root model and you’ll automatically receive events from all models in the hierarchy.

Note that the registerChildModel method above takes an unused propertyName argument. In the full implementation of this class, events with the provided property name are monitored. When an event with the provided property name is fired, childModelListener is detached from the old child model and attached to any new child model. This ensures that the listenable model is always listening to the current child models.

Events for Collections

Any model event support would not be complete without some consideration of how to handle collections such as sets and lists. To solve this scenario, we created specialized collection classes called ListenableModelSet and ListenableModelList. These collections hold AbstractListenableModels as their elements and fire events whenever their contents change. Since the changes to collections can vary widely, the solution we came up for communicating collection changes with full fidelity is basically to fire events with a copy of the old set as the old value and the new set as the new value. Listeners can then diff the old and new values to determine exactly what changed if necessary. Additionally, each ListenableModelSet or List adds a ChildModelListener to all of its children (themselves AbstractListenableModels), thereby ensuring that events are bubbled from all models in the collection.

Conclusion

Just as we saw with the Adapter piece of the MVA triad, when we componentize the Model piece there are huge gains to be had. Once we started using a base Model class and a consistent eventing infrastructure (PropertyChangeSupport), we could add features that made coding across our entire application a lot more pleasant.

Model–View–Controller (MVC) is an architectural pattern used in software engineering. Successful use of the pattern isolates business logic from user interface considerations, resulting in an application where it is easier to modify either the visual appearance of the application or the underlying business rules without affecting the other. In MVC, the model represents the information (the data) of the application; the view corresponds to elements of the user interface such as text, checkbox items, and so forth; and the controller manages the communication of data and the business rules used to manipulate the data to and from the model.

They go on to show the classic triangle diagram and how it’s baked into various GUI and web frameworks. There’s only one clause in the entire article that hints at something deeper: “Though MVC comes in different flavors…”

Different flavors indeed. In fact MVC is not just a pattern but a whole family of patterns: MVC, MVA, MVP, PAC, Model-Delegate…. It very quickly gets very hairy.

In this article I want to describe one of MVC’s lesser-known variants, the Model-View-Adapter (MVA) pattern, and talk about its advantages over traditional MVC in the context of a Java Swing application.

Architecture

The best place to start is with an architecture diagram. While vanilla MVC is a triangle:

MVA puts the Adapter in a position to strictly mediate between Model and View:

Here a solid line represents a direct relationship while a dashed line represents an indirect relationship via the Observer pattern. Put another way, the Adapter holds a pointer both to the Model and to the View and directly calls methods on both. At the same time, it attaches itself as a listener both to the Model and to the View in order to receive events. It receives property change events from the Model and action events (checkbox ticked, text entered, etc.) from the View, and then routes appropriate changes to the other side. The Adapter is entirely responsible for keeping the Model and the View in sync; the Model and View are both relatively dumb structures, knowing nothing about the other.

The advantages to organizing code this way are:

• All “moving parts” are centralized in one place, the Adapter. No worrying about where to add a listener; no hunting around to find isolated listeners.
• Separation of concerns between the View and the Adapter. The View is responsible for layout and visual presentation while the Adapter is responsible for synchronization and the dynamic aspects of the user interface.
• Better decoupling between Models and Views. Specifically, the View doesn’t need to know anything about the Model.

Additionally, while it will never be possible to fully componentize any variant of the MVC pattern, MVA is more amenable to componentization and thus more of its implementation can be centralized (in a single class) and reused. Once componentized, we can augment the basic functionality with things like:

• Automatic registration and unregistration of listeners when the View enters and exits the Swing component hierarchy, thereby preventing certain kinds of memory leaks.
• Automatic unregistration of listeners when the program shuts down. This can help free up resources like realtime subscriptions.
• Method for swapping a new Model object in for an old Model object.
• Ability to execute a task without listeners attached, to help prevent event-action-event loops.

The downside to using MVA over MVC is that the Adapter tends to take on a lot of the responsibility and can get quite complicated. But in my experience that can be mitigated by having good conventions about which pieces (M, V, A) are allowed to communicate with which other pieces and at what times. Enforcing predictable control flow goes a long way toward managing complexity.

Read on for a code-level description of our implementation of the MVA pattern.

Palantir MVA Implementation

Our half-componentization of MVA resides in a single abstract class named Adapter:

public abstract class Adapter<ViewType extends Component, ModelType> {
// constructor
protected Adapter(ViewType view, ModelType model); { ... }

/**
* Attach listeners to the View's subcomponents (checkboxes etc.).
* Listeners should be stored as member variables in the Adapter
* subclass.
*/
protected abstract void registerViewListeners();

/**
* Detach the same listeners (member variables) that were
* attached in registerViewListeners().
*/
protected abstract void unregisterViewListeners();

/**
* Attach listener(s) to the Model.
*/
protected abstract void registerModelListeners();

/**
* Detach the same listeners (member variables) that were
* attached in registerModelListeners().
*/
protected abstract void unregisterModelListeners();

/**
* Bring the View fully in synch with the Model.  Typically
* this involves querying state from the Model and
* reconfiguring subcomponents of the View accordingly.
*/
protected abstract void fullSynchronize();

protected ModelType getModel() { ... }
protected ViewType getView() { ... }

// other methods elided
}

New View components that want to stay synchronized with a Model must instantiate a subclass of Adapter and implement the abstract methods. The Adapter parent class (itself an example of the Template Method design pattern) will then call into the appropriate abstract methods at the appropriate times. For example, after the View is constructed, as soon as it’s displayed in the Swing component hierarchy the Adapter parent class will automatically call fullSynchronize() (whose implementation should bring the View in line with the Model) and then registerViewListeners() and registerModelListeners(), so the Adapter is poised to react to events. Likewise, when the View is removed from the component hierarchy (when its containing frame is closed, say), both unregisterViewListeners() and unregisterModelListeners() will be called. This can help ensure that no memory will be leaked when a long-life-cycle object (like a system-wide singleton) retains a pointer to a short-life-cycle object (the View) via the Observer pattern.

Dealing With Listener Loops

One problem that confronts UI developers is the problem of “listener loops”: infinite loops that result when the View fires an event, the Adapter (or Controller) responds to it by setting some property on the Model, and an event is propagated from the Model back to the View, starting the whole cycle over again.

One way to combat this is to make sure your Model only fires events when the value that’s being set on the Model is different from the value currently stored in the Model. (This will cut off the infinite loop after one and a half cycles.) It’s a good practice but often isn’t enough, especially when your system is multithreaded and events start to queue up. You can sometimes get into situations where an M-V-C triplet will thrash forever between two different values for one of the Model’s properties.

Our solution to this problem is a protected method (on our Adapter base class) called runWithoutViewListeners:

/**
* Guarantees that the job r will be run:
*    - on the Swing thread
*    - with Model listeners attached
*    - with View listeners DEtached
*/
public final void runWithoutViewListeners(final Runnable r) { ... }

The implementation of this method checks to make sure the view listeners are attached when it’s called, detaches them via a call to unregisterViewListeners(), invokes the Runnable, then reattaches the view listeners via a call to registerViewListeners(). The code inside the Runnable can then make whatever changes it wants to the View without perturbing the Model downstream. Listener loop averted!

More To Come

I hope that’s given you some sense of the territory out there in the wide world of MVC-variants. In a week or two, Derek will show off some of the work he’s done on the M piece of the MVA triad related to “event bubbling.” Stay tuned!

It’s always fun to release a new piece of jargon into the wild. I’ve run into a number of bugs in our codebase that caused by an anti-pattern I’d like to dub The Pokémon Problem.

Much like the game of Whac-a-Mole, this is a class of bugs where fixing every occurrence does not prevent the bug from returning in new code: it is easy for code delta to result in an instance of the bug being re-introduced into the code base. Even if you “catch ‘em all“, nothing prevents someone else from introducing new Pokémon bugs later.

Not only is this bug easy to re-introduce, but it sometimes can be hard to find all currently existing instances of this pattern. Although tools like Eclipse make it easier to track down all the places that code is called, sometimes you’re looking for things that happen in a certain sequence (which tools like Eclipse don’t do a good job of searching for) and dynamic invocation mechanisms like Java Reflection can sometimes make it impossible to be exhaustive. This type of bug is also resistant to automated refactoring: changing the protocol of dealing with this corner of your code will require you to track down all places it was touched and manually refactor them. It generally signals a failure to use sufficient separation of concerns.

In general, this anti-pattern is a result of APIs that require the caller to be responsible for state management of resources that the API owns. This can include things like an object that requires the caller to have run an initialization method before calling any other method on the object. These bugs get even more insidious when a failure to do things in the right order does not cause a hard failure (like throwing an exception) but instead creates some sort of subtle corruption that may not be noticed or cause subsequent calls to fail unexpectedly.

Read on for some strategies on dealing with the Pokémon problem.

Solving the Pokémon Problem

How do we solve the Pokémon problem?  Sometimes it is as easy as writing a unit test to verify that no new Pokémon exist. More often than not, though, you’ll have to use better encapsulation to solve the problem. If the Pokemon problem is the anti-pattern, encapsulation is its opposite.

For an example, I’ll turn to a real problem I ran into in the Palantir Government codebase: Let’s say I have class Parser and it has a method getAttribute(). I want to add the ability to have shortened (obfuscated) tag names to the Parser class.

The wrong approach

One approach I can take:

1. Create a class called AttributeHandler that wraps calls to Parser.getAttribute(). It handles detection of the short or full XML dialect and then returns the appropriate attribute value.
2. Stick an instance of this AttrbributeHandler class in a member variable of Parser and make sure to call AttributeHandler.getAttribute() instead of Parser.getAttribute() when processing attributes.
3. Put comment on Parser.getAttribute() mentioning that it shouldn’t be called directly anymore.

I just introduced a Pokémon problem.  The next person that edits this code may not read my comment and know that calling Parser.getAttribute() is a bug.  They may not test the short tag case and find the bug. How do I fix this Pokémon problem? 

The right approach

Approaching this encapsulation problem has a number of different approaches one could take. Here’s what I chose:

1. Create a class called AttributeHandler that wraps calls to Parser.getAttribute(). It handles detection of the short or full XML dialect and then returns the appropriate attribute value.
2. Stick an instance of this AttrbributeHandler class in a member variable of Parser.
3. Create a wrapper class around Parser that delegates the getAttribute() call to the AttributeHandler member when processing attributes.

(You might ask why I used a delegate instead of sub-classing the parser and overriding the getAttribute() method: it didn’t fit for architectural reasons that aren’t relevant here). Now the developer no longer has access to the Parser.getAttribute() method that would cause the undesired behavior.

In-depth Example: Resource Management

Another common place you might run into the Pokémon problem is code that needs to clean up resources.  Some coders are lazy and don’t want to always write their try/finally/close blocks properly. This can lead to resource leaks; for things like file handles, this usually isn’t a large issue, but for scarce resources like database connections, it’s critical that things get cleaned and returned to the pool. For locks, it’s absolutely essential to avoid deadlocks.

The wrong way

Here’s a simple example of bad resource management. In this (admittedly contrived) scenario, we have class that’s managing a resource for us, in this case it’s a status file. Different parts of the app need to read the status file where, presumably, something is storing its status.

Here’s the naive implementation of the status manager class:

public class PokemonStatusFileManager {

	final File statusFile;

	public PokemonStatusFileManager(File statusFilePath) {
		this.statusFile = statusFilePath;
	}

	public InputStream getStatusFileInputStream() throws FileNotFoundException {
		return new FileInputStream(this.statusFile);
	}
}

Based on this implementation, here’s some code that would use it. Note that first method, parseStatusFileIncorrectly() does no error checking and may not close the InputStream properly if an exception is thrown. The second method does proper resource handling, but it’s kind of ugly to read.

public class PokemonParseStatusFile {

	/**
	 * This is not proper resource management.
	 * @param manager
	 * @throws IOException
	 */
	public static void parseStatusFileIncorrectly(PokemonStatusFileManager manager)
		throws IOException {

		InputStream statusFile = manager.getStatusFileInputStream();
		readFrom(statusFile);
		statusFile.close();

	}

	/**
	 * This is proper resource management, but it's tedious to have to write.
	 * Some coders are too lazy to always do this the right way.
	 * @param manager
	 * @throws IOException
	 */
	public static void parseStatusFileCorrectly(PokemonStatusFileManager manager)
		throws IOException {

		InputStream statusFile = null;
		try {
			statusFile = manager.getStatusFileInputStream();
			readFrom(statusFile);
		} finally {
			// carefully close the resource
			if(statusFile != null) {
				try {
					statusFile.close();
				} catch(Exception e) {
					System.err.println("Error closing statusFile!");
					e.printStackTrace(System.err);
				}
			}
		}
	}

	static void readFrom(InputStream statusFile) throws IOException {
		// do the reading here...
	}
}

So this produces a classic Pokémon problem: everyone who interacts with the StatusFileManager has to do proper resource handling. Now imagine that this is an important lock instead of just a file handle: this Pokémon problem could cause deadlock (which would be bad).

So how do we fix this? The Visitor Pattern.

Solving the Pokémon problem with the visitor pattern

The Visitor Pattern is a potent weapon in fighting the Pokémon problem: it allows you to fully encapsulate access to a resource by injecting into the places it’s needed but preserving overall control of the resource in the code that “owns” the resource. Classically used for controlling things like iteration order, here the visitor pattern is applied as a form of dependency injection to enable lifecycle management.

The visitor pattern as applied to resource management is fairly straightforward:

1. Define an interface, ResourceVisitor with one method, visit(Resource r) (where Resource is the type of resource we’re managing.
2. Define the resource manager with a method that takes a ResourceVisitor as a parameter. Manage the lifecycle of the resource and call ResourceVisitor.visit(r) when appropriate, handling all initialization, error-handling and cleanup.

Here’s our re-spin of the status file manager class. You’ll notice that we’ve moved the resource handling code from the correct example above and encapsulated it in the visitor pattern:

public class UnPokemonStatusFileManager {

	/**
	 * Visitor interface implemented by callers wishing to
	 * interact with the status file.
	 */
	public static interface StatusFileVisitor {
		public void visitStatusFile(InputStream statusFile) throws IOException;
	}

	final File statusFile;

	public UnPokemonStatusFileManager(File statusFilePath) {
		this.statusFile = statusFilePath;
	}

	/**
	 * Note that this method is now private.
	 */
	private InputStream getStatusFileInputStream() throws FileNotFoundException {
		return new FileInputStream(this.statusFile);
	}

	/**
	 * Here's the method that takes the visitor and
	 * fully encapsulates the lifecycle of the status file.
	 */
	public void parseStatusFile(StatusFileVisitor parser)
	throws IOException {
		InputStream statusFile = this.getStatusFileInputStream();
		try {
			parser.visitStatusFile(statusFile);
		} finally {
			// carefully close the resource
			try {
				statusFile.close();
			} catch(Exception e) {
				System.err.println("Error closing statusFile!");
				e.printStackTrace(System.err);
			}
		}
	}
}

Now that we’ve encapsulated the complexity of the resource handling the UnPokemonStatusFileManager class, code that needs to access the status file can be written in a highly correct manner without much work.

public class UnPokemonParseStatusFile {

	public static void parseStatusFile(UnPokemonStatusFileManager statusFileManager)
		throws IOException {

		// generate anonymous visitor to do the processing
		StatusFileVisitor visitor = new StatusFileVisitor() {
			public void visitStatusFile(InputStream statusFile) throws IOException {
				readFrom(statusFile);
			}
		};
		statusFileManager.parseStatusFile(visitor);
	}

	static void readFrom(InputStream statusFile) throws IOException {
		// do the reading here...
	}
}

By encapsulating the full lifecycle of the status file in the visitor pattern, I’ve ensured that it’s always accessed properly. If I change the place where we store status to be a database rather than a file, nothing needs to change except this one class; the calling code remains the same.

We now have easy re-factoring, no resource leaks, and have simplified calling code. And finally: there are no new bugs to be introduced by callers that aren’t sure how to use our resource. Looks like we caught ‘em all!

Wrapping It All Up

So there it is, your new piece of jargon: The Pokémon Problem anti-pattern. You heard it here first! Please post any other great examples to the comments section on this post.

One of the things our customers love to do is print our beautiful object graphs and tape them to the wall for discussion. What they hate to do is print 30 pages, line them up, and tape them to a poster one at a time. So we bought a plotter, and I started plotting.

I needed to print directly to a Java Graphics object. Unfortunately, the available information on large output printing from Java is thin at best. While there are lots of ways to successfully place ink on paper, I was only able to find one that reliably lets the application pick odd paper sizes that plotters use, like 24×19.7 inches. (The term “plotter” used to mean something with pens for printing blueprints and such. Now it just means a large format printer, commonly printers that can use roll paper as a source.)

One of the first things you’ll learn when you start working with printing in Java is that a language intended to be all things to all people (i.e., cross-platform) is utterly lousy at tasks highly specific to a given environment, such as printing. It will not surprise you to hear that native print services on Windows are pretty different from those available on a Mac, which themselves are pretty different from the CUPS system common to Unix systems.

So, by and large, you are reduced to the least common denominator of printing. Part and parcel of this least common denominator is agreeing on what constitutes a piece of paper and sticking to it. This is fine for people thinking, “My paper is 8.5 inches wide by 11 inches tall.” It poses a bit of a problem for people with plotters who are thinking, “My paper is 24 inches wide by as many damned inches tall as I need.” Even relatively powerful programs like PhotoShop or GIMP don’t seem to support plotters well. I believe Photoshop works by specifying the exact paper size you want to use, but any technique in which the easiest solution for the user is to pull out a calculator does not meet with my approval.

Advice in a nutshell

A tutorial by MartyHall provides some valuable insights into printing which I applied to my printing component. While it didn’t address my specific issues, props to a well-done starting point.

I’m starting with the assumption that you’ve got yourself a class that implements the Printable interface so that once the Java printing subsystem is coerced into accepting the appropriate paper size, Printable.print(Graphics, PageFormat, int) will be called correctly and your job will print properly. Since my usage was screen-visible as well as printable, I simplified and asserted that one screen pixel is equivalent to one point (1/72 inch), which is how Java prepares the scaling on the Graphics object passed to Printable.print(…), thus making printouts look about the size of onscreen displays. Then, knowing that the printout needed, for example, 1200 vertical pixels/points, I can automatically calculate that I want to use a paper size 18″ tall or so (1200 / 72 dpi = 16.67″ + margins).

Here’s a summary of the critical steps:

1. Do the math yourself to get the paper size required in points (1/72 of an inch)
2. Implement the Pageable interface to report page count (usually one) and the PageFormat using the computed paper size.
3. Create a PrinterJob configured with the correct PrintService
4. Set your Pageable on the PrinterJob instance
5. Optionally display print setup dialog (PrinterJob.printDialog()) recognizing that changes to orientation or paper size will be ignored.
6. call PrinterJob.print()

Generally, don’t use the standard dialogs

This might seem contrary to desired behavior, but I found that giving the user access to the usual native or cross-platform print dialogs can be a bit sticky because those are places where the user can alter the selected paper; in order to properly print to roll paper sources, we must override any paper settings the dialogs might provide. Ultimately this can be confusing for the user, but that’s an unfortunate side effect of supporting plotters.

Don’t take any advice too seriously

We still choose to display the dialog because complex printers like plotters have many options that the users may need to alter and would otherwise have no way to modify.

StickFigure: looking at plotting in Palantir

This work didn’t happen in a vacuum: we have a need to print large graphs for display. Being able to print them in a large format is one of the simplest and most beloved methods of collaboration.

To give an example of why plotter printing is interesting, we created a stick figure on the graph in Palantir. After creating this monstrosity, we went go print it and examined the layout. The first layout was with letter-size paper, and just by looking at the head, we could see that it was going to be a lot of pages:

Zoomed-in view of the head of the stick figure. (click for full image)

Zooming out to view the entire graph, it’s clear that it’s going to be 35 pages!

Full pagination of the stick figure. (click for full image)

Even switching to the largest standard paper size only gets us to two sheets:

Paginated out to largest standard paper size, still two pages (click for full image)

When we switch over to using the plotter layout, we finally get to the one sheet we were looking for:

Paginated onto a plotter roll. (click for full image)

And finally, Mr. StickFigure comes to life:

Carl juggles with his creation

Some of our customers are pretty serious about data security. To that end, our products need to support and integrate with SSL for both data security and authentication. SSL is very neat technology, but there is a dizzying maze of standards to navigate to figure how to get it all to work.

It turns out that in this age of Google, the fastest way to figure out how to do something is often to Google for key terms and hope that someone has put the relevant details in a blog post somewhere. In trying to figure out how to set up keys on a SunOne Directory Server for testing our LDAP integration, I needed to figure out how to get keys into PKCS#12 format after generating them with OpenSSL.

I’ll spare you the gory details of what it took to figure out how to do this; suffice to say there was some trial and error mixed with a bunch of RTFM. After the jump, the full howto.

Introduction to PKCS#12

PKCS#12 is used by a number of different vendors as their standard key-exchange format for key management, most notably IE and the SunOne/Netscape products. OpenSSL, while remaining the Swiss Army-Knife of crypto tools, doesn’t use PKCS#12 as a native format. However it knows how to convert to it.

The overall strategy here is to convert to PKCS#12 as a last step; we do normal key generation and signing using OpenSSL and then convert the results into PKCS#12 format.

To get a key-pair generated and signed by a certificate authority and then ready for import into one of these tools follow these steps (user input has been bolded):

Generate the key

# openssl genrsa -out  example-key.pem 2048
Generating RSA private key, 2048 bit long modulus
..................................+++
...........................................................................................+++
e is 65537 (0x10001)

Your RSA, 2048-bit public/private key pair now reside in the file named example-key.pem.

Generate the Certificate Signing Request (CSR)

Next we generate the CSR normally.

# openssl req -new -key example-key.pem -out example.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:California
Locality Name (eg, city) [Newbury]:Palo Alto
Organization Name (eg, company) [My Company Ltd]:Palantir Technologies
Organizational Unit Name (eg, section) []:Palantir TechBlog Examples
Common Name (eg, your name or your server's hostname) []:example.palantirtech.com
Email Address []:example@palantirtech.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

A certificate signing request for the public key in example-key.pem is now in example.csr and will look something like this if you look at the contents:

—–BEGIN CERTIFICATE REQUEST—–
MIIC/zCCAecCAQAwgbkxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MRIwEAYDVQQHEwlQYWxvIEFsdG8xHjAcBgNVBAoTFVBhbGFudGlyIFRlY2hub2xv
Z2llczEfMB0GA1UECxMWUEcgQmFja2VuZCBFbmdpbmVlcmluZzEaMBgGA1UEAxMR
ZmxpbnQueW9qb2UubG9jYWwxJDAiBgkqhkiG9w0BCQEWFXJlZ3NAcGFsYW50aXJ0
ZWNoLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMWmxedFCqsx
ypNNYTUbcwVxtTUFMS6pP0y78GAabFvDQHW48IQXgyMrqlLl4//dI7EO/K6+ZdEJ
Go6fuXW5jd/1RuxQRkCsfdVUrrhghVOywpSeWCzwraFniPQOqstQHuWdfygFkpVZ
xsY3pu9cVvs827wvjHZXPpmLRZu5xDom+r8aAdjrQdkCCqbzdNFcxnxdHM03LMCz
BJcAUSMjhvynBRNgzB01bOJOk+72lAuaEhjF0TMvnRMo16mhd/dnaHkqqW+N8q9m
yTH1cTrNrdgJKkJ44K1sSJhFVMgkC2R4GK6RmprwIDVD8YkYLG+Iahw18xQKbCn1
8NomTXbZBAUCAwEAAaAAMA0GCSqGSIb3DQEBBQUAA4IBAQBGR/1jScSVekhy/aR2
xePJcO4jzmMVW4NfxOknxLkDvuDhbP7wGWPtspLXNf45mc3AXtkNgQTwO0siZHyy
HaSLo4BuB4A62ofZfTVn6KYVzxuwskkLRdhlHA6mpM01hMBhmzbIZDyAZqyWQAJ5
rC0Xl+ai/AAdLXO664qR3f2awXCUhJ/vYg7Qw4FHCLeoKcWxJfq9X9Ho5UJaFYTW
6QVFKeODF4Y2vMnBx63RcgOIuJXI38ZK4+k1ONKuB75IhbIu3DtMTIv3kYYtfd6G
P5BXlmgQgxVnFJcDprjEjHEUY4B+0vaWbDqy7U6fucT7EJ+qrSfk0Hf89aaQPKFf
DUwz
—–END CERTIFICATE REQUEST—–

This CSR is sent of to your CA of choice and you get back a certificate, essentially a signature for your public key by the CA.

The contents of the file, PEM-encoded, will look something like this:

—–BEGIN CERTIFICATE—–
MIIDdDCCAlwCAxAAIDANBgkqhkiG9w0BAQQFADB/MR4wHAYDVQQKExVQYWxhbnRp
ciBUZWNobm9sb2dpZXMxEjAQBgNVBAcTCVBhbG8gQWx0bzETMBEGA1UECBMKQ2Fs
aWZvcm5pYTELMAkGA1UEBhMCVVMxJzAlBgNVBAMTHlBhbGFudGlyIENlcnRpZmlj
YXRlIEF1dGhvcml0eTAeFw0wODAyMjYwNjQ5NTVaFw0xMzAyMjQwNjQ5NTVaMH8x
CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMR4wHAYDVQQKExVQYWxh
bnRpciBUZWNobm9sb2dpZXMxHzAdBgNVBAsTFlBHIEJhY2tlbmQgRW5naW5lZXJp
bmcxGjAYBgNVBAMTEWZsaW50Lnlvam9lLmxvY2FsMIIBIjANBgkqhkiG9w0BAQEF
AAOCAQ8AMIIBCgKCAQEAxabF50UKqzHKk01hNRtzBXG1NQUxLqk/TLvwYBpsW8NA
dbjwhBeDIyuqUuXj/90jsQ78rr5l0Qkajp+5dbmN3/VG7FBGQKx91VSuuGCFU7LC
lJ5YLPCtoWeI9A6qy1Ae5Z1/KAWSlVnGxjem71xW+zzbvC+Mdlc+mYtFm7nEOib6
vxoB2OtB2QIKpvN00VzGfF0czTcswLMElwBRIyOG/KcFE2DMHTVs4k6T7vaUC5oS
GMXRMy+dEyjXqaF392doeSqpb43yr2bJMfVxOs2t2AkqQnjgrWxImEVUyCQLZHgY
rpGamvAgNUPxiRgsb4hqHDXzFApsKfXw2iZNdtkEBQIDAQABMA0GCSqGSIb3DQEB
BAUAA4IBAQB0RbDK5D/6ndxSBermRwmUNkUFbSh+e6dB8g4GEnRiRKReoPaBkJeG
UMZU2Rv9xZZBO+vN5USHbY8syfGNdHPNn8nndD5PJk4InBfBgcxlepzVK46sHuar
+VRERk6b7RnP3vvbaBIAEG4h1+bdm5CANCexo8yptJp6m8d3AKmqv+mjTmoLc3Y7
yTFVzCj8AyMJOHg1jE+paQSWDwe/Hya/01a8g6HaCLIVeLuHO0IWmIZ1oK+TiFt8
H7cSCmNwGa9oojK3P9PH7sJbVsV1uhm0vYn0Qdmb7toWu1eKit1NRyeEUFlc2T9B
h3N944gca4k+eMcAOrR7iBWUDwXMTVCr
—–END CERTIFICATE—–

Stick it in its own file, example.crt.

Create the PKCS#12 key

Concatenate the cert with the key file and then have OpenSSL convert it to PKCS#12

# cat example-key.pem example.crt > example.pem
# openssl pkcs12 -export -in example.pem -out example.pkcs12 -name “example”
Enter Export Password: (password is not echoed here, but you must enter something)
Verifying – Enter Export Password: (password is not echoed here, but you must enter something)

Verify it worked

As they say: if you didn’t test it, it doesn’t work. So now we verify that another tool that claims to ingest PCKS#12,

verify that it worked with a different tool. Here I use Java’s keytool:

# keytool -v -list -storetype pkcs12 -keystore example.pkcs12
Enter keystore password:  password (whatever you set it to above)

Keystore type: pkcs12
Keystore provider: SunJSSE

Your keystore contains 1 entry

Alias name: flint
Creation date: Feb 26, 2008
Entry type: keyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=example.palantirtech.com, OU=Palantir TechBlog Examples, O=Palantir Technologies, ST=California, C=US
Issuer: CN=Palantir Certificate Authority, C=US, ST=California, L=Palo Alto, O=Palantir Technologies
Serial number: 100020
Valid from: Mon Feb 25 22:49:55 PST 2008 until: Sat Feb 23 22:49:55 PST 2013
Certificate fingerprints:
         MD5:  C2:4E:B0:62:D8:06:FB:10:77:A5:37:6C:C8:2F:2A:AF
         SHA1: D2:B4:6B:0C:9D:3B:A4:94:B9:BF:25:E5:57:D6:96:FA:FB:84:A6:A7

*******************************************
*******************************************

You’re now good to go. As usual: additions, oversights, and comments welcome.

We often use worker threads to do some long-running process, so often run into two issues using SwingUtilities invokeLater/invokeAndWait, and have developed wrapper code to deal with it. One issue is executing code from both worker threads and the Event Dispatch Thread (EDT). invokeLater and invokeAndWait both throw exceptions if executed on EDT. Second, invokeAndWait isn’t guaranteed — interruption on the calling thread will resume execution before the job is finished. The remainder of this post shows the code we used to solve these issues.

Reducing repetitive code

We often write functions that are executed from the Event Dispatch Thread (EDT) and also from other threads. Since these methods will throw an exception if you’re already on the EDT, it makes sense for us to wrap them in a quick safety check so we avoid an extra three lines of code every place we want to use it. Thus, a first minor improvement:

public static runOnEDT(final Runnable r) throws InvocationTargetException {
	if ( SwingUtilities.isEventDispatchThread() )
		r.run();
	else
		SwingUtilities.invokeLater( r );
}

Note the use of invokeLater — that will be important to the next section. invokeAndWait wouldn’t work the same.

invokeAndWait doesn’t wait

The second gotcha with invokeAndWait is that it doesn’t always. Wait, that is. If a thread is waiting for a Runnable to execute on the EDT and that thread is interrupted, execution of the waiting thread will resume immediately, BEFORE FINISHING the Runnable. I had assumed this meant we failed to wait for the EDT, that the job was never executed and should I execute invokeAndWait again. This was the wrong interpretation. Turns out, all that happened was waiting on completion of the job was interrupted. The Runnable is still enqueued on the EDT and will still execute. Depending on how the timing works out, the EDT Runnable may even be executing at the same time. I just have no idea when it will finish. Very inconvenient.

invokeAndWait throws InterruptedException so you can learn of this situation and deal with it, but it’s not really something you can solve after the error has already occurred. So the following is our solution:

We create a FutureTask to wrap the provided Runnable. We can await completion of the FutureTask via FutureTask.get() which tells us when the provided Runnable has finished (successfully or thrown exceptions, or whatever). After the Runnable finishes, if there were any interruptions during execution, we re-interrupt Thread.currentThread() so the caller knows an interruption occurred and can choose to perform followup action.

public static void runAndBlockSilently(Runnable r) {
	final FutureTask ft = new FutureTask(r, null);
	boolean wasInterrupted = false;
	runSafely( ft );
	while (! ft.isDone() ) {
		try {
			ft.get();
		}
		catch ( InterruptedException e ) {
			wasInterrupted = true;
			// Continue ...
		}
		catch(ExecutionException exEx) {
			Throwable cause = exEx.getCause();
			logger.error( "Exception during BlockingRunnable: ", cause );
			if(cause instanceof RuntimeException)
				throw (RuntimeException) cause;
		}
	}

	if(wasInterrupted) {
		Thread.currentThread().interrupt();
	}
}

Happy Swinging.

-Carl

How do you check for something that doesn’t exist?

The main challenge with programmatically testing for memory leaks is figuring out how to know whether or not you’re leaking memory. If you want to check to see whether a certain object is still allocated, it seems like you’d have to have a reference to it. However, if you have a reference to it, it’s not going to be garbage collected. But if you don’t have a reference to it, how are you supposed to check whether or not it’s been freed?

The power of weak references

Luckily, Java has just the construct for us: weak references. Weak references — objects of class WeakReference — act as references to an object, but, unlike normal references, they will not prevent the object from being garbage collected. Thus, once an object reaches a state in which it is only referenced by weak references, it will be eligible to be reclaimed by the garbage collector.Weak references can also be associated with a ReferenceQueue, which is a queue that holds WeakReference objects whose status has changed (for WeakReference objects, a status change occurs when the object that it is referring to is no longer reachable by strong references). We are guaranteed that, once an object becomes weakly reachable, it will be enqueued in its associated ReferenceQueue.Note: I am simplifying some things and skipping over other things like soft references and phantom references. For a more detailed explanation of these topics, check out this SDN article and this Java.net blog entry.

Writing the unit test

The properties of WeakReference objects make them perfect to use for tests where we want to see whether or not a certain piece of memory has been freed. The general process works as follows:

1. Create an object using a strong reference (assign it to a variable).
2. Create a ReferenceQueue object.
3. Create a WeakReference that refers to the desired object from step 1 and uses the ReferenceQueue from step 2.
4. Do whatever operations are needed to produce the suspected leak.
5. Release the known strong reference to your object (set the variable in step 1 to null).
6. Force garbage collection. (Technically, we are making a request for garbage collection using System.gc() — there’s no way to actually force it to occur. However, it’s the best we can do, and seems to work sufficiently well). If you’re having trouble with forcing garbage collection, anecdotal evidence seems to indicate that requesting garbage collection at least ten times will guarantee that garbage collection actually happens.
7. Check to see if your WeakReference from step 3 has been enqueued. If it has, then it will be freed, and there’s no leak. If it hasn’t, then there’s still a hard reference to it somewhere in your code, which means that it is being leaked.

That’s it! You can now write unit tests that test for memory leaks in your code. This obviously isn’t very effective for leaks that only occur after extensive UI interactions, but we’ve found it extremely useful for testing things such as verifying that our models correctly free all of their child models in the correct manner.

Sample code

Here’s a simple sample unit test that tests to make sure that an object will be freed once its reference is set to null:

	public void testMemory() {
		// create test object
		Object testObject = new Object();
		// create queue and weak reference
		ReferenceQueue queue = new ReferenceQueue();
		WeakReference ref = new WeakReference(testObject, queue);
		// set hard reference to null
		testObject = null;
		// force garbage collection
		System.gc();
		// soft reference should now be enqueued (no leak)
		assertTrue(ref.isEnqueued());
	}

There are many excellent tutorials out there on equals() and hashCode(). The best source is probably Items 7 and 8 of Effective Java by Josh Bloch, and we have a couple copies of this book floating around the office. A good online tutorial is Java theory and practice: Hashing it out, by another Java guru, Brian Goetz.

To test your understanding, make sure that you can answer the following questions. I often ask variants of the first two during interviews. The third comes from an error we’ve seen more than once in our code base.

1. Suppose we have a Media class, composed of title and contents fields. What can go wrong if equals() is based on both fields, while hashCode() is based solely on title?
2. Conversely, what can go wrong if hashCode() is based on both fields, but equals() is based solely on title?
3. Consider a class A which overrides equals() and hashCode(). If you create a class B which extends class A, and use Eclipse to generate your equals() and hashCode() functions for class B, then the output of class B’s hashCode() function will depend on the value of super.hashCode(). What happens if you refactor class B so that it no longer extends class A?