Here at Palantir, a lot of our automatic tests are full-chain tests. A backend server is fired up, client code runs against it, and everything runs much like a production environment. This makes intuitive sense because it’s a faithful approximation of how the system will run in the field.

However, there are some disadvantages to this:

• Full-pass tests don’t always localize the problem. Tests on a client class might fail even if it was the service that behaved incorrectly.
• These full-pass tests are relatively slow. Client code is running against an actual remote service. If a client is being tested, the server code still has to do work — sometimes a lot of work — even if that isn’t the focus of the test.
• The constraints of the test are loose. Full-chain tests can mostly only see whether the operation finished correctly. It’s much harder to figure out whether the operation was done efficiently and without making unnecessary service calls.
• They’re very little setup flexibility. If you want an RPC to return a specific value, you have little choice but to have your test get the service into a state where it can return that value. This is easy in some cases, but prohibitively difficult in others.
• Client tests are forced to share any non-determinism leaked from the service. For example, under real conditions, a request to call A might respond before call B, and sometimes the other way around. This can result in flaky tests or tests that don’t always simulate the conditions you want to exercise.

What’s to be done? Fortunately, there’s an option that handles these cases elegantly. We also test with jMock, a library that dynamically generates mock objects from arbitrary interfaces. These mock objects can be configured to check that particular methods are called with particular inputs a particular number of times, and then give prescribed responses.

Hit the link to see a concrete example of jMock in action.
Read the rest of this entry »

Here at Palantir we use test-driven development (or TDD for short). Integrated tools like Eclipse and JUnit simplify writing and running unit tests. However, once you need to test a broader swath of functionality, it’s time to write functional, integration, and system tests. While technically not ‘unit testing’, the testing framework that JUnit provides is basically the same infrastructure that you want to leverage for writing these more involved types of testing.

When you’re developing enterprise software, functional testing often means getting your clients to talk to your servers. For the main Palantir Government product, we integrate the process of bringing the server up and down with the Ant scripts that run our automated unit tests: our testing tasks bring up the server, run the test suite, and then kill the server. This works great and produces nice results.

When I started working on our authentication server, the pattern that we had used before didn’t work for me. While the Palantir Government tests ran with a single, static configuration file, I needed to run the authentication server with multiple configurations in the course of running through the all the different functional tests. I determined that I needed a way to programmatically bring the server up and down for testing. In JUnit parlance, I needed a way to programmatically launch the server component as part of my setup() function for my unit tests and stop it in my teardown().

With my itch-to-scratch firmly in hand (or some other mixed metaphor), I set out to figure out how to invoke new Java processes from inside a unit test. The solution I came up with (with source code and examples) after the jump.
Read the rest of this entry »

One of the early architectural challenges that we faced in building the Palantir Finance product was coming up with a good design for firing events from data models to their listeners. There are many different concepts in our product such as charts, portfolios, and indices which are all maintained by different developers. Initially, each developer had their own system for firing events when a data model changed. This quickly became a drag on development as tools became more integrated because we had to learn each others’ event methodologies and translate between the different systems.

The solution was to select a single event firing system. We wanted something that was easy-to-use yet powerful enough to express all the changes that might be made to a data model. Java’s Property Change Support (PCS) was a good fit because it can support arbitrary events in a very lightweight fashion.

Read on for details of our implementation…
Read the rest of this entry »

I used to think I understood MVC. In undergraduate CS programs, MVC is taught as an off-the-shelf pattern, explained once and then ready for use in the real world. Wikipedia also makes it seem pretty simple:

Model–View–Controller (MVC) is an architectural pattern used in software engineering. Successful use of the pattern isolates business logic from user interface considerations, resulting in an application where it is easier to modify either the visual appearance of the application or the underlying business rules without affecting the other. In MVC, the model represents the information (the data) of the application; the view corresponds to elements of the user interface such as text, checkbox items, and so forth; and the controller manages the communication of data and the business rules used to manipulate the data to and from the model.

They go on to show the classic triangle diagram and how it’s baked into various GUI and web frameworks. There’s only one clause in the entire article that hints at something deeper: “Though MVC comes in different flavors…”

Different flavors indeed. In fact MVC is not just a pattern but a whole family of patterns: MVC, MVA, MVP, PAC, Model-Delegate…. It very quickly gets very hairy.

In this article I want to describe one of MVC’s lesser-known variants, the Model-View-Adapter (MVA) pattern, and talk about its advantages over traditional MVC in the context of a Java Swing application.

Read the rest of this entry »

It’s always fun to release a new piece of jargon into the wild. I’ve run into a number of bugs in our codebase that caused by an anti-pattern I’d like to dub The Pokémon Problem.

Much like the game of Whac-a-Mole, this is a class of bugs where fixing every occurrence does not prevent the bug from returning in new code: it is easy for code delta to result in an instance of the bug being re-introduced into the code base. Even if you “catch ‘em all“, nothing prevents someone else from introducing new Pokémon bugs later.

Not only is this bug easy to re-introduce, but it sometimes can be hard to find all currently existing instances of this pattern. Although tools like Eclipse make it easier to track down all the places that code is called, sometimes you’re looking for things that happen in a certain sequence (which tools like Eclipse don’t do a good job of searching for) and dynamic invocation mechanisms like Java Reflection can sometimes make it impossible to be exhaustive. This type of bug is also resistant to automated refactoring: changing the protocol of dealing with this corner of your code will require you to track down all places it was touched and manually refactor them. It generally signals a failure to use sufficient separation of concerns.

In general, this anti-pattern is a result of APIs that require the caller to be responsible for state management of resources that the API owns. This can include things like an object that requires the caller to have run an initialization method before calling any other method on the object. These bugs get even more insidious when a failure to do things in the right order does not cause a hard failure (like throwing an exception) but instead creates some sort of subtle corruption that may not be noticed or cause subsequent calls to fail unexpectedly.

Read on for some strategies on dealing with the Pokémon problem.
Read the rest of this entry »

Carl juggles with his creation

One of the things our customers love to do is print our beautiful object graphs and tape them to the wall for discussion. What they hate to do is print 30 pages, line them up, and tape them to a poster one at a time. So we bought a plotter, and I started plotting.

I needed to print directly to a Java Graphics object. Unfortunately, the available information on large output printing from Java is thin at best. While there are lots of ways to successfully place ink on paper, I was only able to find one that reliably lets the application pick odd paper sizes that plotters use, like 24×19.7 inches. (The term “plotter” used to mean something with pens for printing blueprints and such. Now it just means a large format printer, commonly printers that can use roll paper as a source.)

One of the first things you’ll learn when you start working with printing in Java is that a language intended to be all things to all people (i.e., cross-platform) is utterly lousy at tasks highly specific to a given environment, such as printing. It will not surprise you to hear that native print services on Windows are pretty different from those available on a Mac, which themselves are pretty different from the CUPS system common to Unix systems.

So, by and large, you are reduced to the least common denominator of printing. Part and parcel of this least common denominator is agreeing on what constitutes a piece of paper and sticking to it. This is fine for people thinking, “My paper is 8.5 inches wide by 11 inches tall.” It poses a bit of a problem for people with plotters who are thinking, “My paper is 24 inches wide by as many damned inches tall as I need.” Even relatively powerful programs like PhotoShop or GIMP don’t seem to support plotters well. I believe Photoshop works by specifying the exact paper size you want to use, but any technique in which the easiest solution for the user is to pull out a calculator does not meet with my approval.
Read the rest of this entry »

Some of our customers are pretty serious about data security. To that end, our products need to support and integrate with SSL for both data security and authentication. SSL is very neat technology, but there is a dizzying maze of standards to navigate to figure how to get it all to work.

It turns out that in this age of Google, the fastest way to figure out how to do something is often to Google for key terms and hope that someone has put the relevant details in a blog post somewhere. In trying to figure out how to set up keys on a SunOne Directory Server for testing our LDAP integration, I needed to figure out how to get keys into PKCS#12 format after generating them with OpenSSL.

I’ll spare you the gory details of what it took to figure out how to do this; suffice to say there was some trial and error mixed with a bunch of RTFM. After the jump, the full howto.
Read the rest of this entry »

One of the most misunderstood aspects of multithreaded Swing applications is care and feeding of SwingUtilities.invokeAndWait. Hans Muller and Kathy Walrath authored a nice article that includes an overview of when to use invokeLater or its slightly more risky sibling, invokeAndWait.

We often use worker threads to do some long-running process, so often run into two issues using SwingUtilities invokeLater/invokeAndWait, and have developed wrapper code to deal with it. One issue is executing code from both worker threads and the Event Dispatch Thread (EDT). invokeLater and invokeAndWait both throw exceptions if executed on EDT. Second, invokeAndWait isn’t guaranteed — interruption on the calling thread will resume execution before the job is finished. The remainder of this post shows the code we used to solve these issues.

Read the rest of this entry »

Memory leaks are no fun — to find them, you usually have to do a work flow while using a profiler like YourKit, force garbage collection, capture a memory snapshot, and then manually go through the snapshot looking for objects that you expected to be gone but are still there.Even after you’ve finally fixed the memory leak, how can you make sure that the issue doesn’t resurface later? Every good developer knows that, if you fix a bug, you should probably also have a JUnit test for that bug so that it doesn’t happen again. But how can you test for memory leaks programmatically?Find out after the jump… Read the rest of this entry »

Whenever you override the equals() and hashCode() functions of a class, it is very important that you do so correctly. Bugs in these functions are often quite insidious, because they will not fail catastrophically. Your application will compile and probably do something reasonable, passing any smoke test you throw at it. However, the output will be subtly incorrect. Moreover, by the time you realize that some has gone wrong, there is little information about what caused the error. This is because the error occurred 10 steps ago, nested five levels deep, when a member of a HashSet went MIA. While a NullPointerException is normally trivial to diagnose and fix, you can easily lose several hours tracking down an equals()/hashCode() bug.

There are many excellent tutorials out there on equals() and hashCode(). The best source is probably Items 7 and 8 of Effective Java by Josh Bloch, and we have a couple copies of this book floating around the office. A good online tutorial is Java theory and practice: Hashing it out, by another Java guru, Brian Goetz.

To test your understanding, make sure that you can answer the following questions. I often ask variants of the first two during interviews. The third comes from an error we’ve seen more than once in our code base.

1. Suppose we have a Media class, composed of title and contents fields. What can go wrong if equals() is based on both fields, while hashCode() is based solely on title?
2. Conversely, what can go wrong if hashCode() is based on both fields, but equals() is based solely on title?
3. Consider a class A which overrides equals() and hashCode(). If you create a class B which extends class A, and use Eclipse to generate your equals() and hashCode() functions for class B, then the output of class B’s hashCode() function will depend on the value of super.hashCode(). What happens if you refactor class B so that it no longer extends class A?