Our frontend engineering candidates go through many of the same types of interviews as our backend candidates, i.e., algo and coding. But because they’ll be building the user-facing parts of our system, we want to make sure candidates have some design chops as well. Hence the UI design interview.

Why have a design interview for an engineer?

Perhaps my favorite part of being on the frontend team at Palantir is that we get to work on both the design and the implementation of our own UIs, from inception to finished product. Engineers collaborate on the design of the product with other engineers as well as with designers, and we openly debate the merits of all our decisions. The UI interview simulates this kind of cooperative design and debate. Typically, this means you’ll be asked to design and/or evaluate one or two pieces of UI during the interview, while your interviewer plays both collaborator and critic.

Read on for some helpful points to keep in mind as you work on a design problem with your interviewer.

It’s all about the user

The user is the ultimate arbiter of success for any interface. If users can accomplish their tasks in a simple, efficient, intuitive way, then we’re doing our job right. Therefore, we must keep the user in mind at every stage of interface design.

One common mistake that you can make up front is to assume YOU are the user. Because working on a computer is a solitary activity, it’s easy to forget that everyone experiences an interface in a different way. Depending on what you’re designing, the user may be a complete novice or a seasoned sysadmin.

It’s important to imagine what your users are like. If it helps, give the user a name, age, and occupation. Ask yourself these questions:

• In what context will they use this feature? At work? At home? On a TV from 10 feet away?
• Have they used a similar interface before?
• How savvy are they with computers in general? Do they know how to copy and paste, drag and drop, or open a context menu?

When designing a feature for the interface, walk the user through the feature. Make a rough sketch of the main components (buttons, lists, text blocks) on a whiteboard or sheet of paper. Then simulate how your user might interact with the feature, by drawing in their input and selections.

While sketching your proposed interface, put yourself in the user’s shoes. Ask yourself these questions:

• What will they be doing when they want to start doing X?
• How do they discover the feature?
• What will they want to do after?
• How frequently will the user do X?
• What if X fails to complete properly?

and so on. Once you’ve asked yourself those questions, consider how the answers should impact your design and modify it accordingly.

tl;dr: The user is paramount. Each of your decisions should be justified by its impact on the user.

The interview is highly interactive

Some candidates are reluctant to contradict the interviewer or provide alternative ideas. We want the opposite. If you have a good idea, advocate for your position. I prefer candidates who will contradict me, as long as they back up their ideas with arguments, stories, or examples. The more you can articulate the reason for your beliefs, the better.

Go ahead and keep throwing ideas out there. I’ll shoot down some of them, but don’t be daunted. It may take 10 ideas before we discover a gem. (This is how all design works, not just in an interview setting.)

Be creative, but don’t re-invent the wheel

I’ve seen many candidates jump through incredibly awkward (albeit fancy) hoops just to display some very simple data. If you have a list of data, display it as a list. In general, being familiar with UI conventions is helpful because they encode a lot of hard-earned wisdom.

If you’ve created an interface that will enable the user to do their task as quickly and painlessly as possible, then stop. Don’t weigh down the interface with unnecessary features. As Deiter Rams famously said, “Good design is as little as possible.” This holds true for user interface design just as well as product design. Consider Alan Cooper’s rule of thumb, “A user will take as much time to make a choice as there are choices.” If a user is inundated with options and features, they find the interface more difficult to use. A simple, direct approach is frequently the best one.

If you want a quick and easy measure of the simplicity of a feature, just count the number of clicks to complete the task using your interface. If the user has to switch from mouse to keyboard, count that twice.

How to prepare

If you’ve done some design in the past, especially in a collaborative setting, just come and be yourself — you’ll do fine. But if you’re relatively inexperienced, here are some ways you can improve your design skills:

1. If you’re still in school, take an HCI or infoviz course, especially a project-based course. This will give you some actual design experience, as well as a rich vocabulary and set of concepts for thinking about user interfaces.
2. The design mindset is a muscle, so exercise it every chance you get. Constantly ask yourself, “How could this be different? How could it be better?” The more you ask yourself these questions, the more they’ll become unconscious and automatic. Soon you’ll be wondering about the design of everyday things — in the shower, on the train, and hopefully every time you interact with a piece of software. If you develop the skills to analyze other people’s designs, you’ll be able to apply those skills to your own own.
3. Actually build something and pay attention to the UI. Go through more than one design before (tentatively) settling on the one you’re going to build. Draw wireframes and/or mockups. Iterate constantly, even after you think you’re done.
4. Ask someone to critique your work. If you can find someone with an HCI background (or other design skills), she’ll show you things you couldn’t see yourself. Once this has happened a few times, you’ll start internalizing your critics; you’ll start asking yourself, “What am I missing here?” If you find someone who doesn’t have a design background, treat them like a user. This means taking their feedback very seriously but leaving some room for skepticism. Notoriously, users don’t always know what they want.
5. Read up on UI/UX/HCI/infoviz. There are a lot of good books and blogs out there. Some of the ones that we’ve found particularly helpful are About Face by Alan Cooper, Now You See It by Stephen Few, and Don’t Make Me Think by Steve Krug.

I hope this post will help you design better interfaces, and perhaps I’ll be working with you soon!

Note: this third installment in our series on doing your best in interviews. Previously: “How to Rock an Algorithms Interview” and “The Coding Interview”.

One interview that candidates often struggle with is the systems design interview. Even if you know your algorithms and write clean code, that code needs to run on a computer somewhere — and then things quickly get complicated. A truly unbelievable amount of complexity lies beneath something as simple as visiting Google in your browser. While most of that complexity is abstracted away from the end user, as a system designer you have to face it head on, and the more you can handle, the better.

At Palantir, many of our teams give a systems design interview along with an algorithms interview and a couple of coding interviews. We don’t expect anyone to be an expert at all three disciplines (although some are). We’re looking for generalists with depth — people who are good at most things, and great at some. If systems design isn’t your strength, that’s okay, but you should at least be able to talk and reason competently about a complex system.

Read on to learn about what we’re looking for and how you can prepare.

We’re measuring three things

Nominally, this interview appears to require knowledge of systems and a knack for design — and it does. What makes it interesting, though, and sets it apart from a coding or an algorithms interview, is that whatever solution you come up with during the interview is just a side effect. What we actually care about is the process.

In other words, the systems design interview is all about communication.

This reflects what actually working at Palantir is like. As engineers we have a tremendous amount of freedom. We aren’t asked to implement fully-specced features. Instead we take ownership of open-ended problems, and it’s our job to come up with the best solution to each. We need people we can trust to do the right thing without a lot of supervision — people who can own large projects and take them consistently in the right direction. Invariably, this means being able to communicate effectively with the people around you. Working on problems with huge scope isn’t something you can do in a vacuum.

It’s an open-ended conversation

Usually we’ll start by asking you to design a system that performs a given task. The prompt will be simple, but don’t be fooled — these problems are wide and bottomless, and the point of the interview is to see how much volume you can cover in 45 minutes.

For the most part, you’ll be steering the conversation. It’s up to you to understand the problem. That might mean asking questions, sketching diagrams on the board, and bouncing ideas off your interviewer. Do you know the constraints? What kind of inputs does your system need to handle? You have to get a sense for the scope of the problem before you start exploring the space of possible solutions. And remember, there is no single right answer to a real-world problem. Everything is a tradeoff.

Topics

Systems are complex, and when you’re designing a system you’re grappling with its full complexity. Given this, there are many topics you should be familiar with, such as:

• Concurrency. Do you understand threads, deadlock, and starvation? Do you know how to parallelize algorithms? Do you understand consistency and coherence?
• Networking. Do you roughly understand IPC and TCP/IP? Do you know the difference between throughput and latency, and when each is the relevant factor?
• Abstraction. You should understand the systems you’re building upon. Do you know roughly how an OS, file system, and database work? Do you know about the various levels of caching in a modern OS?
• Real-World Performance. You should be familiar with the speed of everything your computer can do, including the relative performance of RAM, disk, SSD and your network.
• Estimation. Estimation, especially in the form of a back-of-the-envelope calculation, is important because it helps you narrow down the list of possible solutions to only the ones that are feasible. Then you have only a few prototypes or micro-benchmarks to write.
• Availability and Reliability. Are you thinking about how things can fail, especially in a distributed environment? Do know how to design a system to cope with network failures? Do you understand durability?

Remember, we’re not looking for mastery of all these topics. We’re looking for familiarity. We just want to make sure you have a good lay of the land, so you know which questions to ask and when to consult an expert.

How to prepare

How do you get better at something? If your answer isn’t along the lines of “practice” or “hard work,” then I have a bridge to sell you. Just like you have to write a lot of code to get better at coding and do a lot of drills to get really good at basketball, you’ll need practice to get better at design. Here are some activities that can help:

• Do mock design sessions. Grab an empty room and a fellow engineer, and ask her to give you a design problem, preferably related to something she’s worked on. Don’t think of it as an interview — just try to come up with the best solution you can. Design interviews are similar to actual design sessions, so getting better at one will make you better at the other.
• Work on an actual system. Contribute to OSS or build something with a friend. Treat your class projects as more than just academic exercises — actually focus on the architecture and the tradeoffs behind each decision. As with most things, the best way to learn is by doing.
• Do back-of-the-envelope calculations for something you’re building and then write micro-benchmarks to verify them. If your micro-benchmarks don’t match your back-of-the-envelope numbers, some part of your mental model will have to give, and you’ll learn something in the process.
• Dig into the performance characteristics of an open source system. For example, take a look at LevelDB. It’s new and clean and small and well-documented. Read about the implementation to understand how it stores its data on disk and how it compacts the data into levels. Ask yourself questions about tradeoffs: which kinds of data and sizes are optimal, and which degrade read/write performance? (Hint: think about random vs. sequential writes.)
• Learn how databases and operating systems work under the hood. These technologies are not only tools in your belt, but also a great source of design inspiration. If you can think like a DB or an OS and understand how each solves the problems it was designed to solve, you’ll be able to apply that mindset to other systems.

Final thought: relax and be creative

The systems design interview can be difficult, but it’s also a place to be creative and to take joy in the imagining of systems unbuilt. If you listen carefully, make sure you fully understand the problem, and then take a clear, straightforward approach to communicating your ideas, you should do fine.

Good luck!

Note: this part is part two of our series on doing your best in interviews. Part one: “How to Rock an Algorithms Interview”.

Here at Palantir algorithms are important, but code is our lifeblood. We live and die by the quality of the code we ship. It’s no surprise, then, that coding ability is what we stress the most in our interview process. A candidate can get by with mediocre algorithm skills (depending on the role), but no one can skimp on coding.

Suppose you’re confident in your ability to write great software. Your task in a coding interview (of which there will be several) is to show the interviewers that you in fact do have the programming chops — that you’re an experienced coder who knows how to write solid, production-quality code.

This is easier said than done. After all, coding in your favorite IDE from the comfort of $familiar_place is very different from coding on a whiteboard (on a problem you’re totally unfamiliar with) in a pressure-filled 45-minute interview. We realize that the interview environment is not the real world, and we adjust our expectations accordingly. Nonetheless, there are a number of things you can do to put your best foot forward during the interview.

First, though, we’d like to give you a sense for what we look for during a coding interview. Most important is the ability to write clean and correct code — it’s not enough just to be correct. A lot of people will be interacting with your code once you’re on the job, so it should be readable, maintainable, and extensible where appropriate. If your solution is clean and correct, and you produced it in a reasonable amount of time without a lot of help, you’re in good shape. But even if you stumble a bit, there are other ways to demonstrate your ability. As you work, we also watch for debugging ability, problem-solving and analytical skills, creativity, and an understanding of the ecosystem that surrounds production code.

With our evaluation criteria in mind, here are some suggestions we hope will help you perform at your very best.

Before you start coding

• Make sure you understand the problem. Don’t hesitate to ask questions. Specifically, if any of the problem requirements seem loosely defined or otherwise unclear, ask your interviewer to make things more concrete. There is no penalty for asking for clarifications, and you don’t want to miss a key requirement or proceed on unfounded assumptions.
• Work through simple examples. This can be useful both before you begin and after you’ve finished coding. Working through simple examples before coding can give you additional clarity on the nature of the problem — it may help you notice additional cases or patterns in the problem that you would otherwise have missed had you been thinking more abstractly.
• Make a plan. Be wary of jumping into code without thinking about your program’s high-level structure. You don’t have to work out every last detail (this can be difficult for more meaty problems), but you should give the matter sufficient thought. Without proper planning, you may be forced to waste your limited time reworking significant parts of your program.
• Choose a language. At Palantir, we don’t care what languages you know as long as you have a firm grasp on the fundamentals (decomposition, object-oriented design, etc.). That said, you need to be able to communicate with your interviewer, so choose something that both of you can understand. In general, it’s easier for us if you use Java or C++, but we’ll try to accommodate other languages. If all else fails, devise your own pseudo-code. Just make sure it’s precise (i.e. not hand-wavy) and internally consistent, and explain your choices as you go.

While you’re coding

• Think out loud. Explain your thought process to your interviewer as you code. This helps you more fully communicate your solution, and gives your interviewer an opportunity to correct misconceptions or otherwise provide high-level guidance.
• Break the problem down and define abstractions. One crucial skill we look for is the ability to handle complexity by breaking problems into manageable sub-problems. For anything non-trivial, you’ll want to avoid writing one giant, monolithic function. Feel free to define helper functions, helper classes, and other abstractions to reach a working solution. You can leverage design patterns or other programming idioms as well. Ideally, your solution will be well-factored and as a result easy to read, understand, and prove correct.
• Delay the implementation of your helper functions. (this serves a corollary to the previous point) Write out the signature, and make sure you understand the contract your helper will enforce, but don’t implement it right away. This serves a number of purposes: (1) it shows that you’re familiar with abstractions (by treating the method as an API); (2) it allows you to maintain momentum towards the overall solution; (3) it results in fewer context-switches for your brain (you can reason about each level of the call stack separately); and (4) your interviewer may grant you the implementation for free, if he or she considers it trivial.
• Don’t get caught up in trivialities. At Palantir we are much more interested in your general problem solving and coding abilities than your recall of library function names or obscure language syntax. If you can’t remember exactly how to do something in your chosen language, make something up and just explain to your interviewer that you would look up the specifics in the documentation. Likewise, if you utilize an abstraction or programming idiom which admits a trivial implementation, don’t be afraid to just write out the interface and omit the implementation so you can concentrate on more important aspects of the problem (e.g., “I’m going to use a circular buffer here with the following interface without writing out the full implementation”).

Once you have a solution

• Think about edge cases. Naturally, you should strive for a solution that’s correct in all observable aspects. Sometimes there will be a flaw in the core logic of your solution, but more often your only bugs will be in how you handle edge cases. (This is true of real-world engineering as well.) Make sure your solution works on all edge cases you can think of. One way you can search for edge-case bugs is to…
• Step through your code. One of the best ways to check your work is to simulate how your code executes against a sample input. Take one of your earlier examples and make sure your code produces the right result. Huge caveat here: when mentally simulating how your code behaves, your brain will be tempted to project what it wants to happen rather than what actually says happen. Fight this tendency by being as literal as possible. For example, if you’re calculating a string index with code like str.length()-suffix.length(), don’t just assume you know where that index will land; actually do the math and make sure the value is what you were hoping for.
• Explain the shortcuts you took. If you skipped things for reasons of expedience that you would otherwise do in a “real world” scenario, please let us know what you did and why. For example, “If I were writing this for production use, I would check an invariant here.” Since whiteboard coding is an artificial environment, this gives us a sense for how you’ll treat code once you’re actually on the job.

As an addendum, here are a few suggestions for books we like about the art of software construction:

Clean Code: A Handbook of Agile Software Craftsmanship – Robert C. Martin
Code Complete: A Practical Handbook of Software Construction – Steve McConnell
The Practice of Programming – Brian Kernighan, Rob Pike
Design Patterns: Elements of Reusable Object-Oriented Software – Erich Gamma, et al.
Effective Java – Joshua Bloch

We do a lot of interviewing at Palantir, and let me tell you: it’s hard. I don’t mean that we ask tough questions (although we do). I mean that the task of evaluating a candidate is hard.

The problem? Given a whiteboard and one hour, determine whether the person across from you is someone you’d like to work with, in the trenches, for the next n years. A candidate’s performance during an interview is only weakly correlated with his or her true potential, but we’re stuck with the problem of turning the chickenscratch on the whiteboard into an ‘aye’ or ‘nay’. Sometimes it feels like a high-stakes game of reading tea leaves. Believe me we’re doing our best, but we’re often left the nagging worry that we’re passing up brilliant people who just had a bad day or who didn’t click with a particular problem.

In an effort to improve this situation, we wanted to write up a guide that will help candidates make sense of this process, or at least the part known as an Algorithms Interview. At Palantir we ask questions that test for a lot of different skills — coding, design, systems knowledge, etc. — but one of our staple interviews is to ask you to design an algorithm to solve a particular problem.

It usually starts like this:

Given X, figure out an efficient way to do Y.

First: Make sure you understand the problem. You’re not going to lose points asking for clarifications or talking through the obvious upfront. This will also buy you time if your brain isn’t kicking in right away. Nobody expects you to solve a problem in the first 30 seconds or even the first few minutes.

Once you understand the problem, try to come up with a solution – any solution whatever. As long as it’s valid, it doesn’t matter if your solution is trivial or ugly or extremely inefficient. What matters is that you’ve made progress. This does two things: (1) it forces you to engage with the structure of the problem, priming your brain for improvements you can make later, and (2) it gives you something in the bank, which will in turn give you confidence. If you can achieve a brute force solution to a problem, you’ve cleared a major hurdle to solving it in a more efficient way.

Now comes the hard part. You’ve given an O(n^3) solution and your interviewer asks you to do it faster. You stare at the problem, but nothing’s coming to you. At this point, there are a few different moves you can make, depending on the problem at hand and your own personality. Almost all of these can help on almost any problem:

1. Start writing on the board. This may sound obvious, but I’ve had dozens of candidates get stuck while staring at a blank wall. Maybe they’re not visual people, but still I think it’s more productive to stare at some examples of the problem than to stare at nothing. If you can think of a picture that might be relevant, draw it. If there’s a medium-sized example you can work through, go for it. (Medium-sized is better than small, because sometimes the solution to a small example won’t generalize.) Or just write down some propositions that you know to be true. Anything is better than nothing.




2. Talk it through. And don’t worry about sounding stupid. If it makes you feel better, tell your interviewer, “I’m just going to talk out loud. Don’t hold me to any of this.” I know many people prefer to quietly contemplate a problem, but if you’re stuck, talking is one way out of it. Sometimes you’ll say something that clearly communicates to your interviewer that you understand what’s going on. Even though you might not put much stock in it, your interviewer may interrupt you to tell you to pursue that line of thinking. Whatever you do, please DON’T fish for hints. If you need a hint, be honest and ask for one.




3. Think algorithms. Sometimes it’s useful to mull over the particulars of the problem-at-hand and hope a solution jumps out at you (this would be a bottom-up approach). But you can also think about different algorithms and ask whether each of them applies to the problem in front of you (a top-down approach). Changing your frame of reference in this way can often lead to immediate insight. Here are some algorithmic techniques that can help solve more than half the problems we ask at Palantir:
   • Sorting (plus searching / binary search)
   • Divide-and-conquer
   • Dynamic programming / memoization
   • Greediness
   • Recursion
   • Algorithms associated with a specific data structure (which brings us to our fourth suggestion…)




4. Think data structures. Did you know that the top 10 data structures account for 99% of all data structure use in the real world? Probably not, because I just made those numbers up — but they’re in the right ballpark. Yes, on occasion we ask a problem whose optimal solution requires a Bloom filter or suffix tree, but even those problems tend to have a near-optimal solution that uses a much more mundane data structure. The data structures that are going to show up most frequently are:
   • Array
   • Stack / Queue
   • Hashset / Hashmap / Hashtable / Dictionary
   • Tree / binary tree
   • Heap
   • Graph

   You should know these data structures inside and out. What are the insertion/deletion/lookup characteristics? (O(log n) for a balanced binary tree, for example.) What are the common caveats? (Hashing is tricky, and usually takes O(k) time when k is the size of the object being hashed.) What algorithms tend to go along with each data structure? (Dijkstra’s for a graph.) But when you understand these data structures, sometimes the solution to a problem will pop into your mind as soon as you even think about using the right one.




5. Think about related problems you’ve seen before and how they were solved. Chances are, the problem you’ve been presented is a problem that you’ve seen before, or at least very similar. Think about those solutions and how they can be adapted to specifics of the problem at hand. Don’t get tripped up by the form that the problem is presented – distil it down to the core task and see if matches something you’ve solved in the past.




6. Modify the problem by breaking it up into smaller problems. Try to solve a special case or simplified version of the problem. Looking at the corner cases is a good way to bound the complexity and scope of the problem. A reduction of the problem into a subset of the larger problem can give a base to start from and then work your way up to the full scope at hand.

   Looking at the problem as a composition of smaller problems may also be helpful. For example, “find a number in a sorted array which has been shifted cyclically by an unknown constant k” can be solved by (1) first figuring out “k” and then (2) figuring out how to perform binary search on a shifted array).




7. Don’t be afraid to backtrack. If you feel like a particular approach isn’t working, it might be time to try a different approach. Of course you shouldn’t give up too easily. But if you’ve spent a few minutes on an approach that isn’t bearing any fruit and doesn’t feel promising, back up and try something else. I’ve seen more candidates who overcommit than undercommit, which means you should (all else equal) be a little more willing to abandon an unpromising approach.

Incidentally, trying out a few different approaches (rather than sticking with a single approach) tends to work well in interviews, because the problems we choose for an interview usually have many different solutions. Happily, the same is true for the problems we solve on the job =)

public abstract class ForwardingCollection<E> extends ForwardingObject implements Collection<E> {
  @Override protected abstract Collection<E> delegate();

  public boolean add(E element) {
    return delegate().add(element);
  }

  // ... (more overridden methods) ...
}

Things worth noting:

• This class overrides the delegate() method to return a more specific type.
• The class contains no instance variables and does not have to be marked Serializable.

Now, here’s an example of how to implement a decorator using a forwarding class (also from Google Collections):

  public class ConstrainedCollection<E> extends ForwardingCollection<E> {
    private final Collection<E> delegate;
    private final Constraint<? super E> constraint;

    public ConstrainedCollection(Collection<E> delegate, Constraint constraint) {
      this.delegate = checkNotNull(delegate);
      this.constraint = checkNotNull(constraint);
    }
    @Override protected Collection<E> delegate() {
      return delegate;
    }
    @Override public boolean add(E element) {
      constraint.checkElement(element);
      return super.add(element);
    }
    @Override public boolean addAll(Collection<? extends E> elements) {
      return super.addAll(checkElements(elements, constraint));
    }
  }

This class implements a collection that checks a constraint on all elements added to the collection. Note that writing the decorator is easy given the forwarding class — only the methods relevant to the decorator need to be overridden.

Here at Palantir, a lot of our automatic tests are full-chain tests. A backend server is fired up, client code runs against it, and everything runs much like a production environment. This makes intuitive sense because it’s a faithful approximation of how the system will run in the field.

However, there are some disadvantages to this:

• Full-pass tests don’t always localize the problem. Tests on a client class might fail even if it was the service that behaved incorrectly.
• These full-pass tests are relatively slow. Client code is running against an actual remote service. If a client is being tested, the server code still has to do work — sometimes a lot of work — even if that isn’t the focus of the test.
• The constraints of the test are loose. Full-chain tests can mostly only see whether the operation finished correctly. It’s much harder to figure out whether the operation was done efficiently and without making unnecessary service calls.
• They’re very little setup flexibility. If you want an RPC to return a specific value, you have little choice but to have your test get the service into a state where it can return that value. This is easy in some cases, but prohibitively difficult in others.
• Client tests are forced to share any non-determinism leaked from the service. For example, under real conditions, a request to call A might respond before call B, and sometimes the other way around. This can result in flaky tests or tests that don’t always simulate the conditions you want to exercise.

What’s to be done? Fortunately, there’s an option that handles these cases elegantly. We also test with jMock, a library that dynamically generates mock objects from arbitrary interfaces. These mock objects can be configured to check that particular methods are called with particular inputs a particular number of times, and then give prescribed responses.

Hit the link to see a concrete example of jMock in action.

jMock in action

Let’s say I want to test my object viewer page in Palantir Web, but I don’t want to fire up a dispatch server at all. First, I create my mock service object.

Mockery context = new Mockery();
final PalantirService service = context.mock(PalantirService.class);

Then, I set the expectations of my mock object. In this case, I want to tell my mock object to expect a call to PalantirService.getObject() and PalantirService.getDataSources(). getObject() will return a specific object. Any call made to the service apart from these will make the test fail.

context.checking(new Expectations() {{
        oneOf(service).getObject(realm.getId(), myObject.getId());
        will(returnValue(myObject));
        oneOf(service).getDataSources(myObject.getDataSources());
}});

Now, I create the object I want to test and inject the service.

ObjectViewController controller = new ObjectViewController();
controller.setService(service);

And then we fire away.

ModelMap model = new ModelMap();
controller.doGet(myObject.getId(), model);

Now that the controller (the class we’re exercising) has gone off and populated the model, we check to see that the model is populated correctly. Just like we would in any other test.

assertEquals(myObject.getName(), model.get("objectName"));
assertEquals(myObject, model.get("object"));

But in addition, we also assert that the expectations specified above were satisfied.

context.assertIsSatisfied();

Not only can we be sure that the right calls were made with the right parameters, but we can also be sure that no calls besides the expected calls were made. So the next time you want more speed or control over your tests, take a look at jMock or another framework like it. It’s a powerful tool in the effort to test your best!

Here at Palantir we use test-driven development (or TDD for short). Integrated tools like Eclipse and JUnit simplify writing and running unit tests. However, once you need to test a broader swath of functionality, it’s time to write functional, integration, and system tests. While technically not ‘unit testing’, the testing framework that JUnit provides is basically the same infrastructure that you want to leverage for writing these more involved types of testing.

When you’re developing enterprise software, functional testing often means getting your clients to talk to your servers. For the main Palantir Government product, we integrate the process of bringing the server up and down with the Ant scripts that run our automated unit tests: our testing tasks bring up the server, run the test suite, and then kill the server. This works great and produces nice results.

When I started working on our authentication server, the pattern that we had used before didn’t work for me. While the Palantir Government tests ran with a single, static configuration file, I needed to run the authentication server with multiple configurations in the course of running through the all the different functional tests. I determined that I needed a way to programmatically bring the server up and down for testing. In JUnit parlance, I needed a way to programmatically launch the server component as part of my setup() function for my unit tests and stop it in my teardown().

With my itch-to-scratch firmly in hand (or some other mixed metaphor), I set out to figure out how to invoke new Java processes from inside a unit test. The solution I came up with (with source code and examples) after the jump.

The Six Ingredients

So there are six ingredients that go into spawning a new VM:

• The classpath to use for the new VM
• The name of the class to run
• The directory to be used as the current directory for the process
• The command line arguments to pass to the process
• The set of Java system properties to use for this process
• The environment to pass to the process

Let’s look at each item individually.

Classpath

The classpath will tell the spawned VM where to load classes from. In JavaInvoke, we use the existing classpath (from the spawning VM) as a starting point and then prepend any new entries to allow overriding the classpath for the spawned VM.

This takes a lot of the tedium out of having to figuring out what to put in the classpath. Most likely, you want something similar to what you already have, if not completely identical.

We get the classpath from System.getProperty("java.class.path") and can add new entries by prepending the new entry, using the value of File.pathSeparatorChar as the entry delimiter. Using File.pathSeparatorChar makes the code cross-platform friendly (since the path separator is ‘;’ on Windows and ‘:’ on Unix (Linux, Solaris, OS/X, etc.).

Caveat: if you change the working directory and your original classpath was constructed using relative paths, you’ll probably have trouble getting anything to run (since your classpath will no longer point to right locations).

Class name

Pretty simple: what do you want to run in the spawned VM? The class must have a static void main(String args[]) defined, and it must be available for loading via the classpath.

Working Directory

If it should be different from the current working directory (CWD) of the running process, then set it and JavaInvoke will change it in the environment.

Command line arguments

If the process needs any command line arguments, including VM options, specify them in a string array. Note that not all of these arguments will necessarily make it to your main method, since the VM executable will parse it first and remove the VM arguments, passing through the program arguments.

Java System Properties

System properties can be used to control many aspects of how a VM runs. You can set them programmatically in your code or you can set set them on the command line by passing -Dkey=value. Our JavaInvoke implementation will take a Map of properties as a convenience argument; all it does is rewrite the map into the command line.

Process environment

This is an operating-system level construct. This is the set of environment variables, also in a Map that you would like merged with the current environment. This would be the place that you set things like LD_LIBRARY_PATH on Unix.

Dealing with input and output

So you might ask the question, “where does the output from the process go?” Or more troubling, “How do I send the process some input?” The Java Process object has methods to deal with this, allowing you to get streams that give you access to the input, output, and error streams of spawned process. That API is straight-forward to deal with, just like any other use of the java.io streams.

However, we want to make the typical case really easy: pulling the output from the spawned process back to the parent that spawned it. To that end, we add into the mix a class called OutputPiper. It fires up a thread that pulls all input from the spawned process, tags it with an identifier, and then outputs to the spawner’s stdout/stderr.

OutputPiper

(as extracted from ProcessSpawner.java)

	public static class OutputPiper extends Thread  {
		InputStream in;
		PrintStream out;
		String tag = null;

		public OutputPiper(String tag, InputStream in,PrintStream out) {
			this.in = in;
			this.out = out;
			this.tag = tag;
			// make sure that we don't keep the VM alive
			this.setDaemon(true);
			this.setName("OutputPiper-" + tag);
			out.println("Starting output piper for tag: " + tag);
			this.start();
		}

		@Override
		public void run() {
			try {
				BufferedReader reader = new BufferedReader(new InputStreamReader(in));
				String line = null;
				do {
					line = reader.readLine();
					if(line != null) {
						out.println(tag + ": " + line);
					}
				}while(line != null);
			}
			catch (Exception e) {
				//
			}
			out.println("Output piper exiting for tag: " + tag);
		}

		public static OutputPiper createOutputPiper(String tag, InputStream in, PrintStream out) {
			OutputPiper rc = new OutputPiper(tag, in,out);
			return rc;
		}
	}

Outpiper extends Thread so that all the output will arrive back to the controlling process in a timely manner. For each given process, we spawn off two OutputPipers, one for stdout and one for stderr, corresponding to the Process.getInputStream() and the Process.getErrorStream().

ProcessSpawner & JavaInvoke

There are two key classes in the example:

• ProcessSpawner.java – Essentially a wrapper around ProcessBuilder, a generic process spawner that makes it simple to invoke processes that that use OutputPipers to forward their output back to their parent. This class allows you to specify the working directory, process environment, and command line for the process to be invoked.
• JavaInvoke.java – a specialized subclass of ProcessSpawner, this class makes spawning new VMs a piece of cake, doing the necessary translation for Java system properties, setting the proper classpath environment variable with potential overrides, and fills in the fully qualified class name to run.

The Example & Source Code

I’ve put together a running example that implements a trivial client and server in JUnit test. The setup() method spawns the server and then the tests run the client code against the server, tearing it down after each test. It’s available in the PalantirVMSpawnerExample.zip zip file. Unzip it, run the run.sh or run.bat script as appropriate. It should generate output that looks like this:

-----------------------------------------------------
Starting test testAck
INFO [main] JavaInvoke - CLASSPATH=./lib/devblog-vmspawner.jar
INFO [main] ProcessSpawner - Build process spawner for the following command line:
INFO [main] ProcessSpawner - /home/pteng/java/i586/jdk1.5.0_14/jre/bin/java com.palantir.blog.processspawner.Server
Starting output piper for tag: server-stdout
Starting output piper for tag: server-stderr
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler2]: Got message: some message
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler3]: Got message: SHUTDOWN
Output piper exiting for tag: server-stdout
Output piper exiting for tag: server-stderr
Finished test testAck
-----------------------------------------------------
-----------------------------------------------------
Starting test testShutdown
INFO [main] JavaInvoke - CLASSPATH=./lib/devblog-vmspawner.jar
INFO [main] ProcessSpawner - Build process spawner for the following command line:
INFO [main] ProcessSpawner - /home/pteng/java/i586/jdk1.5.0_14/jre/bin/java com.palantir.blog.processspawner.Server
Starting output piper for tag: server-stdout
Starting output piper for tag: server-stderr
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: Spawning socket handler
server-stdout: Waiting for connection
server-stdout: [Socket Handler3]: Got message: SHUTDOWN
Output piper exiting for tag: server-stdout
Output piper exiting for tag: server-stderr
Took 3 ms to send shutdown.
Took 335 ms for process to die.
Finished test testShutdown
-----------------------------------------------------
SUCCESS: all 2 tests passed

The source is included in the zip file, but if you wanted to look at it or link to it on the web, here are the classes involved:

• Client.java
• Example.java
• JavaInvoke.java
• ProcessSpawner.java
• Server.java
• ServerSpawningTest.java

And as an added bonus, there’s an Ant build.xml that will let you tweak and rebuild the demo yourself.

Comments and questions welcome. Enjoy.

The solution was to select a single event firing system. We wanted something that was easy-to-use yet powerful enough to express all the changes that might be made to a data model. Java’s Property Change Support (PCS) was a good fit because it can support arbitrary events in a very lightweight fashion.

Read on for details of our implementation…

Property Change Support

Java’s PropertyChangeSupport class (PCS) basically allows an object to easily fire events consisting of 4 pieces of information:

• source object – the thing that fired the event
• property name – allows the listener to tell events for different things apart
• old value – the old value for the property
• new value – the new value for the property

PCS handles all the bookkeeping for adding and removing listeners and firing events. It is very useful for creating listenable models, but we wanted to make it just a little bit easier by having an abstract class that exposed the add/remove listener calls and took care of initializing PCS:

public abstract class AbstractListenableModel implements Serializable {

    private static final long serialVersionUID = 1L;

    private transient PropertyChangeSupport pcs;

    protected AbstractListenableModel() {
        this.init();
    }

    /**
    * Adds a property change listener to the model.
    */
    public final void addPropertyChangeListener(PropertyChangeListener listener) {
        this.pcs.addPropertyChangeListener(listener);
    }

    /**
    * Removes a property change listener from the model.
    */
    public final void removePropertyChangeListener(PropertyChangeListener listener) {
        this.pcs.removePropertyChangeListener(listener);
    }

    /**
    * Fires a property change event to listeners of the model.
    */
    protected final void firePropertyChange(String propertyName, Object oldValue, Object newValue) {
        this.pcs.firePropertyChange(propertyName, oldValue, newValue);
    }

    /**
    * Initializes transient fields during deserialization.
    */
    protected Object readResolve() {
        this.init();
        return this;
    }

    /**
    * Initializes transient fields.
    */
    private void init() {
        this.pcs = new PropertyChangeSupport(this);
    }
}

AbstractListenableModel is basically just a simple wrapper for exposing the functionality of PCS. By extending this abstract class, it’s very easy to create a listenable model:

public final class MyModel extends AbstractListenableModel {

    public static final String PROP_FOO = MyClass.class.getName() + ".Foo";

    private int foo;

    public int getFoo() {
        return this.foo;
    }

    public void setFoo(int foo) {
        //
        // The semantics of the following line are a little hard to unpack,
        // but it does exactly what it needs to do, and the tradeoff
        // for conciseness over immediate readability is worth it for
        // large models with lots of properties.
        //
        // First, the JVM starts to create a stack frame for the call
        // into firePropertyChange().  It begins binding parameter values
        // from the left to the right.  The pointer to the String contained
        // in PROP_FOO is passed in first, then the current value of
        // this.foo is passed in, then the expression
        //        this.foo = foo
        // is evaluated (setting this.foo to the new value of foo), which
        // returns the new value of foo.  All the parameters are then
        // passed down into firePropertyChange(), which checks whether
        // the oldValue is equal to the newValue.  If they're not equal,
        // it fires the event.  If they are equal, it ignores the event.
        //
        this.firePropertyChange(PROP_FOO, this.foo, this.foo = foo);
    }
}

In this example, MyModel contains a single property called foo. When the value of foo is changed, a property change event will be fired to listeners of the model.

You may notice that the value of PROP_FOO is prefixed by the name of the class. This ensure that naming collisions do not occur for scenarios in which the same listener is used to listen to multiple models which happen to use the same property name. This scenario becomes much more likely in the case of event bubbling, which I’ll talk about next.

Event Bubbling

Imagine a scenario in which we have a nested model:

Normally, if a listener needs to receive events from both models A and B, it will need to add itself as a listener to each individual model. While this solution would work, it’s a little cumbersome, especially when model B can get swapped out for model B’—the listener then has to keep itself synched to the internal state of model A. It would be nice if model A could just automatically forward all the events from model B (or B’) via its PCS support so that a listener only needs to attach itself to one model instead of multiple models. With a bit more code in AbstractListenableModel, this is possible:

public abstract class AbstractListenableModel implements Serializable {

    private transient PropertyChangeListener childModelListener;

    ...

    /**
    * Registers a child model to this model.
    */
    protected void registerChildModel(ListenableModel childModel, String propertyName) {
        childModel.addPropertyChangeListener(this.childModelListener);
    }

    /**
    * Initializes transient fields.
    */
    private void init() {
        ...
        this.childModelListener = new ChildModelListener();
    }

    /**
    * Listener for property change events fired from child models.
    */
    private final class ChildModelListener implements PropertyChangeListener {
        public void propertyChange(PropertyChangeEvent event) {
            // This is where the bubbling happens
            pcs.firePropertyChange(event);
        }
    }
}

Now, whenever model B fires a property change event, this event will also be fired by model A. This makes it much easier for the listener to listen to events arbitrarily deep in the model hierarchy, because each event fired by a child model gets re-fired (bubbled) by all its ancestors. All you have to do is attach a listener to the root model and you’ll automatically receive events from all models in the hierarchy.

Note that the registerChildModel method above takes an unused propertyName argument. In the full implementation of this class, events with the provided property name are monitored. When an event with the provided property name is fired, childModelListener is detached from the old child model and attached to any new child model. This ensures that the listenable model is always listening to the current child models.

Events for Collections

Any model event support would not be complete without some consideration of how to handle collections such as sets and lists. To solve this scenario, we created specialized collection classes called ListenableModelSet and ListenableModelList. These collections hold AbstractListenableModels as their elements and fire events whenever their contents change. Since the changes to collections can vary widely, the solution we came up for communicating collection changes with full fidelity is basically to fire events with a copy of the old set as the old value and the new set as the new value. Listeners can then diff the old and new values to determine exactly what changed if necessary. Additionally, each ListenableModelSet or List adds a ChildModelListener to all of its children (themselves AbstractListenableModels), thereby ensuring that events are bubbled from all models in the collection.

Conclusion

Just as we saw with the Adapter piece of the MVA triad, when we componentize the Model piece there are huge gains to be had. Once we started using a base Model class and a consistent eventing infrastructure (PropertyChangeSupport), we could add features that made coding across our entire application a lot more pleasant.

Model–View–Controller (MVC) is an architectural pattern used in software engineering. Successful use of the pattern isolates business logic from user interface considerations, resulting in an application where it is easier to modify either the visual appearance of the application or the underlying business rules without affecting the other. In MVC, the model represents the information (the data) of the application; the view corresponds to elements of the user interface such as text, checkbox items, and so forth; and the controller manages the communication of data and the business rules used to manipulate the data to and from the model.

They go on to show the classic triangle diagram and how it’s baked into various GUI and web frameworks. There’s only one clause in the entire article that hints at something deeper: “Though MVC comes in different flavors…”

Different flavors indeed. In fact MVC is not just a pattern but a whole family of patterns: MVC, MVA, MVP, PAC, Model-Delegate…. It very quickly gets very hairy.

In this article I want to describe one of MVC’s lesser-known variants, the Model-View-Adapter (MVA) pattern, and talk about its advantages over traditional MVC in the context of a Java Swing application.

Architecture

The best place to start is with an architecture diagram. While vanilla MVC is a triangle:

MVA puts the Adapter in a position to strictly mediate between Model and View:

Here a solid line represents a direct relationship while a dashed line represents an indirect relationship via the Observer pattern. Put another way, the Adapter holds a pointer both to the Model and to the View and directly calls methods on both. At the same time, it attaches itself as a listener both to the Model and to the View in order to receive events. It receives property change events from the Model and action events (checkbox ticked, text entered, etc.) from the View, and then routes appropriate changes to the other side. The Adapter is entirely responsible for keeping the Model and the View in sync; the Model and View are both relatively dumb structures, knowing nothing about the other.

The advantages to organizing code this way are:

• All “moving parts” are centralized in one place, the Adapter. No worrying about where to add a listener; no hunting around to find isolated listeners.
• Separation of concerns between the View and the Adapter. The View is responsible for layout and visual presentation while the Adapter is responsible for synchronization and the dynamic aspects of the user interface.
• Better decoupling between Models and Views. Specifically, the View doesn’t need to know anything about the Model.

Additionally, while it will never be possible to fully componentize any variant of the MVC pattern, MVA is more amenable to componentization and thus more of its implementation can be centralized (in a single class) and reused. Once componentized, we can augment the basic functionality with things like:

• Automatic registration and unregistration of listeners when the View enters and exits the Swing component hierarchy, thereby preventing certain kinds of memory leaks.
• Automatic unregistration of listeners when the program shuts down. This can help free up resources like realtime subscriptions.
• Method for swapping a new Model object in for an old Model object.
• Ability to execute a task without listeners attached, to help prevent event-action-event loops.

The downside to using MVA over MVC is that the Adapter tends to take on a lot of the responsibility and can get quite complicated. But in my experience that can be mitigated by having good conventions about which pieces (M, V, A) are allowed to communicate with which other pieces and at what times. Enforcing predictable control flow goes a long way toward managing complexity.

Read on for a code-level description of our implementation of the MVA pattern.

Palantir MVA Implementation

Our half-componentization of MVA resides in a single abstract class named Adapter:

public abstract class Adapter<ViewType extends Component, ModelType> {
// constructor
protected Adapter(ViewType view, ModelType model); { ... }

/**
* Attach listeners to the View's subcomponents (checkboxes etc.).
* Listeners should be stored as member variables in the Adapter
* subclass.
*/
protected abstract void registerViewListeners();

/**
* Detach the same listeners (member variables) that were
* attached in registerViewListeners().
*/
protected abstract void unregisterViewListeners();

/**
* Attach listener(s) to the Model.
*/
protected abstract void registerModelListeners();

/**
* Detach the same listeners (member variables) that were
* attached in registerModelListeners().
*/
protected abstract void unregisterModelListeners();

/**
* Bring the View fully in synch with the Model.  Typically
* this involves querying state from the Model and
* reconfiguring subcomponents of the View accordingly.
*/
protected abstract void fullSynchronize();

protected ModelType getModel() { ... }
protected ViewType getView() { ... }

// other methods elided
}

New View components that want to stay synchronized with a Model must instantiate a subclass of Adapter and implement the abstract methods. The Adapter parent class (itself an example of the Template Method design pattern) will then call into the appropriate abstract methods at the appropriate times. For example, after the View is constructed, as soon as it’s displayed in the Swing component hierarchy the Adapter parent class will automatically call fullSynchronize() (whose implementation should bring the View in line with the Model) and then registerViewListeners() and registerModelListeners(), so the Adapter is poised to react to events. Likewise, when the View is removed from the component hierarchy (when its containing frame is closed, say), both unregisterViewListeners() and unregisterModelListeners() will be called. This can help ensure that no memory will be leaked when a long-life-cycle object (like a system-wide singleton) retains a pointer to a short-life-cycle object (the View) via the Observer pattern.

Dealing With Listener Loops

One problem that confronts UI developers is the problem of “listener loops”: infinite loops that result when the View fires an event, the Adapter (or Controller) responds to it by setting some property on the Model, and an event is propagated from the Model back to the View, starting the whole cycle over again.

One way to combat this is to make sure your Model only fires events when the value that’s being set on the Model is different from the value currently stored in the Model. (This will cut off the infinite loop after one and a half cycles.) It’s a good practice but often isn’t enough, especially when your system is multithreaded and events start to queue up. You can sometimes get into situations where an M-V-C triplet will thrash forever between two different values for one of the Model’s properties.

Our solution to this problem is a protected method (on our Adapter base class) called runWithoutViewListeners:

/**
* Guarantees that the job r will be run:
*    - on the Swing thread
*    - with Model listeners attached
*    - with View listeners DEtached
*/
public final void runWithoutViewListeners(final Runnable r) { ... }

The implementation of this method checks to make sure the view listeners are attached when it’s called, detaches them via a call to unregisterViewListeners(), invokes the Runnable, then reattaches the view listeners via a call to registerViewListeners(). The code inside the Runnable can then make whatever changes it wants to the View without perturbing the Model downstream. Listener loop averted!

More To Come

I hope that’s given you some sense of the territory out there in the wide world of MVC-variants. In a week or two, Derek will show off some of the work he’s done on the M piece of the MVA triad related to “event bubbling.” Stay tuned!

It’s always fun to release a new piece of jargon into the wild. I’ve run into a number of bugs in our codebase that caused by an anti-pattern I’d like to dub The Pokémon Problem.

Much like the game of Whac-a-Mole, this is a class of bugs where fixing every occurrence does not prevent the bug from returning in new code: it is easy for code delta to result in an instance of the bug being re-introduced into the code base. Even if you “catch ‘em all“, nothing prevents someone else from introducing new Pokémon bugs later.

Not only is this bug easy to re-introduce, but it sometimes can be hard to find all currently existing instances of this pattern. Although tools like Eclipse make it easier to track down all the places that code is called, sometimes you’re looking for things that happen in a certain sequence (which tools like Eclipse don’t do a good job of searching for) and dynamic invocation mechanisms like Java Reflection can sometimes make it impossible to be exhaustive. This type of bug is also resistant to automated refactoring: changing the protocol of dealing with this corner of your code will require you to track down all places it was touched and manually refactor them. It generally signals a failure to use sufficient separation of concerns.

In general, this anti-pattern is a result of APIs that require the caller to be responsible for state management of resources that the API owns. This can include things like an object that requires the caller to have run an initialization method before calling any other method on the object. These bugs get even more insidious when a failure to do things in the right order does not cause a hard failure (like throwing an exception) but instead creates some sort of subtle corruption that may not be noticed or cause subsequent calls to fail unexpectedly.

Read on for some strategies on dealing with the Pokémon problem.

Solving the Pokémon Problem

How do we solve the Pokémon problem?  Sometimes it is as easy as writing a unit test to verify that no new Pokémon exist. More often than not, though, you’ll have to use better encapsulation to solve the problem. If the Pokemon problem is the anti-pattern, encapsulation is its opposite.

For an example, I’ll turn to a real problem I ran into in the Palantir Government codebase: Let’s say I have class Parser and it has a method getAttribute(). I want to add the ability to have shortened (obfuscated) tag names to the Parser class.

The wrong approach

One approach I can take:

1. Create a class called AttributeHandler that wraps calls to Parser.getAttribute(). It handles detection of the short or full XML dialect and then returns the appropriate attribute value.
2. Stick an instance of this AttrbributeHandler class in a member variable of Parser and make sure to call AttributeHandler.getAttribute() instead of Parser.getAttribute() when processing attributes.
3. Put comment on Parser.getAttribute() mentioning that it shouldn’t be called directly anymore.

I just introduced a Pokémon problem.  The next person that edits this code may not read my comment and know that calling Parser.getAttribute() is a bug.  They may not test the short tag case and find the bug. How do I fix this Pokémon problem? 

The right approach

Approaching this encapsulation problem has a number of different approaches one could take. Here’s what I chose:

1. Create a class called AttributeHandler that wraps calls to Parser.getAttribute(). It handles detection of the short or full XML dialect and then returns the appropriate attribute value.
2. Stick an instance of this AttrbributeHandler class in a member variable of Parser.
3. Create a wrapper class around Parser that delegates the getAttribute() call to the AttributeHandler member when processing attributes.

(You might ask why I used a delegate instead of sub-classing the parser and overriding the getAttribute() method: it didn’t fit for architectural reasons that aren’t relevant here). Now the developer no longer has access to the Parser.getAttribute() method that would cause the undesired behavior.

In-depth Example: Resource Management

Another common place you might run into the Pokémon problem is code that needs to clean up resources.  Some coders are lazy and don’t want to always write their try/finally/close blocks properly. This can lead to resource leaks; for things like file handles, this usually isn’t a large issue, but for scarce resources like database connections, it’s critical that things get cleaned and returned to the pool. For locks, it’s absolutely essential to avoid deadlocks.

The wrong way

Here’s a simple example of bad resource management. In this (admittedly contrived) scenario, we have class that’s managing a resource for us, in this case it’s a status file. Different parts of the app need to read the status file where, presumably, something is storing its status.

Here’s the naive implementation of the status manager class:

public class PokemonStatusFileManager {

	final File statusFile;

	public PokemonStatusFileManager(File statusFilePath) {
		this.statusFile = statusFilePath;
	}

	public InputStream getStatusFileInputStream() throws FileNotFoundException {
		return new FileInputStream(this.statusFile);
	}
}

Based on this implementation, here’s some code that would use it. Note that first method, parseStatusFileIncorrectly() does no error checking and may not close the InputStream properly if an exception is thrown. The second method does proper resource handling, but it’s kind of ugly to read.

public class PokemonParseStatusFile {

	/**
	 * This is not proper resource management.
	 * @param manager
	 * @throws IOException
	 */
	public static void parseStatusFileIncorrectly(PokemonStatusFileManager manager)
		throws IOException {

		InputStream statusFile = manager.getStatusFileInputStream();
		readFrom(statusFile);
		statusFile.close();

	}

	/**
	 * This is proper resource management, but it's tedious to have to write.
	 * Some coders are too lazy to always do this the right way.
	 * @param manager
	 * @throws IOException
	 */
	public static void parseStatusFileCorrectly(PokemonStatusFileManager manager)
		throws IOException {

		InputStream statusFile = null;
		try {
			statusFile = manager.getStatusFileInputStream();
			readFrom(statusFile);
		} finally {
			// carefully close the resource
			if(statusFile != null) {
				try {
					statusFile.close();
				} catch(Exception e) {
					System.err.println("Error closing statusFile!");
					e.printStackTrace(System.err);
				}
			}
		}
	}

	static void readFrom(InputStream statusFile) throws IOException {
		// do the reading here...
	}
}

So this produces a classic Pokémon problem: everyone who interacts with the StatusFileManager has to do proper resource handling. Now imagine that this is an important lock instead of just a file handle: this Pokémon problem could cause deadlock (which would be bad).

So how do we fix this? The Visitor Pattern.

Solving the Pokémon problem with the visitor pattern

The Visitor Pattern is a potent weapon in fighting the Pokémon problem: it allows you to fully encapsulate access to a resource by injecting into the places it’s needed but preserving overall control of the resource in the code that “owns” the resource. Classically used for controlling things like iteration order, here the visitor pattern is applied as a form of dependency injection to enable lifecycle management.

The visitor pattern as applied to resource management is fairly straightforward:

1. Define an interface, ResourceVisitor with one method, visit(Resource r) (where Resource is the type of resource we’re managing.
2. Define the resource manager with a method that takes a ResourceVisitor as a parameter. Manage the lifecycle of the resource and call ResourceVisitor.visit(r) when appropriate, handling all initialization, error-handling and cleanup.

Here’s our re-spin of the status file manager class. You’ll notice that we’ve moved the resource handling code from the correct example above and encapsulated it in the visitor pattern:

public class UnPokemonStatusFileManager {

	/**
	 * Visitor interface implemented by callers wishing to
	 * interact with the status file.
	 */
	public static interface StatusFileVisitor {
		public void visitStatusFile(InputStream statusFile) throws IOException;
	}

	final File statusFile;

	public UnPokemonStatusFileManager(File statusFilePath) {
		this.statusFile = statusFilePath;
	}

	/**
	 * Note that this method is now private.
	 */
	private InputStream getStatusFileInputStream() throws FileNotFoundException {
		return new FileInputStream(this.statusFile);
	}

	/**
	 * Here's the method that takes the visitor and
	 * fully encapsulates the lifecycle of the status file.
	 */
	public void parseStatusFile(StatusFileVisitor parser)
	throws IOException {
		InputStream statusFile = this.getStatusFileInputStream();
		try {
			parser.visitStatusFile(statusFile);
		} finally {
			// carefully close the resource
			try {
				statusFile.close();
			} catch(Exception e) {
				System.err.println("Error closing statusFile!");
				e.printStackTrace(System.err);
			}
		}
	}
}

Now that we’ve encapsulated the complexity of the resource handling the UnPokemonStatusFileManager class, code that needs to access the status file can be written in a highly correct manner without much work.

public class UnPokemonParseStatusFile {

	public static void parseStatusFile(UnPokemonStatusFileManager statusFileManager)
		throws IOException {

		// generate anonymous visitor to do the processing
		StatusFileVisitor visitor = new StatusFileVisitor() {
			public void visitStatusFile(InputStream statusFile) throws IOException {
				readFrom(statusFile);
			}
		};
		statusFileManager.parseStatusFile(visitor);
	}

	static void readFrom(InputStream statusFile) throws IOException {
		// do the reading here...
	}
}

By encapsulating the full lifecycle of the status file in the visitor pattern, I’ve ensured that it’s always accessed properly. If I change the place where we store status to be a database rather than a file, nothing needs to change except this one class; the calling code remains the same.

We now have easy re-factoring, no resource leaks, and have simplified calling code. And finally: there are no new bugs to be introduced by callers that aren’t sure how to use our resource. Looks like we caught ‘em all!

Wrapping It All Up

So there it is, your new piece of jargon: The Pokémon Problem anti-pattern. You heard it here first! Please post any other great examples to the comments section on this post.