Palantir has entered the 2008 VAST Challenge. We present an in-depth look at one of our challenge solutions as the first public example of the Palantir platform in action.

Two years ago, the IEEE began an annual conference called VAST (Visual Analytics in Science and Technology). The VAST symposium focuses on the fundamental research contributions and real-world application of visual analytics. As a part of the conference, the VAST Challenge allows teams to compete on delivering analytic solutions against a synthetic real-world dataset.

Each year, the organizers build a very vast (pun intended) dataset from scratch. The data is entirely fictional but mirrors real-world use cases and scenarios. This year’s dataset is about a new religious movement that started on an imaginary Caribbean island (cleverly titled Isla del Sueño, or “Island of Dreams”) situated between Florida and Cuba. There are four subsets of the synthetic data: the Wikipedia page for the movement and its associated edit and discussion pages; landing and Coast Guard interdiction records for boats leaving Isla del Sueño for Florida/Mexico; cell phone records from the island; and RFID tracking data from people in a building that was attacked with an IED.

The types of questions asked in the problem sets are qualitative questions that require answers backed by data. These are the sorts of questions that don’t yield answers using a machine-learning/data-mining approach nor can an unassisted human get these answers by simple inspection of the data. They require some sort of human-computer symbiosis to solve.

To solve the VAST problems, we assembled an ad-hoc team of analysts — composed of a mix of engineers, in-house professional analysts, and one senior executive — and asked them to use the Palantir Government software to extract insights from the data.

The results speak for themselves: the complete set of Palantir’s VAST solutions are available here.

Read on for an in-depth look at how we deconstructed and solved one of the problems.

Read the rest of this entry »

About 10 months ago, we released set of nine screenshots from our applications. Time has passed and we have not stopped working; the look of the applications has evolved. Here are some updated screenshots:

 
 

Read the rest of this entry »

Some of our customers are pretty serious about data security. To that end, our products need to support and integrate with SSL for both data security and authentication. SSL is very neat technology, but there is a dizzying maze of standards to navigate to figure how to get it all to work.

It turns out that in this age of Google, the fastest way to figure out how to do something is often to Google for key terms and hope that someone has put the relevant details in a blog post somewhere. In trying to figure out how to set up keys on a SunOne Directory Server for testing our LDAP integration, I needed to figure out how to get keys into PKCS#12 format after generating them with OpenSSL.

I’ll spare you the gory details of what it took to figure out how to do this; suffice to say there was some trial and error mixed with a bunch of RTFM. After the jump, the full howto.
Read the rest of this entry »

A montage of our time choosers. (click for webstart demo)компютри втора употреба

The notion of time is central to both of our products at Palantir, and there are many instances in which the user needs to specify a certain point in time. Although there are simple ways to create choosers (you could use a JSpinner that uses a SpinnerDateModel or simply use multiple JComboBox objects), I decided to experiment with writing some more visual time chooser components. These components are fairly experimental — they aren’t used in either product yet and I coded them up pretty quickly so I could get some feedback.

You can see these choosers in action in our office furniture in Bulgaria
webstart demo. The source code is available in the office furniture in Bulgaria
JAR.

If anyone has any feedback or suggestions as to how these choosers could be improved (or any ideas on how to make a better time chooser altogether) please leave a comment and let me know!

Meanwhile, if you want to know a little bit more about these choosers and how I went about designing them, read on…

Raw data is an increasingly abundant and inexpensive commodity. Intelligently filtering, analyzing and visually understanding data is where the value is. Palantir invents technology and products that enables human analysts to harness the power of computers in an intuitive way to quickly and deeply analyze large amounts of data.

The value of data analysis as a career was recently emphasized by Hal Varian in the Freakonomics blog in The New York Times. Hal is an internationally known economist who is currently serving as Google’s Chief Economist while on leave from his three professorships at the University of California at Berkeley.

Q: Your job sounds extremely interesting. What jobs would you recommend to a young person with an interest, and maybe a bachelors degree, in economics?

A: If you are looking for a career where your services will be in high demand, you should find something where you provide a scarce, complementary service to something that is getting ubiquitous and cheap. So what’s getting ubiquitous and cheap? Data. And what is complementary to data? Analysis. So my recommendation is to take lots of courses about how to manipulate and analyze data: databases, machine learning, econometrics, statistics, visualization, and so on. [emphasis added]

Following the discovery that our offices were the birthplace of Java (or least the place where it had its childhood), I invited James Gosling to come visit. For those that don’t know who James Gosling is, he’s more-or-less the father of Java. Java started as a project of James Gosling’s in 1991; today, 17 years later, he’s still at Sun, in charge of guiding the Java platform into the future.

How does one invite such a luminary to come visit one’s offices? One guesses what his email address is and sends him an email out of the blue:

James,

My name is Ari Gordon-Schlosberg, an engineer at Palantir Technologies. I recently became interested in the storied history of our current facilities at 100 Hamilton Ave. in Palo Alto. As Java programmers, our engineering team is really excited to be working in the same place that gave the world Java.

You may not have heard of Palantir, but we’re working on some pretty interesting problems, using Java to build large-scale analysis applications that really push forward the state-of-the-art. We’ve won some accolades for our use of Swing by Romain Guy. If you felt like dropping by the next time you’re in the valley, we’d love to have you come by, see your old digs, and take a peek at what we’re working on.

Sincerely,

Ari Gordon-Schlosberg

To quote the Microsoft Program Manager’s book of proverbs: 90% of making things happen is sending email.

So James dropped by one Thursday for demos, lunch, and schmoozing with our engineers.

The first order of business was to demo our software to James. We got a bunch of the senior engineers together and showed him an abbreviated demo of both Palantir Government and Palantir Finance. We focused less on the problem-space aspects of the software and more on how we’re using Java to build the application. We went over how both of our apps are completely written in Java and that our GUIs are built with custom Swing components.

The most memorable part of the conversation went something like this:

LEAD DEV: So… what do you think of our applications?

GOSLING: It makes me want to weep.

LEAD DEV: Uh… ?

GOSLING: Yeah, we’ve been working on this infrastructure for years to be able to build applications like this and finally someone is doing it.

The rest of the visit was spent talking about Java, its history and its future. Topics ranged from why it’s hard to get dinosaurs like cable companies and mobile carriers to use modern technology to some of the complication in building an optimizing JIT compiler.

After lunch, I walked him to the elevator to see him off. We said our goodbyes and he stepped into the elevator, which was already occupied by the mailman making his rounds. As the doors closed, I hear the mailman say to James:

“Well, I haven’t seen you around here in a while.”

Palantir started in a very small office on Sand Hill Rd. For a time (while between offices), all work was done in a founder’s bedroom. In late 2005, the Palantir moved to medium-sized, nondescript suite on Page Mill Rd., just across the street from Hewlett-Packard in south Palo Alto. I joined a few months later, as the fifteenth employee. Since that time, we’ve sextupled in size and we’re currently pushing a triple-digit headcount!

We almost ran out of space last year, but managed to find a fantastic office space in downtown Palo Alto, at the intersection of Hamilton and Alma. At the time we acquired the space, we were told that it had been the home of Digital Equipment Corporation’s Western Research Lab. I mentioned this to an acquaintance of mine who mentioned, offhandedly, that this same set of offices was where Java was developed.

A few weeks ago, I got curious about this potentially dangling reference and decided to see if I could transmute it from rumor into fact. It sure would be neat if the fundamental technology that underlies our software was first developed in the very offices in which we work.

Some quick research confirmed the rumors and they’re all true! After the jump, the history of 100 Hamilton Ave.
Read the rest of this entry »

One of the most misunderstood aspects of multithreaded Swing applications is care and feeding of SwingUtilities.invokeAndWait. Hans Muller and Kathy Walrath authored a nice article that includes an overview of when to use invokeLater or its slightly more risky sibling, invokeAndWait.

We often use worker threads to do some long-running process, so often run into two issues using SwingUtilities invokeLater/invokeAndWait, and have developed wrapper code to deal with it. One issue is executing code from both worker threads and the Event Dispatch Thread (EDT). invokeLater and invokeAndWait both throw exceptions if executed on EDT. Second, invokeAndWait isn’t guaranteed — interruption on the calling thread will resume execution before the job is finished. The remainder of this post shows the code we used to solve these issues.

Read the rest of this entry »

I often ask candidates if they’re familiar with what we do at Palantir. Most people think they are. “Oh, you’re that data viz. company,” or, worse, “You guys do data mining, right?” At least they’ve heard of us and at least they’re on the right track, but I cringe anyway. We aren’t just a “data visualization” company and we don’t do “data mining.” It’s almost impossible to convey the scope and complexity of what we do in a few short minutes—or to do so without taking the conversation to an eye-glazing level of abstraction.

The following is my attempt at describing what we do at a high level without oversimplifying. I hope that after reading this a candidate will ‘get’ what we’re about, or at least understand enough not to apply tiny labels to our expansive vision.

Read the rest of this entry »

Memory leaks are no fun — to find them, you usually have to do a work flow while using a profiler like YourKit, force garbage collection, capture a memory snapshot, and then manually go through the snapshot looking for objects that you expected to be gone but are still there.Even after you’ve finally fixed the memory leak, how can you make sure that the issue doesn’t resurface later? Every good developer knows that, if you fix a bug, you should probably also have a JUnit test for that bug so that it doesn’t happen again. But how can you test for memory leaks programmatically?Find out after the jump… Read the rest of this entry »