We bring data to life: Palantir & the VAST Challenge
Monday, July 21st, 2008
Palantir has entered the 2008 VAST Challenge. We present an in-depth look at one of our challenge solutions as the first public example of the Palantir platform in action.
Two years ago, the IEEE began an annual conference called VAST (Visual Analytics in Science and Technology). The VAST symposium focuses on the fundamental research contributions and real-world application of visual analytics. As a part of the conference, the VAST Challenge allows teams to compete on delivering analytic solutions against a synthetic real-world dataset.
Each year, the organizers build a very vast (pun intended) dataset from scratch. The data is entirely fictional but mirrors real-world use cases and scenarios. This year’s dataset is about a new religious movement that started on an imaginary Caribbean island (cleverly titled Isla del Sueño, or “Island of Dreams”) situated between Florida and Cuba. There are four subsets of the synthetic data: the Wikipedia page for the movement and its associated edit and discussion pages; landing and Coast Guard interdiction records for boats leaving Isla del Sueño for Florida/Mexico; cell phone records from the island; and RFID tracking data from people in a building that was attacked with an IED.
The types of questions asked in the problem sets are qualitative questions that require answers backed by data. These are the sorts of questions that don’t yield answers using a machine-learning/data-mining approach nor can an unassisted human get these answers by simple inspection of the data. They require some sort of human-computer symbiosis to solve.
To solve the VAST problems, we assembled an ad-hoc team of analysts — composed of a mix of engineers, in-house professional analysts, and one senior executive — and asked them to use the Palantir Government software to extract insights from the data.
The results speak for themselves: the complete set of Palantir’s VAST solutions are available here.
Read on for an in-depth look at how we deconstructed and solved one of the problems.
