NASA’s Mars Rover

The team in control of NASA’s Mars Rover spacecraft now have Big Data driven analytical engines at their fingertips.

The same open source ElasticSearch technology used by the likes of Netflix and Goldman Sachs is now being put to use planning the actions of the Curiosity rover, which landed on the red planet in 2012. And the next mission, due to launch in 2020 with the primary mission of finding evidence of ancient life on the planet, is being built around the technology from the ground up.

NASA’s Jet Propulsion Labs, which runs the day-to-day mission planning, has rebuilt its analytics systems around ElasticSearch which now processes all of the data transmitted from the Rover during its four daily scheduled uploads.

These uploads cover tens of thousands of data points – every reading taken by Curiosity’s onboard sensors including temperature on the Martian surface, atmospheric composition, and precise data on the rover’s equipment, tools and actions.

All of Curiosity’s operations are planned a day in advance based on data received the previous day. This move to a real time analytics vastly speeds up the time in which decisions can be taken by mission control.

ElasticSearch – which recently achieved its 50 millionth download – means that patterns and anomalies in the datasets can be spotted far more quickly. Correlations which could provide mission-critical insights are more likely to become apparent, leading to a greater rate of scientific discovery and less danger or malfunction or failure.

NASA JPL data scientist Dan Isla told me “This has been very transformational from an operational point of view. We can get very quick insights over multiple days’ worth of data, or even the entire mission, in just a few seconds, without delays. We can actually ask questions and get answers quicker than we can think up new questions.”

One application is anomaly resolutions. When it appears that there is a problem with the spacecraft, precise details of its operations can immediately be analyzed to find out when the last time this situation occurred, and what other elements were in play at the time.

The system is also used for the vital function of managing power generated by the rover’s radioisotope thermoelectric and solar power generators. The trickle charge system only makes around 100 watts of power available to the craft and its instruments at any one time consistently.

This means that power management must be carefully planned in advance. As Isla explains, “using ElasticSearch we can budget this power much more effectively, we can look at the data and see ‘ok, this is what we used yesterday, and this is the state of the batteries, and this is our plan coming up…’ we can build a power model very quickly. “With ElasticSearch everyone can do analytics, ask questions at the same time and get the same results, we’re all working together now. Everyone’s very agile now.”

This isn’t NASA’s only project to use ElasticSearch. It was incorporated into the Mars Rover program after being successfully tested in the Soil Moisture Active Packing mission, launched last year. SMAP uses high powered radar and radiometer to analyze and capture high resolution data on soil moisture across the globe. A full moisture map of the Earth’s surface is generated every three days – data creation an order of magnitude greater than that of the Mars Rover. While the rover has so far generated around 1 billion data points, the SMAP project is already pushing 20 billion.

Perhaps the most exciting aspect of Mars exploration today is the possibility of discovering whether or not the planet was ever home to life. It is thought that ElasticSearch will greatly speed up the answering of this question.

Isla tells me “Curiosity was designed to find out whether Mars was ever a habitable environment. During the course of the mission we’ve been able to confirm that yes, it was. The next mission – in 2020 – we’re taking life detection instruments and we need more capabilities, more instruments and we need to be able to do it faster.

“We can’t be spending hours analyzing the data, so we need to take the ElasticSearch approach that we implemented fairly early into Curiosity and do that from day one. We’re definitely baselining this tech as our operational data system.”

For Big Data certification – http://www.vskills.in/certification/Big-data