Falcon launch under Apache Hadoop

Falcon launch under Apache Hadoop from Hortonworks and inMobi

Hortonworks and inMobi ( performance based mobile ad network) have developed and launched product named Falcon, which is a complete data lifecycle management framework for Apache Hadoop.

Falcon enables users to configure, manage and orchestrate data motion, disaster recovery and data retention workflows in support of business continuity and data governance use cases.

Falcon is a data processing and management solution for Hadoop designed for data motion, coordination of data pipelines, lifecycle management, and data discovery. Falcon enables end consumers to quickly onboard their data and its associated processing and management tasks on Hadoop clusters.

The framework’s was developed at inMobi as it’s need were not addressed by Hadoop, so the engineers there built one for their own use. As is frequently done in Open Source communities, inMobi engineers shared what they had built with the members of the Apache Hadoop community and a team from Hortonworks signed on to make Falcon more broadly useful so that it could be submitted for acceptance into the Apache Incubator.

Falcon will enable easy data management via declarative mechanism for Hadoop. Users of Falcon platform simply define infrastructure endpoints, data sets and processing rules declaratively. These declarative configurations are expressed in such a way that the dependencies between these configured entities are explicitly described. This information about inter-dependencies between various entities allows Falcon to orchestrate and manage various data management functions.

The key use cases that Falcon addresses are:

• Data Motion
• Process orchestration and scheduling
• Policy-based Lifecycle Management
• Data Discovery
• Operability/Usability

With these features it is possible for users to onboard their data sets with a comprehensive and holistic understanding of how, when and where their data is managed across its lifecycle. Complex functions such as retrying failures, identifying possible SLA breaches or automated handling of input data changes are now simple directives. All the administrative functions and user level functions are available via RESTful APIs. CLI is simply a wrapper over the RESTful APIs.

For more updates or resources contact us at [email protected]

For Government certification, visit www.vskills.in and register for exam on 13 July, 2013 now!!

Follow us on facebook visit www.facebook.com/vskills.org