Big Data News: Greenplum and Eagle… The latest on Big Data and Open Source

Uli Bethke Big Data, Data Warehouse, Hadoop, HUG Ireland, Technology

Where the rubber meets the road... Greenplum goes open source and Ebay Eagle is now an Apache incubator project

imgGoogleServerRoom

As another week draws to a close, there is a tingling sensation of excitement in the air. It is reasonable to think that fancy dress parties of Halloween (nb: In Ireland, Halloween is a cultural event that celebrates the arrival of the ‘ghosts of winter’), but in our big data community, we are (slightly) more excited about the open source publications of Greenplum by Pivotal and pending; Apache Eagle originating from Ebay.

Ebay having numerous use cases for Hadoop overcame many challenges including monitoring and security issues in their distributed environment to develop and now open source Eagle through the Apache Foundation. From October 26th, 2015 Apache Eagle is an Apache incubator project and we have good reason to watch that space based on Ebay’s original offering. The features of Eagle are varied and include:

  • Anomalous data access based on usage patterns/user behaviour
  • Intrusion discovery and security breaches
  • Data Loss Discovery and prevention
  • Policy distribution, detection and alerting

Eagle is also highly scalable in a distributed environment, real time (to sub second) returns and extensible with existing monitoring systems and networks. As pattern recognition underpins the tool, the clever use of its streaming API and a machine learning module deserves a mention. It’s a promising use of algorithms to identify issues by anomaly detection within established usage patterns. A fine product from eBay that the open source community will now be able to work on an improve over time.

Pivotal have open sourced Greenplum which is a massive boost to open source movement around massively parallel data warehouses. It will be in essence a move to a skinny open source model that is Hadoop friendly. It is lauded with development and integration of the ‘best in class’ optimizer called “Orca”. With the acquisition of Quickstep, Pivotal have also integrated a best in class bare metal columnar computing feature into the new “skinny model”. If you would like to know more, why not join Hadoop User Group Ireland today as Greenplum will feature in a morning briefing the community group will host on November 28th in Dublin City Centre.

So, exciting times for our community and with the onset of coming week, we look forward to bringing you more discoveries of what’s new in our “Big Data” world! Have a great weekend all!

About Sonra

We are a Big Data company based in Ireland. We are experts in data lake implementations, clickstream analytics, real time analytics, and data warehousing on Hadoop and Spark. We also run the Hadoop User Group (HUG) Ireland. We can help with your Big Data implementation. You can get in touch today, we would love to hear from you!