Big Data News: HUG Ireland’s 1st 2016 Big Data Event, Airbnb’s Predictive Model using NPS and Hive Optimization

Uli Bethke Apache, Big Data, Business Intelligence, Data Warehouse, ETL, Hadoop, Hive, HUG Ireland, Technology

Hadoop User Group (HUG) Ireland packed the house with a great evening on Apache Mesos/Myriad and an overview of Airbnb’s Predictive Model

IMG_2377

After a restful holiday season, the new year kicked off in style for Hadoop User Group (HUG) Ireland with its opening 2016 event at Synchronoss on January 11th. We heard from Mary Mangru, President of DAMA Ireland about the (Multinational) Data Management organisation’s new Irish chapter. Her brief overview of this new industry body was followed by a fascinating talk by Michael Hausenblas of Mesosphere on Apache Mesos, DCOS and Apache Myriad. It’s truly a great development making hadoop a more integrated part of your data management platform by being able to run multiple instances by user or group on the the same nodes (multi tenancy) and run side by side operations apps with analytics apps. So what can Myriad do? From what I can see, it brings Hadoop Yarn and Mesos together creating synergies and is relatively easy to deploy. Bryan Quinn of Synchronoss wrapped up the evening with some very practical insights and advice on running ETL jobs on and off Hadoop. The medium to low level computing insights gave a very clear production view of how the “real job” gets done! Our thanks to all presenters for a great event and we look forward to HUG Ireland’s next event on Monday, February 8th at Bank of Ireland, Grand Canal Square.

Airbnb shared their predictive model that predicts rebooking based on user rating and behaviour. They adopted the NPS (Net Promoter Score) which is a loyalty rating model of positive (9-10), passive (7-8) and negative (0-6) evaluation weighted to see who will promote and who will detract from their experience after using the Airbnb product. Their model goes onto use correlated control points such as length of stay, room type, trip length and so on to increase accuracy and maintain quality in their outputs. Likely to Recommend (LTR) is also an important aspect of the predictive outputs as it has an aggregate accuracy rate of 63.58% for Airbnb, which is only marginally improved by other review categories being factored into the model for final output. Fascinating stuff that demonstrates the power of big data in business!

Hive has become a big data analytics mainstay underpinning some of the core value proposition for distributed computing to date. Keeping up with updates is a challenge as is getting the best out of the existing technology! The latter in relation to Hive is what Jörn Franke talks about in his article Demonstrating the Potential of Existing Technologies. He explores a few approaches to querying hive, that bear the performance concerns of scalability in mind. HIs advice on Bloom Filters and their use in queries is particularly interesting given the use cases for it at scale and how effective bloom filters will be on certain memory chunk sizes at scale assuming columns are sorted, etc. It’s a must read for any big data professional using Hive to query and analyse big data sets.

As another week comes to a close, our community has seen great achievements this week with more to come that keep our interest peaked as we travel the big data path of progression into the future. On that note, have a great weekend and a great week to come…

About Sonra

We are a Big Data company based in Ireland. We are experts in data lake implementations, clickstream analytics, real time analytics, and data warehousing on Hadoop and Spark. We also run the Hadoop User Group (HUG) Ireland. We can help with your Big Data implementation. You can get in touch today, we would love to hear from you!