The future of ETL and the limitations of data virtualisation and NoETL.

Uli Bethke ETL

Data has limited value if we don’t transform, integrate, model (either data modelling and building predictive models) or cleanse it. Collecting raw data for no apparent reason or business case leads to the widespread data hoarding disorder. The concept of NoETL and related ideas of data federation for logically integrating data from disparate data stores (now rebranded as data virtualisation) ...

About the author

Uli Bethke LinkedIn Profile

Uli has 18 years’ hands on experience as a consultant, architect, and manager in the data industry. He frequently speaks at conferences. Uli has architected and delivered data warehouses in Europe, North America, and South East Asia. He is a traveler between the worlds of traditional data warehousing and big data technologies.

Uli is a regular contributor to blogs and books, holds an Oracle ACE award, and chairs the the Hadoop User Group Ireland. He is also a co-founder and VP of the Irish chapter of DAMA, a non for profit global data management organization. He has co-founded the Irish Oracle Big Data User Group.

Flexter 1.2.0 our ETL tool for JSON/XML has been released. 20x faster. Supports very large XML and JSON

Uli Bethke ETL, Flexter, JSON, XML

We have release Flexter 1.2.0 this week. We have added some significant new features and improvements. Flexter now supports conversion of JSON to a database, text (CSV/TSV), and Hadoop/Spark (ORC, Parquet, Avro). We now support the conversion of very large XML files of multi-GB sizes without any pre-processing. We have made performance improvements to the core parser. Flexter has always ...

About the author

Uli Bethke LinkedIn Profile

Uli has 18 years’ hands on experience as a consultant, architect, and manager in the data industry. He frequently speaks at conferences. Uli has architected and delivered data warehouses in Europe, North America, and South East Asia. He is a traveler between the worlds of traditional data warehousing and big data technologies.

Uli is a regular contributor to blogs and books, holds an Oracle ACE award, and chairs the the Hadoop User Group Ireland. He is also a co-founder and VP of the Irish chapter of DAMA, a non for profit global data management organization. He has co-founded the Irish Oracle Big Data User Group.

JSON. To ETL or to NoETL? The big data question.

Uli Bethke ETL, JSON

NoETL. The little brother of NoSQL You have probably come across the term NoSQL. It was coined a few years back to describe a class of database systems that can scale across a large number of nodes for distributed (and sometimes global processing) of transactions (OLTP). Very early technologies were DynamoDB and Cassandra. These technologies trade in scalability for consistency ...

About the author

Uli Bethke LinkedIn Profile

Uli has 18 years’ hands on experience as a consultant, architect, and manager in the data industry. He frequently speaks at conferences. Uli has architected and delivered data warehouses in Europe, North America, and South East Asia. He is a traveler between the worlds of traditional data warehousing and big data technologies.

Uli is a regular contributor to blogs and books, holds an Oracle ACE award, and chairs the the Hadoop User Group Ireland. He is also a co-founder and VP of the Irish chapter of DAMA, a non for profit global data management organization. He has co-founded the Irish Oracle Big Data User Group.

Flexter, Informatica, and Redshift work Hand in Hand to convert ESMA XML

Anvesh Gali ETL, Uncategorized, XML

In this walk-through, we combine two powerful software platforms to present a highly efficient and user-friendly method to perform ETL of complex XML files. This implementation uses Flexter, which is a powerful tool for converting complex XML files to a database or text and Informatica for ETL. We will convert ESMA XML files (these files contain the reporting specifications and ...

Big Data News: HUG Ireland’s 1st 2016 Big Data Event, Airbnb’s Predictive Model using NPS and Hive Optimization

Uli Bethke Apache, Big Data, Business Intelligence, Data Warehouse, ETL, Hadoop, Hive, HUG Ireland, Technology

Hadoop User Group (HUG) Ireland packed the house with a great evening on Apache Mesos/Myriad and an overview of Airbnb’s Predictive Model After a restful holiday season, the new year kicked off in style for Hadoop User Group (HUG) Ireland with its opening 2016 event at Synchronoss on January 11th. We heard from Mary Mangru, President of DAMA Ireland about ...

About the author

Uli Bethke LinkedIn Profile

Uli has 18 years’ hands on experience as a consultant, architect, and manager in the data industry. He frequently speaks at conferences. Uli has architected and delivered data warehouses in Europe, North America, and South East Asia. He is a traveler between the worlds of traditional data warehousing and big data technologies.

Uli is a regular contributor to blogs and books, holds an Oracle ACE award, and chairs the the Hadoop User Group Ireland. He is also a co-founder and VP of the Irish chapter of DAMA, a non for profit global data management organization. He has co-founded the Irish Oracle Big Data User Group.