Query offload from relational data warehouses to cheaper distributed storage seems to be all the rage these days. In this blog post we examine what works and what the limitations are. What is Query Offload? Let’s first define what we mean by query offload. With query offload we either move data from more expensive storage to cheaper storage or from ...
Flexter, Informatica, and Redshift work Hand in Hand to convert ESMA XML
In this walk-through, we combine two powerful software platforms to present a highly efficient and user-friendly method to perform ETL of complex XML files. This implementation uses Flexter, which is a powerful tool for converting complex XML files to a database or text and Informatica for ETL. We will convert ESMA XML files (these files contain the reporting specifications and ...
Redshift's Window Functions Advanced use case - Sessionization
The approach presented in the previous post has some advantages, for example, the logic can be easily inverted to extract the information about the periods without the free credit. It has, however, a hard to miss downside - it does not allow us to aggregate all the top-ups, ie. what if we want to include all those €5 or €10 ...
Are Data Lakes Fake News?
The problem with the data lake Are data lakes fake news? The quick answer is yes and in this post I will show you why. The biggest problem I have with data lakes is that the term has been overloaded by vendors and analysts with meanings. Sometimes it seems that anything that does not fit into the traditional data warehouse ...
