Query Offload with Redshift Spectrum. Use Cases and Limitations

Jiří Mauritz Data Warehouse, Redshift

Query offload from relational data warehouses to cheaper distributed storage seems to be all the rage these days. In this blog post we examine what works and what the limitations are. What is Query Offload? Let’s first define what we mean by query offload. With query offload we either move data from more expensive storage to cheaper storage or from ...

Flexter, Informatica, and Redshift work Hand in Hand to convert ESMA XML

Anvesh Gali ETL, Uncategorized, XML

In this walk-through, we combine two powerful software platforms to present a highly efficient and user-friendly method to perform ETL of complex XML files. This implementation uses Flexter, which is a powerful tool for converting complex XML files to a database or text and Informatica for ETL. We will convert ESMA XML files (these files contain the reporting specifications and ...

Redshift's Window Functions Advanced use case - Sessionization

Jiří Mauritz Data Warehouse, Redshift, Window Functions

The approach presented in the previous post has some advantages, for example, the logic can be easily inverted to extract the information about the periods without the free credit. It has, however, a hard to miss downside - it does not allow us to aggregate all the top-ups, ie. what if we want to include all those €5 or €10 ...