Window Function ROWS and RANGE on Redshift and BigQuery

Jiří Mauritz Data Warehouse, Redshift, Window Functions

Frames in window functions allow us to operate on subsets of the partitions by breaking the partition into even smaller sequences of rows. SQL provides syntax to express very flexible definitions of a frame. We described the syntax in the first post on Window functions and demonstrated some basic use cases in the post on Data Exploration with Window Functions ...

Spark and Hadoop in Risk Line of Business at Bank of America

Anvesh Gali Big Data, Business Intelligence, Data Science, Data Warehouse

Do you want to gain knowledge about Big Data? Do you want to dig into the field of Risk Line of Business at Bank of America? Come join us to explore these questions. Presentation 1 Andrea Fagan will talk about the history of big data in the Risk Line of Business at Bank of America. What works and what doesn't. ...

Query Offload with Redshift Spectrum. Use Cases and Limitations

Jiří Mauritz Data Warehouse, Redshift

Update: An earlier version of this article claimed that Spectrum runs Presto under the hood. This is incorrect. Spectrum uses its own query layer. Query offload from relational data warehouses to cheaper distributed storage seems to be all the rage these days. In this blog post we examine what works and what the limitations are. What is Query Offload? Let’s ...

Flexter, Informatica, and Redshift work Hand in Hand to convert ESMA XML

Anvesh Gali ETL, Uncategorized, XML

In this walk-through, we combine two powerful software platforms to present a highly efficient and user-friendly method to perform ETL of complex XML files. This implementation uses Flexter, which is a powerful tool for converting complex XML files to a database or text and Informatica for ETL. We will convert ESMA XML files (these files contain the reporting specifications and ...