Successfully Transitioning your Team from Data Warehousing to Big Data

Uli Bethke Big Data, Data Warehouse

You are planning to complement your traditional data warehouse architecture with big data technologies. Now what? Should you upskill your existing data warehouse team? Or do Big Data technologies require a completely different set of skills? What do we mean by big data technologies anyway? For the purpose of this article, I define big data as any distributed technology that ...

Comparing Window Function Features by Database Vendors

Jiří Mauritz Data Warehouse, Redshift, SQL for Analysis, Window Functions

We will round off the series on window functions with comparison of what database vendors offer. There are various mutations of window functions and every vendor supports a different subset or feature. Some also add extra window functions or features beyond standard ANSI SQL. One of the most powerful features is user-defined aggregate functions (UDAF), which some databases allow using ...

Window Function ROWS and RANGE on Redshift and BigQuery

Jiří Mauritz Data Warehouse, Redshift, Window Functions

Frames in window functions allow us to operate on subsets of the partitions by breaking the partition into even smaller sequences of rows. SQL provides syntax to express very flexible definitions of a frame. We described the syntax in the first post on Window functions and demonstrated some basic use cases in the post on Data Exploration with Window Functions ...

Spark and Hadoop in Risk Line of Business at Bank of America

Anvesh Gali Big Data, Business Intelligence, Data Science, Data Warehouse

Do you want to gain knowledge about Big Data? Do you want to dig into the field of Risk Line of Business at Bank of America? Come join us to explore these questions. Presentation 1 Andrea Fagan will talk about the history of big data in the Risk Line of Business at Bank of America. What works and what doesn't. ...

Query Offload with Redshift Spectrum. Use Cases and Limitations

Jiří Mauritz Data Warehouse, Redshift

Update: An earlier version of this article claimed that Spectrum runs Presto under the hood. This is incorrect. Spectrum uses its own query layer. Query offload from relational data warehouses to cheaper distributed storage seems to be all the rage these days. In this blog post we examine what works and what the limitations are. What is Query Offload? Let’s ...