Data Architecture & Advisory Services
We have listed some common data architecture challenges that our customers experience. We can also resolve these challenges for you.
Enterprise Data Architecture
We can help you with:
- Big data warehouse architecture
- Data warehousing in the cloud
- Master Data Management (MDM). Integrating MDM with your data warehouse
- Architecting a real-time data warehouse
- Architecting a data warehouse error hospital for monitoring data quality
- Data warehouse modelling
- Architecting and integrating a data science platform into your data architecture
Data lake/data reservoir architecture
Data lakes have gotten a bit of a bad press recently. We have detailed the various issues with the concept of the data lake in a recent blog post.
However, approached the right way the concept of the data lake (more appropriately named raw data reservoir) can be a valuable asset. We help you work through the various challenges of implementing a data lake architecture:
- How does the data lake strategy align with the wider data and business strategy? Do you really need a data lake?
- What is the logical data lake architecture and how does it align with the data warehouse and sandboxes for data preparation and data science.
- How should we design the ingestion process? How should we handle semi-structured data? Where should we mask PII?
What is the process to get data into the data lake?
- How do we model the data lake? What storage is best for your scenario? What storage format works best for your scenario?
- Data governance. How can we make information in the lake findable? What are the standards and naming conventions?
- Metadata and data catalog management
- Who are the data lake consumers
These are just some of the many questions we can help you with on your data lake journey.
Make sure to download our data lake checklist.
Data Warehouse Modernization
Traditional data warehouse architecture has served us well for the last few decades. During most of the nineties and noughties we have only seen evolutionary changes. Architecture, tools, and technologies were pretty static. Ok, we had some innovations, e.g. columnar storage or ELT, but by and large not that much changed during that time.
The process of digitisation, the importance of new types of data, and the explosion of data volumes first witnessed by Google and other web companies changed all of this. It has exposed various issues of the traditional world of data warehousing.
- Data warehouses are bursting at the seams
- As a result license costs are soaring
- How do we process text, audio, images, video?
- The traditional data warehouse life cycle does not provide answers fast enough
- Real-time analytics was never a part of the data warehouse architecture
- … and neither was advanced analytics
We can help you address these common challenges.
Data Warehouse Leader Mentoring
We mentor business intelligence or data warehouse managers and architects. If you need external advice with a question or are stuck with a problem please feel free to reach out to us.
We can mentor you on an ad-hoc or permanent basis. Hourly rates apply.
We help you identify the best vendor for your particular requirement and use case. We cover data warehouse, Hadoop, data preparation, ETL, cloud data warehousing, and data science platforms.
- Understand requirements (interviews, questionnaires, review documentation and systems)
- Design vendor feature matrix
- Design RFP
- Review RFP answers
- Vendor interviews
- Design PoC
- Score PoC outcome
Data advisory services for start-ups
Are you a start-up with a data product and need advice on data architecture, tools, and tech?
Get 30 minutes of free consultation with our CEO Uli Bethke or our CTO Maciek Kocon.