Leveraging Open-Source Tools for Building a Quality Data Warehouse
In recent years the modern data stack has seen a great increase in the number of tools available on the market, making the landscape overwhelming and difficult to navigate. In this presentation, we will give an overview of how Kognic built a data warehouse with quality as its core value. We will present our architecture and explain how we leverage some popular open-source tools in our setup, highlighting their main features and shortcomings. Particular emphasis will be put on how data is transformed for the data mart using a test-driven approach that aims to stay true to software-development best practices.
Matteo is a Data Engineer at Kognic, where, together with the Data Enablement team he provides infrastructures and services for the company’s analytical needs. With a background in Computational Mathematics, Matteo spent numerous years working in the realm of both Data Science and Data Engineering. In 2022, he made a complete transition to Data Engineering by joining Kognic.
