Typically, data visualization is the last mile in a data pipeline, as it presents insights in a way that is easily understood by users. When insights are fresh and relevant, humans can act upon them on time.
However, the process of implementing a visually appealing real-time dashboard is not as simple as it is thought. The challenges in data collection and processing at scale, and delivering the metrics to users on time make things difficult.
In this talk, we deconstruct a real-time analytics dashboard into several layers as data collection, metrics computation, and insights serving. Then we take a real real-world use case, an IoT dashboard, and build it from scratch while walking through each layer in detail, taking open-source technology components for the implementation.
In the second half of the talk, we discuss the challenges in the process and find ways to mitigate them.
This talk is ideal for anyone interested in the practical application of real-time data processing and visualization. Attendees will gain a comprehensive understanding of each layer, its importance, and how they interact with each other to create a seamless, real-time dashboard.