Analyze with Insight

This page describes Insight, Fundamentum's no-code data warehouse: how it turns your device streams into tables you can query in SQL, and how it keeps each client's data isolated.

What Insight does

Insight takes the data flowing through your project's topics and turns it into a star schema, ready for analysis:

• fact tables hold your events and measurements, one row per occurrence;
• dimension tables describe the things those facts refer to, such as devices, registries or your own entities.

You do not write the pipeline: you configure it. Apache Flink pipelines read your topics, join each fact with its dimensions, rename and convert fields, and write the result into Apache Doris, an analytical database built for fast SQL queries on large volumes.

How it works

1. Create an instance. An Insight instance groups the resources of one data warehouse (its pipeline, its database and its SQL users) and belongs to a project. A project can have several instances.
2. Define your tables. Declare your fact and dimension tables and their columns through the Public API.
3. Configure the pipeline. Choose the topics that feed your facts and your dimensions, then map each column of a fact table to a field of the source, a field of a dimension, or a constant. The Insight interface guides you through it.
4. Query. Create SQL users for your instance and connect with any SQL client.

Key features

• No code. Pipelines are configured through the Insight interface or the Public API, not programmed.
• Streams joined automatically. Facts are enriched with their dimensions as they arrive.
• Historized dimensions. Each dimension is kept twice: its full history, with every version of each entity, and its current value. You can analyze facts against the state of an entity at the time, or as it is now.
• Isolated database per instance. Each instance has its own database, with dedicated resources and its own users, so your data and your query load stay separate from everyone else's.
• Any SQL client. Insight speaks the MySQL protocol, so BI tools, Grafana, notebooks and command-line clients connect to it directly.
• Access control. Who can manage instances and pipelines is governed by Fundamentum roles, like the rest of the platform.

Typical uses

• Dashboards of fleet activity in your BI tool or Grafana.
• Long-term analysis of device states, beyond the history kept for each device.
• Reports that combine device data with your own business entities, sent to Fundamentum as dimension topics.

Go further

• Query your data with Insight
• Topics: the fact and dimension topics that feed Insight.
• Collect, route and transform data
• Public API