Custom dashboard for Great Expectations
Nowadays, Great Expectations is a very viable option for most of the organizations to introduce data quality solution for their data platform. Simplicity, being easily customizable and relying on the toolbox that is mostly known to modern data engineers, are the common factors which help with lowering the adaption barrier.
However, there is one part of Great Expectation, which I hoped to see significant improvement some day. The static dashboards! They are:
In one of our recent projects at DataChef, we found ourselves, faced with these limitations, and thought it's time to build on top of the flexibility which Great Expectation already provides. The idea was simple. When running the suits, we wanted to generate metrics based on the findings, so we can use any modern dashboarding tool, to visualize the reports. For us, the following were the main pros of this approach:
Relying on existing monitoring dashboards, and providing a holistic view over the whole life cycle of the data products (not just data quality).
