Visual Debugging Now! - Nick Arner
Over the last seven or so years of working in robotics and computer vision, the bitter lesson I’ve learned is that the bug is almost always in the part of the algorithm you didn’t visualize.
Many domains, including robotics, foundationally rely on highly performant numerical software. But since humans are adapted to understand space visually, we don’t have good intuitions for numerical data. Things that are obvious to human vision like “does this have a realistic pose in space” are very unclear in numerical representations - can you picture a quaternion intuitively from its vector representation? Can you see from a Hessian matrix whether your optimization is converging?
Visualizing this type of data is the obvious thing to do to make it understandable by humans, but this is very difficult for a variety of reasons.
In Python there are readily available viz modules, and people use them. They suck because they’re mostly not interactive and are effectively stuck in the 70s. There are JavaScript things you can use to make interactive but you have to understand JavaScript as well as Python (on top of understanding the work you’re actually trying to get done) to try to use them.
