‘What is a Manifold?’, Redux: Some Non-Examples

submited by

Style Pass

2021-06-22 06:00:07

Manifolds are still the ‘bread and butter’ of modern machine learning research. I discussed them in an earlier post, aptly titled ‘What is a manifold?’. Here, I want to describe some non-examples, i.e. spaces that are not manifolds. The main motivation for this post is to dispel some false notions about manifolds—the term has been so common in certain circles that it is almost synonymous with ‘data.’ However, as I tried to explain in my previous post, a manifold is a space that is exceptionally well-behaved and, most importantly, homogeneous. To reiterate my favourite definition:1 a $d$-dimensional manifold $\mathcal{M}$ is a space that locally looks like a $d$-dimensional Euclidean space, i.e. like some $\mathbb{R}^d$. The important thing is that $d$ must not vary; every point in $\mathcal{M}$ needs to satisfy this definition for the same value of $d$.

The ‘manifold hypothesis’ refers to the assumption that a given data set $\mathbf{X}$ is actually a discrete sample of some manifold $\mathcal{M}$. Often, the hypothesis is extended by stating that $\mathcal{M}$ must have a significantly lower dimension than the data $\mathbf{X}$. If $\mathbf{X}$ is sampled from some $\mathbb{R}^D$, we would hope that $\mathcal{M}$ is $d$-dimensional, with $d \ll D$.