This is a collection of personal notes while taking Nina Miolane's famous graduate course at UCSB Electrical and Computer Engineering, Geometric Machine Learning. I highly recommend checking out the course GitHub repo for more information and course material.
Manifolds
A manifold is more than just a cool word, it's a mathematical object that looks like a curved space. More specifically, a manifold is a topological space that locally looks like Euclidean space. And when we say that a manifold "looks like Euclidean space locally," we mean that if we zoom in enough at any point on the manifold, the space looks like a flat or Euclidean space.
For example, consider the surface of a sphere. If we zoom in very closely to a small patch on the sphere, say, Santa Barbara, it looks like a flat or Euclidean plane. Similarly, if we zoom in very closely to a patch in Australia, it also looks like a Euclidean plane. However, when we look at the sphere as a whole, it is clear that it is curved and not a flat space. Does that make sense?
Hence, a manifold is a space that can be covered by small patches, or coordinate charts, such that each patch looks like a flat or Euclidean space. These patches are glued together using mathematical functions called transition maps, which relate the coordinates of neighboring patches. This allows us to study the manifold as a whole, even though it may be curved or have a complex shape.
Each point on the manifold has a neighborhood that is homeomorphic to an open set in Euclidean space. Two objects are homeomorphic if they can be deformed into each other by a continuous, invertible mapping.
Let's keep defining a manifold, but more formally. A manifold \( M \) can be seen as a smooth surface with any dimension, where the dimension indicates the number of degrees of freedom that a data point has on this surface.
Theorem. A nonempty subset \( M \subseteq \mathbf{R}^N \) is a manifold if and only if any of the following conditions hold:
- Local Parameterization. For every \( p \in M \), there are two open subsets \( V \subseteq \mathbf{R}^d \) and \( U \subseteq \mathbf{R}^N \) with \( p \in U \) and \( 0 \in V \), and a smooth function \( f : V \to  \mathbf{R}^N \) such that \( f(0)=p \), \( f \) is a homeomorphism∗ between \( V \)  and \(  U \cap M \) , and \( f \)  is an immersion at 0.
- Local Implicit Function. For every \( p \in M \), there exist an open set \( U \in \mathbf{R}^N \) and a smooth map \( f : U \to \mathbf{R}^{N-d} \) that is a submersion at \( p \), such that \( U \cap M = f^{-1}(\{0\}) \).
- Local Graph. For every \( x \in M \), there exist an open neighborhood \( U \subseteq \mathbf{R}^N \) of \( x \) , a neighborhood \( V \subseteq \mathbf{R}^d \) of 0 and a smooth map \( f : V \to \R^{N-d} \) such that \( U \cap M = \text{graph}(f) \)
Hypersphere Example
Let's prove that a hypersphere is a manifold using the theorem above. This is the simplest manifold, and will be used in several examples, so make sure you understand this one!
A \(d \)-dimensional hypersphere generalizes a 1-dimensional circle and a 2-dimensional sphere to \(d \) dimensions. A \(d \)-dimensional hypersphere is the set of all points in \( \R^{d+1} \) Â that are a given distance, called the radius \(a \), from 0.
\[ S = \{ x \in \R^{d+1} \mid \Vert x \Vert_2^2 = a \} \]
Proving that a hypersphere is a manifold
We know from the definition that points on a hypersphere \( S \) verify \( \Vert x \Vert_2^2 = a^2 \).
We define the function \( f(x) = \Vert x \Vert^2 - a^2 \) that will be zero for all points that lie on \( S \):
\[ x \in S \Longleftrightarrow f(x) = 0 \]
which tells us that
\[ x \in S \Longleftrightarrow  x \in f^{-1}(\{0\})  \]
Hence, this matches the definition of a manifold \( S = f^{-1}(\{0 \}) \) where \( S \) is the set of points \(x \) that satisfy the condition \( \Vert x \Vert^2 = a^2 \). Therefore, \( S \) is a manifold. \( \blacksquare \)
Hyphersphere in geomstats
import numpy as np
from geomstats.geometry.hypersphere import Hypersphere
sphere = Hypersphere(dim=2);
print(f"The sphere has dimension {sphere.dim}")
point = np.array([1, 6, 0]);
print(f"Point is on the sphere: {sphere.belongs(point)}")
Tangent Vectors and Tangent Spaces
Tangent Vector
Let \( d \leq N \in \N \), \( M \) be an embedded manifold in \( \R^N \) of dimension \( d \) and \(p \in M \). A vector \( v \in \R^N \) is tangent to \( M \) at \( p \) if there exists an open interval \( I \) centered around 0, and a curve \( \gamma : I \to M \) such that
\[ \gamma (0) = p \text{ and } Â \dot \gamma (0) = v \]
We denote \( T_p M \) as the set of tangent vectors at \( p \).
import numpy as np
import geomstats.visualization as viz
fig = plt.figure(figsize=(10, 10))
ax = fig.add_subplot(111, projection="3d")
point = np.array([-0.65726771, -0.02678122, 0.7531812])
point = point / np.linalg.norm(point)
vector = np.array([1, 0, 0.8])
ax = viz.plot(points=point, ax=ax, space="S2",
s=200, alpha=0.8, label="Point")
arrow = viz.Arrow3D(point, vector=vector)
arrow.draw(ax, color="black")
ax.legend();
Tangent Space
The tangent space at a certain point \(p\) on a manifold \(M\) is written \(T_p M \) and is comprised of all of the possible tangent vectors that exist at that point.
Thus, the tangent space of a 1-dimensional manifold (curve) is also one dimensional, and the tangent space of a 2-dimensional manifold (a 2-dimensional surface) is also 2-dimensional.
Similarly, for every n-dimensional manifold, there exists an n-dimensional tangent space at each point on the manifold, and the tangent space is comprised of all possible tangent vectors on that manifold.