👋
NOTE
This is a collection of personal notes while taking Nina Miolane's famous graduate course at UCSB Electrical and Computer Engineering, Geometric Machine Learning. I highly recommend checking out the course GitHub repo for more information and course material.

## Manifolds

A manifold is more than just a cool word, it's a mathematical object that looks like a curved space. More specifically, a manifold is a topological space that locally looks like Euclidean space. And when we say that a manifold "looks like Euclidean space locally," we mean that if we zoom in enough at any point on the manifold, the space looks like a flat or Euclidean space.

For example, consider the surface of a sphere. If we zoom in very closely to a small patch on the sphere, say, Santa Barbara, it looks like a flat or Euclidean plane. Similarly, if we zoom in very closely to a patch in Australia, it also looks like a Euclidean plane. However, when we look at the sphere as a whole, it is clear that it is curved and not a flat space. Does that make sense?

Hence, a manifold is a space that can be covered by small patches, or coordinate charts, such that each patch looks like a flat or Euclidean space. These patches are glued together using mathematical functions called transition maps, which relate the coordinates of neighboring patches. This allows us to study the manifold as a whole, even though it may be curved or have a complex shape.

Each point on the manifold has a neighborhood that is homeomorphic to an open set in Euclidean space. Two objects are homeomorphic if they can be deformed into each other by a continuous, invertible mapping.

Let's keep defining a manifold, but more formally. A manifold $$M$$ can be seen as a smooth surface with any dimension, where the dimension indicates the number of degrees of freedom that a data point has on this surface.

Theorem. A nonempty subset $$M \subseteq \mathbf{R}^N$$ is a manifold if and only if any of the following conditions hold:

1. Local Parameterization. For every $$p \in M$$, there are two open subsets $$V \subseteq \mathbf{R}^d$$ and $$U \subseteq \mathbf{R}^N$$ with $$p \in U$$ and $$0 \in V$$, and a smooth function $$f : V \to \mathbf{R}^N$$ such that $$f(0)=p$$, $$f$$ is a homeomorphism∗ between $$V$$  and $$U \cap M$$ , and $$f$$  is an immersion at 0.
2. Local Implicit Function. For every $$p \in M$$, there exist an open set $$U \in \mathbf{R}^N$$ and a smooth map $$f : U \to \mathbf{R}^{N-d}$$ that is a submersion at $$p$$, such that $$U \cap M = f^{-1}(\{0\})$$.
3. Local Graph. For every $$x \in M$$, there exist an open neighborhood $$U \subseteq \mathbf{R}^N$$ of $$x$$ , a neighborhood $$V \subseteq \mathbf{R}^d$$ of 0 and a smooth map $$f : V \to \R^{N-d}$$ such that $$U \cap M = \text{graph}(f)$$

## Hypersphere Example

Let's prove that a hypersphere is a manifold using the theorem above. This is the simplest manifold, and will be used in several examples, so make sure you understand this one!

A $$d$$-dimensional hypersphere generalizes a 1-dimensional circle and a 2-dimensional sphere to $$d$$ dimensions. A $$d$$-dimensional hypersphere is the set of all points in $$\R^{d+1}$$  that are a given distance, called the radius $$a$$, from 0.

$S = \{ x \in \R^{d+1} \mid \Vert x \Vert_2^2 = a \}$

### Proving that a hypersphere is a manifold

We know from the definition that points on a hypersphere $$S$$ verify $$\Vert x \Vert_2^2 = a^2$$.

We define the function $$f(x) = \Vert x \Vert^2 - a^2$$ that will be zero for all points that lie on $$S$$:

$x \in S \Longleftrightarrow f(x) = 0$

which tells us that

$x \in S \Longleftrightarrow x \in f^{-1}(\{0\})$

Hence, this matches the definition of a manifold $$S = f^{-1}(\{0 \})$$ where $$S$$ is the set of points $$x$$ that satisfy the condition $$\Vert x \Vert^2 = a^2$$. Therefore, $$S$$ is a manifold. $$\blacksquare$$

### Hyphersphere in geomstats

import numpy as np
from geomstats.geometry.hypersphere import Hypersphere

sphere = Hypersphere(dim=2);
print(f"The sphere has dimension {sphere.dim}")
point = np.array([1, 6, 0]);
print(f"Point is on the sphere: {sphere.belongs(point)}")

### Tangent Vectors and Tangent Spaces

#### Tangent Vector

Let $$d \leq N \in \N$$, $$M$$ be an embedded manifold in $$\R^N$$ of dimension $$d$$ and $$p \in M$$. A vector $$v \in \R^N$$ is tangent to $$M$$ at $$p$$ if there exists an open interval $$I$$ centered around 0, and a curve $$\gamma : I \to M$$ such that

$\gamma (0) = p \text{ and } \dot \gamma (0) = v$

We denote $$T_p M$$ as the set of tangent vectors at $$p$$.

import numpy as np
import geomstats.visualization as viz

fig = plt.figure(figsize=(10, 10))

point = np.array([-0.65726771, -0.02678122, 0.7531812])
point = point / np.linalg.norm(point)

vector = np.array([1, 0, 0.8])

ax = viz.plot(points=point, ax=ax, space="S2",
s=200, alpha=0.8, label="Point")
arrow = viz.Arrow3D(point, vector=vector)
arrow.draw(ax, color="black")
ax.legend();

#### Tangent Space

The tangent space at a certain point $$p$$ on a manifold $$M$$ is written $$T_p M$$ and is comprised of all of the possible tangent vectors that exist at that point.

Thus, the tangent space of a 1-dimensional manifold (curve) is also one dimensional, and the tangent space of a 2-dimensional manifold (a 2-dimensional surface) is also 2-dimensional.

Similarly, for every n-dimensional manifold, there exists an n-dimensional tangent space at each point on the manifold, and the tangent space is comprised of all possible tangent vectors on that manifold.