Embedding

from words to networks

Module 8: spectral, neural and hyperbolic embeddings

Sadamori Kojaku

go.skojaku.com/m08slides

What is an embedding?

An embedding is a method that places a graph in a continuous space.

Sadamori Kojaku, 2026

Why embed a network?

An embedding places every node at a point of a vector space, for three reasons.

Machine learning takes vectors; a graph is discrete data.

Graph computation becomes linear algebra, with tools made for vector spaces.

A network follows a few low-dimensional principles, so it can be compressed.

Sadamori Kojaku, 2026

Hand-made node features

Prediction models need a vector of numbers for every node, and for networks these numbers have been designed by hand.

  • degree kik_i
  • number of triangles tit_i
  • return probability (Pt)ii(\mathbf P^t)_{ii} of a random walk
  • each task needs a new choice of features

Sadamori Kojaku, 2026

Representation learning

The node vectors xi∈Rd\vb x_i \in \R^d are trained on data, not designed by hand.

Sadamori Kojaku, 2026

Outline

  1. Spectral embedding
  2. Neural embedding
  3. Hyperbolic embedding
Sadamori Kojaku, 2026

Embedding

Spectral embedding

Sadamori Kojaku, 2026

Compressing a matrix

A spectral embedding keeps an N×dN \times d matrix U\mathbf U that rebuilds the N×NN \times N adjacency matrix as closely as possible.

min⁡U∈RN×d ∥A−UU⊤∥F\min_{\mathbf U \in \R^{N \times d}} \ \| \mathbf A - \mathbf U \mathbf U^{\top} \|_F

  • PCA compresses a data matrix in the same way
  • here the matrix is A\mathbf A and the vectors are its eigenvectors

Sadamori Kojaku, 2026

Eigendecomposition

A symmetric matrix is a weighted sum of outer products of its eigenvectors.

A=∑k=1Nλk ukuk⊤\mathbf A = \sum\nolimits_{k=1}^{N} \lambda_k \, \mathbf u_k \mathbf u_k^{\top}

uk\mathbf u_k: unit eigenvector with eigenvalue λk\lambda_k; ukuk⊤\mathbf u_k \mathbf u_k^{\top} is an N×NN \times N matrix.

Zachary's karate club: A\mathbf A and its first three terms

Sadamori Kojaku, 2026

Keeping the large terms

Keeping the dd terms with the largest ∣λk∣|\lambda_k| gives the rank-dd approximation Ad\mathbf A_d.

Ad=∑k=1dλk ukuk⊤\mathbf A_d = \sum\nolimits_{k=1}^{d} \lambda_k \, \mathbf u_k \mathbf u_k^{\top}

Sadamori Kojaku, 2026

Optimality of the truncation

Ad\mathbf A_d is the closest matrix of rank at most dd to A\mathbf A in the Frobenius norm.

∥M∥F=∑i,jMij2\| \mathbf M \|_F = \sqrt{\sum_{i,j} M_{ij}^2}

Ad=arg⁡min⁡rank⁡(B)≤d∥A−B∥F\mathbf A_d = \arg\min_{\operatorname{rank}(\mathbf B) \le d} \| \mathbf A - \mathbf B \|_F

∥A−Ad∥F 2=∑k>dλk2\| \mathbf A - \mathbf A_d \|_F^{\,2} = \sum_{k > d} \lambda_k^2

  • the error is the sum of the squared eigenvalues that are dropped

Eckart, C., & Young, G. (1936). The approximation of one matrix by another of lower rank. Psychometrika, 1, 211–218.
Sadamori Kojaku, 2026

Derivation of the truncation error

Write B=∑k≤dσkqkqk⊤\mathbf B = \sum_{k \le d} \sigma_k \vb q_k \vb q_k^\top with orthonormal axes qk\vb q_k; minimize over σk\sigma_k, then over qk\vb q_k.

∥A−B∥F2=∥A∥F2−2∑k≤dσk qk⊤Aqk+∑k≤dσk2qk⊤ql=δkl∂/∂σk=0 ⇒ σk=qk⊤Aqkone weight at a time∥A−B∥F2=∥A∥F2−∑k≤d(qk⊤Aqk)2max⁡∥qk∥=1(qk⊤Aqk)2 ⇒ Aqk=λkqkLagrange multiplier∥A−Ad∥F2=∑kλk2−∑k≤dλk2=∑k>dλk2∥A∥F2=∑kλk2\begin{aligned} \| \mathbf A - \mathbf B \|_F^{2} &= \| \mathbf A \|_F^{2} - 2 \sum\nolimits_{k \le d} \sigma_k\, \vb q_k^\top \mathbf A \vb q_k + \sum\nolimits_{k \le d} \sigma_k^2 && \why{$\vb q_k^\top \vb q_l = \delta_{kl}$}\\ \partial / \partial \sigma_k = 0 \ &\Rightarrow\ \sigma_k = \vb q_k^\top \mathbf A \vb q_k && \why{one weight at a time}\\ \| \mathbf A - \mathbf B \|_F^{2} &= \| \mathbf A \|_F^{2} - \sum\nolimits_{k \le d} (\vb q_k^\top \mathbf A \vb q_k)^2 \\ \max_{\| \vb q_k \| = 1} (\vb q_k^\top \mathbf A \vb q_k)^2 \ &\Rightarrow\ \mathbf A \vb q_k = \lambda_k \vb q_k && \why{Lagrange multiplier}\\ \| \mathbf A - \mathbf A_d \|_F^{2} &= \sum\nolimits_{k} \lambda_k^2 - \sum\nolimits_{k \le d} \lambda_k^2 = \sum\nolimits_{k > d} \lambda_k^2 && \why{$\| \mathbf A \|_F^{2} = \sum_k \lambda_k^2$} \end{aligned}

  • the axes are eigenvectors of A\mathbf A, and qk⊤Aqk=λk\vb q_k^\top \mathbf A \vb q_k = \lambda_k; keeping the dd largest λk2\lambda_k^2 gives the smallest error
Sadamori Kojaku, 2026

Matrices that represent a network

The matrix to decompose is a choice, and the Laplacian and the normalized Laplacian both come from graph cuts.

name definition
adjacency matrix A\mathbf A, with Aij=1A_{ij} = 1 if ii and jj are linked
degree matrix D=diag⁡(k1,…,kN)\mathbf D = \operatorname{diag}(k_1, \dots, k_N)
Laplacian L=D−A\mathbf L = \mathbf D - \mathbf A
normalized Laplacian Lsym=I−D−1/2AD−1/2\mathbf L_{\mathrm{sym}} = \mathbf I - \mathbf D^{-1/2} \mathbf A \mathbf D^{-1/2}
  • for A\mathbf A, keep the eigenvectors with the largest eigenvalues
  • for L\mathbf L and Lsym\mathbf L_{\mathrm{sym}}, keep those with the smallest eigenvalues
Sadamori Kojaku, 2026

Cutting a network in two

A cut splits the nodes into SS and TT; Ratio Cut divides the cut size by the group sizes.

cut(S,T)=∑i∈S∑j∈TAij\mathrm{cut}(S,T) = \sum_{i \in S} \sum_{j \in T} A_{ij}

RatioCut(S,T)=cut(S,T)(1∣S∣+1∣T∣)\mathrm{RatioCut}(S,T) = \mathrm{cut}(S,T) \left( \frac{1}{|S|} + \frac{1}{|T|} \right)

S={p}S = \{p\}: 1×(1+110)=1.101 \times (1 + \tfrac{1}{10}) = 1.10

SS = left clique and pp: 1×(16+15)=0.371 \times (\tfrac{1}{6} + \tfrac{1}{5}) = 0.37

Sadamori Kojaku, 2026

Group indicator vector

A cut is encoded by a vector x\vb x that takes one value on SS and another on TT.

xi={  ∣T∣/∣S∣i∈S−∣S∣/∣T∣i∈Tx_i = \begin{cases} \ \ \sqrt{|T|/|S|} & i \in S \\ -\sqrt{|S|/|T|} & i \in T \end{cases}

x\vb x for the second cut: ∣S∣=6|S| = 6, ∣T∣=5|T| = 5

  • ∑ixi=0\sum_i x_i = 0: shifting x\vb x by a constant does not change x⊤Lx\vb x^\top \mathbf L \vb x, so the mean is fixed (and x∝1\vb x \propto \mathbf 1 is out)
  • ∑ixi2=N\sum_i x_i^2 = N: fixes the scale; without it x=0\vb x = \mathbf 0 is the trivial minimum
Sadamori Kojaku, 2026

Ratio Cut as a quadratic form

With the indicator x\vb x, the Ratio Cut is x⊤Lx/N\vb x^\top \mathbf L \vb x / N, where L=D−A\mathbf L = \mathbf D - \mathbf A.

x⊤Lx=12∑i,jAij(xi−xj)2=cut(S,T) (xS−xT)2=cut(S,T) N2∣S∣ ∣T∣=N RatioCut(S,T)\begin{aligned} \vb x^\top \mathbf L \vb x &= \tfrac{1}{2} \sum_{i,j} A_{ij} (x_i - x_j)^2\\ &= \mathrm{cut}(S,T)\,(x_S - x_T)^2\\ &= \mathrm{cut}(S,T)\,\frac{N^2}{|S|\,|T|}\\ &= N \,\mathrm{RatioCut}(S,T) \end{aligned}

  • xi−xjx_i - x_j is 00 inside a group, ±(xS−xT)\pm (x_S - x_T) between groups
  • only cut edges give a nonzero term

SS = left clique and pp; the outlined cells of A\mathbf A are the cut edge.

Sadamori Kojaku, 2026

Relaxation to real values

Letting x\vb x take any real value turns the problem into one that can be solved.

min⁡x x⊤Lxs.t.  x⊤1=0,  x⊤x=N,  xi∈{∣T∣/∣S∣, −∣S∣/∣T∣}\begin{aligned} & \min_{\vb x} \ \vb x^\top \mathbf L \vb x \\ & \text{s.t.} \ \ \vb x^\top \mathbf 1 = 0, \ \ \vb x^\top \vb x = N, \ \ \hl{x_i \in \{\sqrt{|T|/|S|},\, -\sqrt{|S|/|T|}\}} \end{aligned}

min⁡x∈RN x⊤Lxs.t.x⊤1=0,  x⊤x=N\min_{\vb x \in \R^N} \ \vb x^\top \mathbf L \vb x \quad \text{s.t.} \quad \vb x^\top \mathbf 1 = 0, \ \ \vb x^\top \vb x = N

  • the minimum can only decrease; the groups are read from the signs of x\vb x
Sadamori Kojaku, 2026

Solving the relaxed problem

The Lagrangian turns the constrained minimization into an eigenvalue problem.

L(x,λ)=x⊤Lx−λ (x⊤x−N)∂L/∂x=2Lx−2λx=0⟹ Lx=λx,x⊤Lx=λN\begin{aligned} \mathcal L(\vb x, \lambda) &= \vb x^\top \mathbf L \vb x - \lambda\,(\vb x^\top \vb x - N)\\ \partial \mathcal L / \partial \vb x &= 2 \mathbf L \vb x - 2 \lambda \vb x = \mathbf 0\\ &\Longrightarrow \ \mathbf L \vb x = \lambda \vb x, \quad \vb x^\top \mathbf L \vb x = \lambda N \end{aligned}

  • the cost is λN\lambda N, so a small λ\lambda is better
  • λ=0\lambda = 0 belongs to x∝1\vb x \propto \mathbf 1, which x⊤1=0\vb x^\top \mathbf 1 = 0 excludes
  • the solution is the eigenvector of the second smallest eigenvalue

Sadamori Kojaku, 2026

Second eigenvector of the Laplacian

On Zachary's karate club, the sign of the second eigenvector of L\mathbf L gives the two factions.

Entries of the second eigenvector, sorted; Mr. Hi's faction and the officer's faction

  • the sign matches the faction for 32 of the 34 members (outlined: 3 and 9)
Sadamori Kojaku, 2026

Normalized Cut

Normalized Cut divides the cut size by the group volumes.

volume VS=∑i∈SkiV_S = \sum_{i \in S} k_i counts edge ends, not nodes; V=VS+VT=2mV = V_S + V_T = 2m

NCut(S,T)=cut(S,T)(1VS+1VT)\mathrm{NCut}(S,T) = \mathrm{cut}(S,T) \left( \frac{1}{V_S} + \frac{1}{V_T} \right)

xi={  VT/VSi∈S−VS/VTi∈Tx_i = \begin{cases} \ \ \sqrt{V_T / V_S} & i \in S \\ -\sqrt{V_S / V_T} & i \in T \end{cases}

x\vb x for the second cut: VS=23V_S = 23, VT=21V_T = 21

  • x⊤D1=∑ikixi=0\vb x^\top \mathbf D \mathbf 1 = \sum_i k_i x_i = 0 and x⊤Dx=∑ikixi2=V\vb x^\top \mathbf D \vb x = \sum_i k_i x_i^2 = V
  • sizes are counted in edge ends, so D\mathbf D weights each node by its degree
Sadamori Kojaku, 2026

Normalized Cut as a quadratic form

The same expansion gives x⊤Lx=V NCut(S,T)\vb x^\top \mathbf L \vb x = V \, \mathrm{NCut}(S,T).

x⊤Lx=12∑i,jAij(xi−xj)2=cut(S,T) (xS−xT)2=cut(S,T) V2VS VT=V NCut(S,T)\begin{aligned} \vb x^\top \mathbf L \vb x &= \tfrac{1}{2} \sum_{i,j} A_{ij} (x_i - x_j)^2\\ &= \mathrm{cut}(S,T)\,(x_S - x_T)^2\\ &= \mathrm{cut}(S,T)\,\frac{V^2}{V_S\,V_T}\\ &= V \,\mathrm{NCut}(S,T) \end{aligned}

  • xS−xT=VT/VS+VS/VT=V/VSVTx_S - x_T = \sqrt{V_T / V_S} + \sqrt{V_S / V_T} = V / \sqrt{V_S V_T}
  • same blocks as Ratio Cut, other values

Same cut, with VS=23V_S = 23 and VT=21V_T = 21.

Sadamori Kojaku, 2026

Relaxation of Normalized Cut

The relaxation is the one used for Ratio Cut, with D\mathbf D in the constraints.

min⁡x x⊤Lxs.t.  x⊤D1=0,  x⊤Dx=V,  xi∈{VT/VS, −VS/VT}\begin{aligned} & \min_{\vb x} \ \vb x^\top \mathbf L \vb x \\ & \text{s.t.} \ \ \vb x^\top \mathbf D \mathbf 1 = 0, \ \ \vb x^\top \mathbf D \vb x = V, \ \ \hl{x_i \in \{\sqrt{V_T / V_S},\, -\sqrt{V_S / V_T}\}} \end{aligned}

min⁡x∈RN x⊤Lxs.t.x⊤D1=0,  x⊤Dx=V\min_{\vb x \in \R^N} \ \vb x^\top \mathbf L \vb x \quad \text{s.t.} \quad \vb x^\top \mathbf D \mathbf 1 = 0, \ \ \vb x^\top \mathbf D \vb x = V

  • the original minimum is V⋅NCutV \cdot \mathrm{NCut}; the groups are read from the signs of x\vb x
Sadamori Kojaku, 2026

Solving the relaxed problem for Normalized Cut

The Lagrangian gives a generalized eigenvalue problem, Lx=λDx\mathbf L \vb x = \lambda \mathbf D \vb x.

L(x,λ)=x⊤Lx−λ (x⊤Dx−V)∂L/∂x=2Lx−2λDx=0⟹ Lx=λDx,x⊤Lx=λV\begin{aligned} \mathcal L(\vb x, \lambda) &= \vb x^\top \mathbf L \vb x - \lambda\,(\vb x^\top \mathbf D \vb x - V)\\ \partial \mathcal L / \partial \vb x &= 2 \mathbf L \vb x - 2 \lambda \mathbf D \vb x = \mathbf 0\\ &\Longrightarrow \ \mathbf L \vb x = \lambda \mathbf D \vb x, \quad \vb x^\top \mathbf L \vb x = \lambda V \end{aligned}

  • the cost is λV\lambda V, so a small λ\lambda is better
  • λ=0\lambda = 0 belongs to x∝1\vb x \propto \mathbf 1, which x⊤D1=0\vb x^\top \mathbf D \mathbf 1 = 0 excludes
  • the solution is the eigenvector of the second smallest eigenvalue

Sadamori Kojaku, 2026

From Lx=λDx\mathbf L \vb x = \lambda \mathbf D \vb x to the normalized Laplacian

The change of variable y=D1/2x\vb y = \mathbf D^{1/2} \vb x gives an ordinary eigenvalue problem.

Lx=λDx  ⟺x=D−1/2y  LD−1/2y=λD1/2y⟺  D−1/2LD−1/2y=λymultiply by D−1/2⟺  Lsymy=λyD−1/2LD−1/2=I−D−1/2AD−1/2\begin{aligned} \mathbf L \vb x = \lambda \mathbf D \vb x \ \ &\overset{\vb x = \mathbf D^{-1/2} \vb y}{\Longleftrightarrow} \ \ \mathbf L \mathbf D^{-1/2} \vb y = \lambda \mathbf D^{1/2} \vb y\\ &\Longleftrightarrow \ \ \mathbf D^{-1/2} \mathbf L \mathbf D^{-1/2} \vb y = \lambda \vb y && \why{multiply by $\mathbf D^{-1/2}$}\\ &\Longleftrightarrow \ \ \mathbf L_{\mathrm{sym}} \vb y = \lambda \vb y && \why{$\mathbf D^{-1/2} \mathbf L \mathbf D^{-1/2} = \mathbf I - \mathbf D^{-1/2} \mathbf A \mathbf D^{-1/2}$} \end{aligned}

  • x⊤Dx=V\vb x^\top \mathbf D \vb x = V becomes y⊤y=V\vb y^\top \vb y = V: the length of y\vb y
  • x⊤D1=0\vb x^\top \mathbf D \mathbf 1 = 0 becomes y⊤D1/21=0\vb y^\top \mathbf D^{1/2} \mathbf 1 = 0, and D1/21\mathbf D^{1/2} \mathbf 1 is the eigenvector of λ=0\lambda = 0, since LsymD1/21=D−1/2L1=0\mathbf L_{\mathrm{sym}} \mathbf D^{1/2} \mathbf 1 = \mathbf D^{-1/2} \mathbf L \mathbf 1 = \mathbf 0
  • y\vb y is the eigenvector with the second smallest eigenvalue, and x=D−1/2y\vb x = \mathbf D^{-1/2} \vb y
Sadamori Kojaku, 2026

Pen and paper: from the cut to the Laplacian

Derive the Ratio Cut result and the Normalized Cut counterpart on paper (10 minutes).

question task
Q1 ∑ixi=0\sum_i x_i = 0 and ∑ixi2=N\sum_i x_i^2 = N
Q2 x⊤Lx=12∑i,jAij(xi−xj)2\vb x^\top \mathbf L \vb x = \tfrac{1}{2} \sum_{i,j} A_{ij} (x_i - x_j)^2
Q3 x⊤Lx=N RatioCut\vb x^\top \mathbf L \vb x = N\, \mathrm{RatioCut}
Q4 Q1 to Q3 for Normalized Cut
Q5 Lx=λDx⇒Lsymy=λy\mathbf L \vb x = \lambda \mathbf D \vb x \Rightarrow \mathbf L_{\mathrm{sym}} \vb y = \lambda \vb y

Sadamori Kojaku, 2026

Embedding

Neural embedding

Sadamori Kojaku, 2026

A neural network turns a graph into vectors

Sadamori Kojaku, 2026

word2vec: words as vectors

word2vec (Mikolov et al. 2013) gives every word a vector, so that words used in similar contexts get close vectors.

xking−xman+xwoman≈xqueen\begin{aligned} & \vb x_{\text{king}} - \vb x_{\text{man}} + \vb x_{\text{woman}} \\ & \approx \vb x_{\text{queen}} \end{aligned}

  • a relation between words is a direction
  • meaning is compared by linear algebra

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
Sadamori Kojaku, 2026

Prediction from context

word2vec predicts a word from a nearby word; every word has a query vector u\vb u and a key vector v\vb v.

  • a pair (i,j)(i, j) that appears in the text gets a high score ui⊤vj\vb u_i^\top \vb v_j
Sadamori Kojaku, 2026

A vector is a point in space

A vector of dd numbers is one point in a dd-dimensional space.

one vector, one point

all words, points in one space

Sadamori Kojaku, 2026

Training by contrast

A real pair (label 1) is pulled together and a random pair (label 0) is pushed apart.

input ii other jj label yy
favors prepared 1
favors water 0
only mind 1
only stone 0

Pr⁡(y=1∣i,j)=σ(ui⊤vj)ℓ=−log⁡σ(ui⊤vj)−log⁡σ(−ui⊤vj′)\Pr(y=1 \mid i, j) = \sigma(\vb u_i^\top \vb v_j) \quad \ell = -\log \sigma(\vb u_i^\top \vb v_j) - \log \sigma(-\vb u_i^\top \vb v_{j'})

Sadamori Kojaku, 2026

word2vec in motion

A window slides over the text, every position gives pairs, and the pairs move the vectors.

Animation: skip-gram with 2 random pairs per real pair, 36 words in 6 sentences, window of one word on each side. Only the query vectors u are drawn.
Sadamori Kojaku, 2026

Walks as sentences

A random walk on a network is a sequence of nodes, in the same way that a sentence is a sequence of words.

text network
word node
sentence random walk
nearby words nodes close together in a walk

Sadamori Kojaku, 2026

DeepWalk (Perozzi et al. 2014)

DeepWalk trains word2vec on random walks, with nodes in place of words.

  • every node ii gets a query vector ui\vb u_i and a key vector vi\vb v_i; ui\vb u_i is its embedding
  • settings: walks per node, walk length, window size; node2vec biases the walk with pp and qq
Perozzi, B., Al-Rfou, R., & Skiena, S. (2014). DeepWalk: online learning of social representations. KDD.
Sadamori Kojaku, 2026

The walk rule sets what is close

Nodes that appear together in walks get close vectors, so a different walk rule gives a different embedding.

Plain walk: move to a random neighbor

Pr⁡(i→j)=Aij/ki\Pr(i \to j) = A_{ij} / k_i

Another rule is a walk with restart (personalized PageRank): return to the start node ss with probability α\alpha.

Pr⁡(i→j)=α [ j=s ]+(1−α) Aij/ki\Pr(i \to j) = \alpha\,[\,j = s\,] + (1-\alpha)\,A_{ij} / k_i

Sadamori Kojaku, 2026

Lab: character embeddings

In the lab notebook, walkers run on the Les Misérables co-appearance network and word2vec embeds the characters.

  • the walk simulator and the word2vec call are given
  • choose a character and list the closest vectors
  • plot the 77 characters in two dimensions

go.skojaku.com/m08lab

Valjean (lavender) and his 10 closest characters (cyan).

Sadamori Kojaku, 2026

Lab task: personalized PageRank walk

The lab uses personalized PageRank: restart at the start node with probability α\alpha.

def ppr_walk(G, start, length, alpha, rng):
    walk = [start]
    # TODO: with probability alpha, go back to `start`;
    #       otherwise move to a random neighbor
    return walk
  • closest characters to ss for α=0.05\alpha = 0.05 and α=0.5\alpha = 0.5?
  • how do they differ from the plain walk?

Sadamori Kojaku, 2026

Embedding

Hyperbolic embedding

Sadamori Kojaku, 2026

Popularity and similarity

Papadopoulos et al. (2012) add similarity to preferential attachment.

  • popularity: an earlier node has had more time to collect links
  • similarity: nodes that are alike are placed close to each other
Papadopoulos, F. et al. (2012). Popularity versus similarity in growing networks. Nature, 489, 537–540.
Sadamori Kojaku, 2026

Growth rule of the model

Node tt gets the radius ln⁡t\ln t and a random angle, and links to its mm closest nodes.

rt=ln⁡tr_t = \ln t

θt∼Uniform[0,2π)\theta_t \sim \mathrm{Uniform}[0, 2\pi)

link to the mm nodes s<ts < t with the smallest s θsts\,\theta_{st}

  • θst\theta_{st}: angle between ss and tt
Sadamori Kojaku, 2026

Observed degree mixes two effects

The observed degree mixes popularity with the size of the community.

ki  ∝k_i \;\propto

κi\hl{\kappa_i}

hidden degree

×\times

ni\hlc{n_i}

community size

Sadamori Kojaku, 2026

Exponential growth of hyperbolic space

In the hyperbolic plane of curvature −1-1, the circumference of a circle grows exponentially with its radius.

Chyp(r)=2πsinh⁡r≈πerC_{\text{hyp}}(r) = 2\pi \sinh r \approx \pi e^{r}

CEuclid(r)=2πrC_{\text{Euclid}}(r) = 2\pi r

  • the area 2π(cosh⁡r−1)2\pi(\cosh r - 1) also grows as ere^{r}
  • the radius is the logarithm of the number of nodes that fit

Sadamori Kojaku, 2026

Hyperbolic space as a continuous tree

Hyperbolic space is the continuous version of a tree.

Poincaré disk, depth dd at distance 0.9 d0.9\,d.

  • depth dd of a tree holds bdb^d nodes; hyperbolic space grows in the same way
Sadamori Kojaku, 2026

WordNet in the Poincaré disk

General words lie near the center and specific words near the boundary.

  • center: general words
  • boundary: specific words
  • the hierarchy of 80,000 nouns fits in a disk of two dimensions

Image: Poincaré embedding of WordNet nouns (Nickel & Kiela 2017; source to be confirmed).
Sadamori Kojaku, 2026

Summary

  • spectral embedding keeps the eigenvectors with the largest terms of A\mathbf A
    • the Laplacian comes from the continuous relaxation of Ratio Cut; the normalized Laplacian, from Normalized Cut
  • word2vec learns a vector per word from word pairs; DeepWalk applies it to random walks
    • the walk rule decides which nodes are close
  • hyperbolic embedding places nodes by popularity (radius) and similarity (angle)
    • the space grows exponentially, as a tree does
Sadamori Kojaku, 2026

MOCK v2 (2026-11-03). Classroom-size type, two columns (text left, figure right). The lab QR is a placeholder; the growth slide is one frame of the planned animation. Wireframe: review/WIREFRAME.md. Build: python3 figures/make_figures.py; python3 ~/.claude/skills/slide-math/tools/check.py m08-embedding.md