Advanced Topics in Network Science

1 Exercises: Graph Neural Networks

1.1 Theoretical Exercises

Pen and Paper Exercises

• ✍️ Pen and paper exercises

The pen and paper exercises cover fundamental concepts including:

1. Spectral Graph Theory: Understanding eigenvalues and eigenvectors of graph matrices
2. Fourier Analysis on Graphs: Extending classical signal processing to graph domains
3. Convolution Operations: Defining convolution for irregular graph structures
4. Message Passing: Mathematical formulation of information aggregation in graphs
5. Network Architecture Design: Principles for designing effective GNN architectures

1.2 Programming Exercises

Coding Exercise: GCN Implementation

Exercise

:class: note

Let’s implement a simple GCN model for node classification. Coding Exercise

This coding exercise will guide you through:

1. Building a GCN from Scratch: Implementing the basic GCN layer
2. Node Classification: Training GCN for semi-supervised node classification
3. Spectral Filtering: Understanding how GCNs relate to spectral graph theory
4. Comparison with Other Methods: Benchmarking against traditional approaches

Key Learning Objectives

Through these exercises, you will:

• Understand the Mathematics: Connect spectral graph theory to practical GNN implementations
• Implement Core Algorithms: Build GCN, GraphSAGE, GAT, and GIN from fundamental principles
• Apply to Real Problems: Use GNNs for node classification, graph classification, and link prediction
• Analyze Performance: Compare different GNN architectures and understand their strengths/weaknesses
• Debug and Optimize: Learn common pitfalls and optimization strategies for GNNs

Exercise Topics

1. Basic GCN Implementation
   • Implement the GCN layer forward pass
   • Add self-loops and normalization
   • Train on Cora dataset for node classification
2. Spectral Analysis
   • Visualize graph spectra and eigenvectors
   • Implement spectral filtering
   • Compare low-pass vs high-pass filters
3. Advanced Architectures
   • Implement GraphSAGE with different aggregators
   • Build GAT with attention visualization
   • Create GIN and test on graph isomorphism
4. Practical Applications
   • Social network analysis
   • Citation network node classification
   • Molecular property prediction

These exercises bridge theory and practice, ensuring you understand both the mathematical foundations and practical implementation details of Graph Neural Networks.