rosicast

Math

  • Notation
    • Basic math symbols
    • Algebra
    • Calculus
    • Linear algebra
    • Probability
    • Set theory
    • Statistics

Levels

  • Levels

Jax related

  • Haiku: Sonnet for JAX

Models

  • GAT
  • GCN
  • GraphSAGE

Datasets

  • Datasets
  • CORA
  • Heterogeneous graphs and DeepSNAP
    • Device
    • Installation
    • DeepSNAP Basics
    • 1) DeepSNAP Heterogeneous Graph
      • Question 1.1: Assigning Node Type and Node Features
      • Question 1.2: Assigning Edge Types
      • Heterogeneous Graph Visualization
      • Transforming to DeepSNAP representation
        • Question 1.3: How many nodes are of each type (10 Points)
        • Question 1.4: Message Types - How many edges are of each message type (10 Points)
        • Question 1.5: Dataset Splitting - How many nodes are in each dataset split? (10 Points)
        • DeepSNAP Dataset Visualization
    • 2) Heterogeneous Graph Node Property Prediction
      • Setup
      • Dataset
      • Implementing HeteroGNNConv
      • Heterogeneous GNN Wrapper Layer
      • Initialize Heterogeneous GNN Layers
      • HeteroGNN
      • Training and Testing
      • Dataset and Preprocessing
    • Start Training!
      • Training the Mean Aggregation
        • Question 2.1: What is your maximum test set micro F1 score for the best_model when using mean aggregation? (10 points)
        • Question 2.2: What is your maximum test set macro F1 score for the best_model when using the mean aggregation? (10 points)
      • Training the Attention Aggregation
        • Question 2.3: What is your maximum test set micro F1 score for the best_model when using the attention aggregation? (4 points)
        • Question 2.4: What is your maximum test set macro F1 score for the best_model when using the attention aggregation? (4 points)
    • Attention for each Message Type
    • Submission
  • DRKit
  • ogbg-molhiv
    • Load and preprocess the dataset
    • References
  • ogbn-arxiv
    • Leaderboard
    • Setup
    • Load and Preprocess the Dataset
    • References

ogbn-proteins

  • GCN on ogbn-proteins
  • GraphSAGE on ogbn-proteins

ogbn-arxiv

  • ogbn-arxiv_GCN

CS224w_2021_fall

  • CS224W - Colab 0
    • NetworkX Tutorial
      • Setup
      • Graph
      • Node
      • Edge
      • Visualization
      • Node Degree and Neighbor
      • Other Functionalities
      • Documentation
    • PyTorch Geometric Tutorial
      • Installing dependencies
      • Visualization
      • Introduction
      • Dataset
      • Data
      • Edge Index
      • Implementing Graph Neural Networks (GNNs)
        • What is the output of a GNN?
        • Training on the Karate Club Network
      • Documentation
  • CS224W - Colab 1
    • 1) Graph Basics
      • Setup
      • Zachary’s karate club network
      • Question 1: What is the average degree of the karate club network? (5 Points)
      • Question 2: What is the average clustering coefficient of the karate club network? (5 Points)
      • Question 3: What is the PageRank value for node 0 (node with id 0) after one PageRank iteration? (5 Points)
      • Question 4: What is the (raw) closeness centrality for the karate club network node 5? (5 Points)
    • 2) Graph to Tensor
      • Setup
      • PyTorch tensor basics
      • Question 5: Get the edge list of the karate club network and transform it into torch.LongTensor. What is the torch.sum value of pos_edge_index tensor? (10 Points)
      • Question 6: Please implement following function that samples negative edges. Then answer which edges (edge_1 to edge_5) can be potential negative edges in the karate club network? (10 Points)
    • 3) Node Emebedding Learning
      • Setup
      • Visualize the initial node embeddings
      • Question 7: Training the embedding! What is the best performance you can get? Please report both the best loss and accuracy on Gradescope. (20 Points)
      • Visualize the final node embeddings
    • Submission
  • CS224W - Colab 2
    • Device
    • Setup
    • 1) PyTorch Geometric (Datasets and Data)
      • PyG Datasets
      • Question 1: What is the number of classes and number of features in the ENZYMES dataset? (5 points)
      • PyG Data
      • Question 2: What is the label of the graph with index 100 in the ENZYMES dataset? (5 points)
      • Question 3: How many edges does the graph with index 200 have? (5 points)
    • 2) Open Graph Benchmark (OGB)
      • Dataset and Data
      • Question 4: How many features are in the ogbn-arxiv graph? (5 points)
    • 3) GNN: Node Property Prediction
      • Setup
      • Load and Preprocess the Dataset
      • GCN Model
      • Question 5: What are your best_model validation and test accuracies?(20 points)
    • 4) GNN: Graph Property Prediction
      • Load and preprocess the dataset
      • Graph Prediction Model
        • Graph Mini-Batching
        • Implemention
      • Question 6: What are your best_model validation and test ROC-AUC scores? (20 points)
      • Question 7 (Optional): Experiment with the two other global pooling layers in Pytorch Geometric.
    • Submission
  • CS224W - Colab 3
    • Device
    • Installation
    • 1) GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GraphSage Implementation
      • Building Optimizers
      • Training and Testing
      • Let’s Start the Training!
      • Question 1.1: What is the maximum accuracy obtained on the test set for GraphSage? (10 points)
  • CS224W - Colab 4
    • Device
    • Installation
    • 1) GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GAT Implementation
        • Multi-Head Attention
      • Building Optimizers
      • Training and Testing
      • Let’s Start the Training!
      • Question 1: What is the maximum accuracy obtained on test set for GAT? (10 points)
  • CS224W - Colab 5
    • Device
    • Installation
    • DeepSNAP Basics
    • 1) DeepSNAP Heterogeneous Graph
      • Question 1.1: Assigning Node Type and Node Features
      • Question 1.2: Assigning Edge Types
      • Heterogeneous Graph Visualization
      • Transforming to DeepSNAP representation
      • Question 1.3: How many nodes are of each type (10 Points)
      • Question 1.4: Message Types - How many edges are of each message type (10 Points)
      • Question 1.5: Dataset Splitting - How many nodes are in each dataset split? (10 Points)
      • DeepSNAP Dataset Visualization
    • 2) Heterogeneous Graph Node Property Prediction
      • Setup
      • Dataset
      • Implementing HeteroGNNConv
      • Heterogeneous GNN Wrapper Layer
      • Initialize Heterogeneous GNN Layers
      • HeteroGNN
      • Training and Testing
      • Dataset and Preprocessing
      • Start Training!
      • Training the Mean Aggregation
      • Question 2.1: What is your maximum test set micro F1 score for the best_model when using mean aggregation? (10 points)
      • Question 2.2: What is your maximum test set macro F1 score for the best_model when using the mean aggregation? (10 points)
      • Training the Attention Aggregation
      • Question 2.3: What is your maximum test set micro F1 score for the best_model when using the attention aggregation? (4 points)
      • Question 2.4: What is your maximum test set macro F1 score for the best_model when using the attention aggregation? (4 points)
      • Attention for each Message Type
    • Submission

PyTorch_Geometric

  • 1. Introduction: Hands-on Graph Neural Networks
    • Implementing Graph Neural Networks
      • Embedding the Karate Club Network
      • Training on the Karate Club Network
    • Conclusion
  • 2. Node Classification with Graph Neural Networks
    • Training a Multi-layer Perception Network (MLP)
    • Training a Graph Neural Network (GNN)
    • Conclusion
    • (Optional) Exercises
  • 3. Graph Classification with Graph Neural Networks
    • Mini-batching of graphs
    • Training a Graph Neural Network (GNN)
    • (Optional) Exercise
    • Conclusion
  • 4. Scaling Graph Neural Networks
    • Conclusion
  • 5. Point Cloud Classification with Graph Neural Networks
    • Data Handling
    • Point Cloud Generation
    • PointNet++
      • Phase 1: Grouping via Dynamic Graph Generation
    • Phase 2: Neighborhood Aggregation
    • Network Architecture
    • Training Procedure
    • (Optional) Exercises
      • 1. Rotation-invariant PointNet Layer
      • 2. Downsampling Phase via Farthest Point Sampling
  • 6. Explaining GNN Model Predictions using Captum
    • Loading the dataset
    • Visualizing the data
    • Sample visualization
    • Training the model
      • Define the model
      • Define train and test functions
      • Train the model for 100 epochs
    • Explaining the predictions
  • 7. Customizing Aggregations within Message Passing with torch_geometric.nn.aggr
    • Loading the dataset
    • Define train, test and run functions
    • Training GNNs with torch_geometric.nn.aggr package
      • Define a GNN class
      • Original interface with string type as aggregation argument
      • Use a single aggregation module as aggregation argument
      • Use a list of aggregation strings as aggregation argument
      • Use a list of aggregation modules as aggregation argument
      • Use a list of mixed modules and strings as aggregation argument
      • Define multiple learnable aggregations with keyword arguments
      • Define multiple aggregations with MultiAggregation module
    • Conclusion
  • 8. Node Classification with W&B
    • Node Classification with Graph Neural Networks
    • Training a Multi-layer Perception Network (MLP)
    • Training a Graph Neural Network (GNN)
    • Using W&B Sweeps
    • Conclusion
    • (Optional) Exercises

Pyg_Tutorial_Project

  • Tutorial 1: Introduction
    • Load the dataset
      • Dataset properties
      • Dataset shapes
  • Tutorial 2: PyTorch basics
    • Outline
      • Imports
    • Datasets
      • A raw dataset
      • PyTorch Dataset
      • PyTorch DataLoader
    • Models
      • Model inspection
      • The forward method
    • Losses
      • The MSE loss
      • Giving a score to the model (parameters) given the training set
    • Optimization
      • Gradients
      • Handmade optimization
      • Stochastic GD
      • PyTorch optim
  • Tutorial 3: GAT implementation
    • Outline
    • Structure
    • Let’s start from the forward method
      • Linear Transformation
      • Attention Mechanism
      • Masked Attention
        • h_prime vs h
    • Build the layer
    • Use it
  • Tutorial 4: Convolutional Layers - Spectral methods
    • Outline
    • Imports
    • Why convolution in ML
    • Some theory on convolution
      • Definition
      • Fourier transform
      • Definition of the Fourier transform
      • Connection with the Laplacian
      • Summary
    • Convolution on graphs
      • Graph Laplacian
      • Graph spectrum, Fourier transform, and convolution
    • Spectral-convolutional layers in PyTorch Geometric
      • ChebConv
        • Goal:
        • Chebyshev approximation
        • Chebyshev approximation of the filter
        • Property
        • Fast approximated convolution
        • Properties:
      • GCNConv
        • As message passing
  • Tutorial 5: Aggregation
    • Message Passing Class
      • LAF Aggregation Module
      • PNA Aggregation
      • Test the new classes
  • Tutorial 6: GAE & VGAE
    • Graph AutoEncoder GAE
      • Load the data
      • Define the Encoder
      • Define the Autoencoder
      • Are the results (AUC) and (AP) easy to read and compare?
    • Use Tensorboard
      • Import tensorboard
        • Installation: (if needed) “pip install tensorboard”
      • Graph Variational AutoEncoder (GVAE)
  • Tutorial 7: ARGA & ARVGA
    • Imports
    • Define the dataset
    • Define the model
    • Initialize the model
      • Train the model
  • Tutorial 9_1: Recurrent GNNs
    • Graph Neural Network Model
      • Gated Graph Neural Network
  • Tutorial 9_2: Recurrent GNNs
    • Graph Neural Network Model
    • Gated Graph Neural Network
  • Tutorial 11: DeepWalk and node2vec - Implementation details
    • Setup
    • Random walks
      • The data loader
      • Visualization
    • Training
      • Model definition
      • Training function
      • Test function
      • Training
      • Visualization
  • Tutorial 12: GAE for link prediction
    • Simple autoencoder model
  • Tutorial 13: Node2Vec for link prediction
    • Step 1
      • The dataset
      • Use only one moleculas compound
      • Build the graph with
    • Step 2
      • Visualize node embedding
    • Step 3
      • Visualize edge embedding
    • Step 4
  • Tutorial 14: Data Handling in PyG (Part 1)
    • Data
      • Batch
      • Cluster
      • Sampler
      • Datasets
  • Tutorial 15: Data Handling in PyG (Part 2)
    • Custom PyG dataset
      • Open Graph Benchmark datasets
      • Benchmarking Graph Neural Networks Datasets

Representation

  • Graph representation

Dive

  • GAT
    • Installation
    • GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GAT Implementation
        • Multi-Head Attention
      • build the model and see inside
      • Building Optimizers
      • Training and Testing
      • Let’s Start the Training!
    • reference
  • GraphSAGE
    • Installation
    • CORA data
    • GNN Layers
      • Implementing Layer Modules
      • GNN Stack Module
      • Creating Our Own Message Passing Layer
      • GraphSage Implementation
      • Building Optimizers
      • Build the model
      • what happen in the model?
    • Training and Testing
    • Let’s Start the Training!
    • reference

Resources

  • Resources

TodoList

  • Todolist
rosicast
  • »
  • Resources
  • Edit on GitHub

Resources¶

DeepSNAP https://snap.stanford.edu/deepsnap/

PyTorch Geometric https://pytorch-geometric.readthedocs.io/en/latest/

Networkx https://networkx.org/

torch sparce https://github.com/rusty1s/pytorch_sparse

Previous Next

© Copyright 2022, rosicast. Revision 8d286b1b.

Built with Sphinx using a theme provided by Read the Docs.