(original) (raw)
��
Fall 2023: NETS 7332 -- Machine Learning with Graphs (CRN 19900)
<o:p>
General Information <o:p>
Time: Tuesdays & Fridays 1:35 3:15 PM Eastern<o:p> | Place: 177 Huntington Ave, 2nd Floor Conference Room<o:p> |
Instructor: Tina Eliassi-Rad<o:p>Course website on Canvas: https://northeastern.instructure.com/courses/158581 <o:p> | Office hours: Available by appointment. Email eliassi [at] ccs [dot] neu [dot] edu |
<o:p>
Overview <o:p>
This 4-credit PhD-level course covers state-of-the-art research on mining and learning with graphs. Topics include, but are not limited to, vertex classification, graph clustering, link prediction and analysis, graph distances, graph embedding and network representation learning, deep learning on graphs, anomaly detection on graphs, graph summarization, network inference, adversarial learning on networks, and notions of fairness in social networks.
<o:p>
Prerequisites<o:p>
Students are expected to have taken courses on or have knowledge of the following:
<![if !supportLists]>o <![endif]>Calculus and linear algebra<o:p>
<![if !supportLists]>o <![endif]>Basic statistics, probability, machine learning, or data mining<o:p>
<![if !supportLists]>o <![endif]>Algorithms and programming skills (e.g., Python, Julia, C, Java, Matlab, or any programming language of their preference)<o:p>
<o:p>
Textbooks <o:p>
This course does not have a designated textbook. The readings are assigned in the syllabus (see below).
<o:p>
Here are some textbooks (all optional) on machine learning and data mining:
- Deep Learning and Graph Representation Learning
- Charu C. Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer, 2018.
- Michael Bronstein, Joan Bruna, Taco Cohen and Petar Veli kovi. Geometric Deep Learning: Grids, Graphs, Groups, Geodesics, and Gauges. arXiv:2104.13478, April 2021.
- Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning. MIT Press, 2016.
- William L. Hamilton. Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, Vol. 14, No. 3, Pages 1-159, 2020.
- Jure Leskovec. Machine Learning with Graphs Video Lectures, Stanford Online, 2021.
- Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veli kovi. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges, 2021.
- Data Mining and Graph Mining
- Charu C. Aggarwal. Data Mining, The Textbook. Springer 2015.
- Christos Faloutsos, Deepayan Chakrabarti. Graph Mining: Laws, Tools, and Case Studies. Morgan & Claypool Publishers, 2012.
- Jiawei Han, Micheline Kamber, Jian Pei. Data Mining: Concepts and Techniques. Morgan Kaufmann, 3rd edition, 2011.
- David J. Hand, Heikki Mannila, Padhraic Smyth. Principles of Data Mining. A Bradford Book, 2001
- Anand Rajaraman, Jurij Leskovec, and Jeffrey Ullman. Mining of Massive Datasets. Cambridge University Press, v2.1, 2014. (free online) (Errata)
- Pang-Ning Tan, Michael Steinbach, Vipin Kumar. Introduction to Data Mining. Pearson, 2nd edition, 2018.
- Machine Learning
- Charu C. Aggarwal. Linear Algebra and Optimization for Machine Learning: A Textbook. Springer, 2020.
- Christopher Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
- Peter Flach. Machine Learning: The Art and Science of Algorithms that Make Sense of Data. Cambridge University Press, 2012.
- Tom Mitchell. Machine Learning. McGraw Hill, 1997.
- Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. MIT Press, 2012.
- Statistics
- Trevor Hastie, Robert Tibshirani, Jerome Friedman. Elements of Statistical Learning. Springer, 2nd edition, 2009. (free online)
<o:p>
Resources <o:p>
- Probability Review (David Blei, Princeton)
- Probability Theory Review (Arian Maleki and Tom Do, Stanford)
- Linear Algebra Review and Reference (Zico Kolter and Chuong Do, Stanford)
- Theoretical CS Cheat Sheet (Princeton)
- Encyclopedia of Distances (Michel Marie Deza and Elena Deza)
<o:p>
Grading<o:p>
<![if !supportLists]>o <![endif]>Class presentations (40%)<o:p>
<![if !supportLists]>o <![endif]>I will team up students into groups of two.<o:p>
<![if !supportLists]>o <![endif]>Each week has an assigned team. That team is responsible for presenting the readings for that week. <o:p>
<![if !supportLists]>o <![endif]>Besides the readings, each paper is likely to have additional materials on the Web. Examples include supplemental materials, video, code, data, etc. These are helpful for presentations and class projects.<o:p>
<![if !supportLists]>o <![endif]>Class project (50%)<o:p>
<![if !supportLists]>o <![endif]>Each team will choose (by Saturday October 7, 2023 at 11:59 PM Eastern) one of the papers in the syllabus to replicate.
<![if !supportLists]>o <![endif]>In addition to replication, each team will propose extension(s) to the chosen paper and implement those extension(s).
<![if !supportLists]>o <![endif]>Each team will write a report (maximum 6 pages) detailing what was learned. Use the style files at https://paperswithcode.com/static/rc2020/ML-Reproducibility-Challenge-2020-Template.zip. <o:p>
<![if !supportLists]>o <![endif]>Reports are due on Saturday December 9, 2023 at 11:59 PM Eastern.<o:p>
<![if !supportLists]>o <![endif]>Project Presentations (10%)<o:p>
<![if !supportLists]>o <![endif]>Each team will have ~25 minutes to present what they learned when they tried to reproduce their chosen paper, followed by 5 minutes of Q&A.<o:p>
<![if !supportLists]>o <![endif]>The slides for the presentations are due on Friday December 15, 2023 at 12:00 PM Eastern. <o:p>
<o:p>
Schedule/Syllabus (Subject to Change) <o:p>
Date<o:p> | Lecturer<o:p> | Readings<o:p> |
Fri Sep 8 | Tina Eliassi-Rad | Overview<o:p> |
Tue Sep 12 | Tina Eliassi-Rad | Various Representations<o:p> <![if !supportLists]>� <![endif]>The Why, How, and When of Representations for Complex Systems<o:p> <![if !supportLists]>� <![endif]>Non-backtracking Cycles: Length Spectrum Theory and Graph Mining Applications |
Fri Sep 15 | Tina Eliassi-Rad | Role Discovery<o:p> <![if !supportLists]>� <![endif]>It's Who You Know: Graph Mining Using Recursive Structural Features<o:p> <![if !supportLists]>� <![endif]>RolX: Structural Role Extraction & Mining in Large Graphs<o:p> <![if !supportLists]>� <![endif]>Guided learning for Role Discovery (GLRD): Framework, Algorithms, and Applications |
Tue Sep 19 | Graph Machine Learning for Drug Discovery + Graph Machine Learning Benchmarks<o:p> <![if !supportLists]>� <![endif]>AI-Bind: Improving Binding Predictions for Novel Protein Targets and Ligands<o:p> <![if !supportLists]>� <![endif]>Disentangling Node Attributes from Graph Topology for Improved Generalizability in Link Prediction<o:p> <![if !supportLists]>� <![endif]>OGB: Open Graph Benchmark<o:p> <![if !supportLists]>� <![endif]>DGL: Deep Graph Library<o:p> <![if !supportLists]>� <![endif]>TGB: Temporal Graph Benchmark<o:p> <![if !supportLists]>� <![endif]>GraphWorld | |
Fri Sep 22 | Network Comparison and Graph Distances<o:p> <![if !supportLists]>� <![endif]>Network Comparison and the Within-ensemble Graph Distance<o:p> <![if !supportLists]>� <![endif]>netrd: A library for Network Reconstruction and Graph Distances | |
Tue Sep 26 | Node Embedding<o:p> <![if !supportLists]>� <![endif]>STABLE: Identifying and Mitigating Instability in Embeddings of the Degenerate Core<o:p> <![if !supportLists]>� <![endif]>Laplacian Eigenmaps for Dimensionality Reduction and Data Representation<o:p> <![if !supportLists]>� <![endif]>node2vec: Scalable Feature Learning for Networks<o:p> <![if !supportLists]>� <![endif]>Structural Deep Network Embedding<o:p> <![if !supportLists]>� <![endif]>Machine Learning on Graphs: A Model and Comprehensive Taxonomy | |
Fri Sep 29 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | More on Representation Learning<o:p> <![if !supportLists]>� <![endif]>Deep Graph Infomax<o:p> <![if !supportLists]>� <![endif]>Graph Representation Learning via Graphical Mutual Information Maximization<o:p> <![if !supportLists]>� <![endif]>[optional] Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding |
Tue Oct 3 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | Low-rank Representations of Complex Networks<o:p> <![if !supportLists]>� <![endif]>The Impossibility of Low-Rank Representations for Triangle-Rich Complex Networks<o:p> <![if !supportLists]>� <![endif]>Node Embeddings and Exact Low-Rank Representations of Complex Networks |
Fri Oct 6 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | Collective Classification<o:p> <![if !supportLists]>� <![endif]>Collective Classification in Network Data<o:p> <![if !supportLists]>� <![endif]>Graph Belief Propagation Networks<o:p> <![if !supportLists]>� <![endif]>[optional] Cautious Collective Classification |
Tue Oct 10 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Graph Neural Networks: A Review of Methods and Applications<o:p> <![if !supportLists]>� <![endif]>Transformers are Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>[optional] Everything is Connected: Graph Neural Networks |
Fri Oct 13 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | This and That (I)<o:p> <![if !supportLists]>� <![endif]>A Generalization of Transformer Networks to Graphs<o:p> <![if !supportLists]>� <![endif]>Hyperbolic Graph Convolutional Neural Networks |
Tue Oct 17 | Tina Eliassi-Rad | Label Propagation on Graphs<o:p> <![if !supportLists]>� <![endif]>Combining Label Propagation and Simple Models Out-performs Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification<o:p> <![if !supportLists]>� <![endif]>[optional] Message passing all the way up |
Fri Oct 20 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | ML on Heterogeneous Graphs<o:p> <![if !supportLists]>� <![endif]>Modeling Relational Data with Graph Convolutional Networks<o:p> <![if !supportLists]>� <![endif]>Heterogeneous Graph Transformer |
Tue Oct 24 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | W-L Graph Kernels and Power of GNNs<o:p> <![if !supportLists]>� <![endif]>Weisfeiler-Lehman Graph Kernels<o:p> <![if !supportLists]>� <![endif]>How Powerful are Graph Neural Networks?<o:p> <![if !supportLists]>� <![endif]>[optional] A Reduction of a Graph to a Canonical Form and an Algebra arising during this Reduction<o:p> <![if !supportLists]>� <![endif]>[optional] Theory of Graph Neural Networks: Representation and Learning |
Fri Oct 27 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | Stability and Counting in GNNs<o:p> <![if !supportLists]>� <![endif]>Tree Mover s Distance: Bridging Graph Metrics and Stability of Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Can Graph Neural Networks Count Substructures? |
Tue Oct 31 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | GNNs for Recommendation Systems<o:p> <![if !supportLists]>� <![endif]>Neural Graph Collaborative Filtering<o:p> <![if !supportLists]>� <![endif]>Graph Convolutional Neural Networks for Web-Scale Recommender Systems |
Fri Nov 3 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | Ayan Chatterjee will emcee this lecture. <o:p> Hypergraphs<o:p> <![if !supportLists]>� <![endif]>Random Walks on Hypergraphs with Edge-Dependent Vertex Weights <o:p> <![if !supportLists]>� <![endif]>Hypergraph Neural Networks |
Tue Nov 7 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | David Liu will emcee this lecture.<o:p> Explainability in GNNs<o:p> <![if !supportLists]>� <![endif]>Explainability in Graph Neural Networks: A Taxonomic Survey<o:p> <![if !supportLists]>� <![endif]>GNNExplainer: Generating Explanations for Graph Neural Networks |
Fri Nov 10 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | Explainability & Trustworthiness of GNNs<o:p> <![if !supportLists]>� <![endif]>GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Trustworthy Graph Neural Networks: Aspects, Methods and Trends |
Tue Nov 14 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | This and That (II)<o:p> <![if !supportLists]>� <![endif]>Principal Neighbourhood Aggregation for Graph Nets<o:p> <![if !supportLists]>� <![endif]>How does over-squashing affect the power of GNNs? |
Fri Nov 17 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | Oversmooting<o:p> <![if !supportLists]>� <![endif]>A Survey on Oversmoothing in Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs |
Tue Nov 21 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | Fairness & Equality<o:p> <![if !supportLists]>� <![endif]>FAIRGEN: Towards Fair Graph Generation<o:p> <![if !supportLists]>� <![endif]>Information Access Equality on Generative Models of Complex Networks |
Fri Nov 24 | No class Thanksgiving break | |
Tue Nov 28 | Team 1 Alyssa Smith & Mel Allen & Remy LeWinter | Learning on Signed Networks<o:p> <![if !supportLists]>� <![endif]>Learning Signed Network Embedding via Graph Attention<o:p> <![if !supportLists]>� <![endif]>Signed Graph Attention Networks |
Fri Dec 1 | Team 2 Joey Ehlert & Mortiz Laber & Sam Dies | Invariance and Equivariance<o:p> <![if !supportLists]>� <![endif]>E(n) Equivariant Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Invariant and Equivariant Graph Networks |
Tue Dec 5 | Team 3 Julian Gullett & Sagar Kumar & Yixuan Liu | *-aware GNNs<o:p> <![if !supportLists]>� <![endif]>Position-aware Graph Neural Networks<o:p> <![if !supportLists]>� <![endif]>Identity-aware Graph Neural Networks |
Fri Dec 8 | No class Project reports are due on Sat Dec 9 at 11:59pm Eastern. | This and That (III)<o:p> <![if !supportLists]>� <![endif]>[optional] Pitfalls of Graph Neural Network Evaluation<o:p> <![if !supportLists]>� <![endif]>[optional] Classic Graph Structural Features Outperform Factorization-Based Graph Embedding Methods on Community Labeling |
Tue Dec 12 | No class NetSI Qualifying Exam Week | |
Fri Dec 15 | In-class project presentations | Your slides are due at 12pm Eastern. |
<o:p>