🔥 New blog: Graph ML papers at #ICML2022 in 10 categories: from denoising diffusion generation through transformers, expressive and explainable GNNs to algorithmic reasoning, knowledge graphs, and cool applications! Jointly with Zhaocheng Zhu mgalkin.medium.com/graph-machine-…

But seem bad experiences are more common. We spent a whole week for the rebuttal and haven’t got any single response yet😅 our submission is right above the borderline, which does need discussions😢

Eventually 3 reviewers replied and we had some discussions. 2 increased their scores😂 appreciate these responsible reviewers😃

📜🚨📜🚨 NN loss landscapes are full of permutation symmetries, ie. swap any 2 units in a hidden layer. What does this mean for SGD? Is this practically useful? For the past 5 yrs these Qs have fascinated me. Today, I am ready to announce "Git Re-Basin"! arxiv.org/abs/2209.04836

Enjoyed the tutorial this morning, Graph Embedding for RecSys by Michael Bronstein and Ying Xiao really insightful

A recent exchange on Twitter made me realize that many may be using tensors without knowing how they are represented in memory, and how it makes some operations super fast. This is standard stuff common to many tensor libraries, including our beloved NumPy and PyTorch.

One of the best GNN papers this year, an anonymous ICLR submission: openreview.net/pdf?id=r9hNv76… Shows 4 families of efficient+expressive GNNs cannot compute biconnectivity properties (based on cut nodes/edges), but our ESAN framework (arxiv.org/abs/2110.02910) can compute them! 1/n

There are so many submissions on Transformers for graph data at ICLR, this line of work keeps growing✈️ Big trends: scaling to large graphs w/ many nodes, better theory, good results on more and more datasets 📈 Here is a summary thread of several 1/n

At 2100+ (not a typo) pages 🤯, this is almost all the Math you need for Machine Learning without dumbing it down! PDF link: cis.upenn.edu/~jean/math-dee…

I am at #neurips2022 this week. Looking forward to chatting with more people here! Our poster session is on Tuesday Nov 29, 4 PM -- 6 PM (CST) @ Hall J #341. #gnn #graphneuralnetworks Our paper: openreview.net/forum?id=sZAbX…. Feel free to swing by our poster if you're interested.

How to email a professor to admit you/be your advisor, a 🧵 I've mentioned a few poorly executed emails from prospective students. Here's some tips on how to do it well and stand out from the crowd. Other faculty can add their own tips. 1/n

At this year's NeurIPS, Geoffrey Hinton presented the forward-forward (FF) algorithm, an alternative to backprop. I took a stab at implementing FF in PyTorch, and here it is:github.com/mohammadpz/pyt… Below is my understanding of the idea (with a grain of salt): 1/5

Most interesting papers on temporal graph GNNs of the last year?

Thrilled to share - Moirai - Our time-series foundation model. Armed with architectural innovations to help deal with heterogeneity of time-series across various domains and trained on LOTSA data (27b observations), Moirai is able to achieve impressive zero-shot performance!

👋Hi everyone👋, we are releasing our updated time-series foundation model Moirai-1.1-R and our time-series code repo UNI2TS 1.1.0! The new Moirai model achieved significant improvements (~20%) for low-frequency cases like Yearly and Quarterly data in Normalised Mean Absolute

Happy to share that we proposed Moirai-MoE, a new time series foundation model. 😃 [Paper: arxiv.org/abs/2410.10469] [Code: github.com/SalesforceAIRe…] [Models: huggingface.co/collections/Sa…] The short version has been accepted by NeurIPS workshop Time Series in the Age of Large Models.

Salesforce AI Research Introduces Moirai-MoE: A MoE Time Series Foundation Model that Achieves Token-Level Model Specialization Autonomously Researchers from Salesforce AI Research, the National University of Singapore, and the Hong Kong University of Science and Technology