Graph Contrastive Learning Permalink
An overview of “Deep Graph Contrastive Representation Learning”
Posts by Year
2022
What is (AdaLoRA) Adaptive Budget Allocation for LoRA Permalink
An overview of Adaptive Budget Allocation for Parameter Efficient Fine-Tuning
What is QLoRA? Permalink
An overview of (QLoRA) Efficient Finetuning of Quantized LLMs
What Are Intrinsic Dimensions? The Secret Behind LoRA Permalink
An overview of Intrinsic Dimensions and how they enable Low-Rank Domain Adaptation (LoRA)
A Brief Introduction to (LoRA) Low-Rank Adaptation of Large Language Models Permalink
An overview of “Low-Rank Adaptation of Large Language Models”
A Brief Introduction to Mixture Model Networks (MoNet) Permalink
brief overview of the Mixture Model Networks (MoNet) architecture
A Brief Introduction to Graph Attention Networks Permalink
brief overview of the Graph Attention Networks architecture
A Brief Introduction to Residual Gated Graph Convolutional Networks Permalink
brief overview of the Residual Gated Graph Convolutional Network architecture
An Introduction to GraphSAGE Permalink
A brief overview of GraphSAGE
What are Graph Isomorphism Networks? Permalink
A brief overview of Graph Isomorphism Networks (GIN)
Writing a Training Loop in JAX + FLAX Permalink
A simple end-to-end training and evaluation pipeline in JAX, Flax and Optax.
An Introduction to Convolutional Graph Neural Networks Permalink
A beginner friendly introduction to Convolutional Graph Neural Networks (GCNs).
An Introduction to Graph Attention Networks Permalink
A beginner friendly introduction to Attention based Graphical Neural Networks (GATs).
How to Create a Image Classification Model in JAX/Flax Permalink
Learn how to create a simple image classification model in Flax in this short tutorial complete with code and interactive visualizations.
An Introduction to Message Passing Graph Neural Networks (GNNs) Permalink
A beginner friendly introduction to Message Passing Graph Neural Networks (MPGNNs).
Graph Neural Networks (GNNs) with Learnable Structural and Positional Representations Permalink
An in-depth breakdown of “Graph Neural Networks with Learnable Structural and Positional Representations” by Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laur...
MLP-Mixer: An all-MLP Architecture for Vision Permalink
In the past few months there have been various papers proposing MLP based architectures without Attention or Convolutions. This report analyses the paper ‘ML...
2021
DINO: Emerging Properties in Self-Supervised Vision Transformers Permalink
Breakdown of Emerging Properties in Self-Supervised Vision Transformers by Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojan...
What Do Compressed Deep Neural Networks Forget? Permalink
Pruning can be a clever way to reduce a model’s resource greediness. But what gets forgotten when you do?
Feedforward Networks for Image Classification Permalink
In the past few months there have been various papers proposing MLP based architectures without Attention or Convolutions. This report analyses the paper ‘Re...
Bringing Back MLPs Permalink
It’s no news that transformers have dominated the field of deep learning ever since 2017. But in their recent work, titled ‘Pay Attention to MLPs,’ Hanxiao L...
Fourier Transform in Neural Networks ??!! Permalink
Continuing on the recent series of reports analyzing newly proposed pure MLP based architectures. In this report I breakdown “FNet Mixing Tokens with Fourier...