Do’s and Don’ts of Vision Transformers (ViTs) Permalink
A short blogpost covering lessons learned regarding Vision Transformers.
Hi there, My name is Saurav
I am a Computer Science student at the University of Manchester and am interested in Geometric Learning and Self Supervised Learning. Currently, I am working on:
- Benchmarking Self Supervised Learning Methods
- Self Supervised Learning Methods for Graphs
- Knowledge Distillation Deep Dive (Teacher Size vs Student Size. ViT vs Conv vs MLPs)
- Generative models for regulatory DNA sequences based on diffusion models
Recent Posts
Ridge Regression: extending Linear Regression Permalink
An illustration covering Ridge Regression as an extension of Linear Regression using scikit-learn, complete with code and interactive visualizations.
Writing a Training Loop in JAX + FLAX Permalink
A simple end-to-end training and evaluation pipeline in JAX, Flax and Optax.
How to write efficient training loops in PyTorch Permalink
Learn how to writing extremely memory and compute efficient training loops in PyTorch in this short tutorial complete with code and interactive visualizations.
How to use 8 bit Optimizers in PyTorch Permalink
Learn how to use 8 bit optimizers in PyTorch in this short tutorial complete with code and interactive visualizations.