All Reviews

Published

Monday, January 13, 2025

ViT — An Image is worth 16x16 words: Transformers for Image Recognition at scale
While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision…
Oren Bochman
Monday, January 13, 2025

Temporal Abstraction in Reinforcement Learning with the Successor Representation
This paper review is an extended introduction to temporal abstraction using options. It covers lots of advanced concepts in reinforcement learning that were introduced in…
Oren Bochman
Monday, January 13, 2025

FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications
Acquiring the desired font for various design tasks can be challenging and requires professional typographic knowledge. While previous font retrieval or generation works…
Oren Bochman
Monday, January 13, 2025

LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
In (BehnamGhader et al. 2024) the authors consider using LLMs which are mostly decoder only transformers as text encoders. This allows them to use the LLMs for NLP tasks…
Oren Bochman
Monday, January 13, 2025

MambaVision A Hybrid Mamba-Transformer Vision Backbone
In (Hatamizadeh and Kautz 2024), the authors apply the State Space Model (SSM) inherent in recently introduced Mamba architecture, (Gu and Dao 2023), for vision tasks. They…
Oren Bochman
Monday, January 13, 2025

Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning
The announcement of the GPT-5 strawberry model has sparked a lot of interest in this paper which seems to be the theory behind Open.ai’s new model.
Oren Bochman
Monday, January 13, 2025

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
in (Bansal et al. 2024) the authors consider the trade-offs between generating synthetic data using a stronger but more expensive (SE) model versus a weaker but cheaper (WC)…
Oren Bochman
Monday, January 13, 2025

TheoremLlama An End-To-End Framework to Train a General-Purpose Large Language Model to Become a Lean4 Expert
Proving mathematical theorems using computer-verifiable formal languages like Lean significantly impacts mathematical reasoning. One approach to formal theorem proving…
Oren Bochman
Monday, January 13, 2025
 
Tree Attention: Topology-Aware Decoding for Long-Context Attention on GPU Clusters
in (Shyam et al. 2024) the authors propose a new algorithm for parallelizing attention computation across multiple GPUs. This enables cross-device decoding to be performed…
Oren Bochman
Monday, January 13, 2025
 
2BP: 2-Stage Backpropagation
in (Shyam et al. 2024) the authors …
Oren Bochman
Monday, January 13, 2025

Why bother reviewing papers?
Why bother reviewing papers?
Oren Bochman
Monday, January 13, 2025
 
Dependency Parsing using Deep Learning
Remark 2. Explina the two approches to parsing
Oren Bochman
Monday, January 13, 2025

Multi-column Deep Neural Networks for Image Classification
In (Cireşan, Meier, and Schmidhuber 2012) titled “Multi-column Deep Neural Networks for Image Classification”, the authors, Dan Cireşan, Ueli Meier, Juergen Schmidhuber…
Oren Bochman
Monday, January 13, 2025
 
Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors
In (Hinton et al. 2012) titled “Improving Neural Networks by Preventing Co-Adaptation of Feature Detectors”, the authors, Hinton, Geoffrey E., Nitish Srivastava, Alex…
Oren Bochman
Monday, January 13, 2025
 
ImageNet Classification with Deep Convolutional Neural Networks
(Krizhevsky, Sutskever, and Hinton 2012) is a seminal paper in the field of deep learning. It introduced the AlexNet architecture, which won the ImageNet Large Scale Visual…
Oren Bochman
Monday, January 13, 2025

Handwriting beautification using token means
In (Zitnick 2013) the author shows how we can use a model for beautifying handwriting. The problem raised is that there is lots of variation in handwriting for a single…
Oren Bochman
Monday, January 13, 2025

NIN — Network in Network
In (Lin, Chen, and Yan 2014) the authors, Lin, Min, Qiang Chen, and Shuicheng Yan, of this paper titled “Network in Network” paper came up with a way of connencting somee…
Oren Bochman
Monday, January 13, 2025

Dropout: A Simple Way to Prevent Neural Networks from Overfitting
In (Srivastava et al. 2014) the authors, present a novel regularization technique for deep neural networks called “dropout.” The key idea behind dropout is to randomly drop…
Oren Bochman
Monday, January 13, 2025

On Learning To Become a Successful Loser
I tracked this paper due to it being highlighted in (Skyrms 2010) as the source of a model that learns a signaling systems faster. I got me started with the loss domain. I…
Oren Bochman
Thursday, January 2, 2025

Emergent Communication of Generalizations
I think this is an amazing paper. I read it critically and made copious notes to see what I could learn from it. The paper point out some limitations of Lewis referential…
Oren Bochman
Wednesday, October 9, 2024

Evolutionary dynamics of Lewis signaling games: signaling systems vs. partial pooling
(Huttegger et al. 2010)
Oren Bochman
Tuesday, October 8, 2024
 
Why Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries
This paper, “Why Overfitting Isn’t Always Bad: Retrofitting Cross-Lingual Word Embeddings to Dictionaries,” challenges the traditional view that overfitting is inherently…
Oren Bochman
Tuesday, June 11, 2024
 
sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings
Sense2Vec (Trask, Michalak, and Liu 2015) is an interesting deep learning model based on word2vec that can learn more interesting and detailed word vectors from large…
Oren Bochman
Sunday, June 26, 2022

figure 1

Variational Inference with Normalizing Flows
The choice of approximate posterior distribution is one of the core problems in variational inference. Most applications of variational inference employ simple families of…
Oren Bochman
Sunday, June 26, 2022

Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
This Nips 1988 paper is about simplifying neural networks by removing redundant units. The authors’ approach is systematically identifying and removing redundant or…
Oren Bochman
Wednesday, June 22, 2022
 
Simplifying Neural Networks by soft weight sharing
This paper was mentioned in Geoffrey Hinton’s Coursera course as a way to simplify neural networks.
Oren Bochman
Wednesday, June 22, 2022

architecture

VGGNet: Very Deep Convolutional Networks for Large-Scale Image Recognition
In this paper (Simonyan and Zisserman 2015) the authors, Karen Simonyan and Andrew Zisserman from the Visual Geometry Group at Oxford, investigated the effect of increasing…
Oren Bochman
Thursday, December 10, 2015
No matching items