Oren Bochman’s Blog

Published

Friday, December 20, 2024

 
Deep Neural Networks - Notes From Hinton’s Course
My thoughts are that we should be able to do better than this version of dropout. - Shortcoming: - Dropout on units can render the net very poor. - Drop out slows training…
Oren Bochman
Sunday, August 6, 2017
 
Deep Neural Networks - Notes for Lesson 1
Notes for Deep learning focusing on the basics
Oren Bochman
Saturday, July 1, 2017

Deep Neural Networks - Notes for Lesson 10
This module we look at why it helps to combine multiple NN to improve generalization
Oren Bochman
Sunday, October 1, 2017
 
Deep Neural Networks - Notes for Lesson 11
This module deals with Boltzmann machine learning
Oren Bochman
Saturday, October 21, 2017
 
Deep Neural Networks - Notes for Lesson 12
This module deals with Boltzmann machine learning
Oren Bochman
Wednesday, November 1, 2017

Sigmoid belief net

Deep Neural Networks - Notes for Lesson 13
The ups and downs of backpropagation
Oren Bochman
Saturday, November 11, 2017

RDBM Combo

Deep Neural Networks - Notes for Lesson 14
Notes for Deep learning focusing on the basics
Oren Bochman
Tuesday, November 21, 2017
 
Deep Neural Networks - Notes for Lesson 15
Notes for Deep learning focusing on the basics
Oren Bochman
Friday, December 1, 2017

Combining RBMs into a DBM

Deep Neural Networks - Notes for Lesson 16
Notes for Deep learning focusing on the basics
Oren Bochman
Sunday, December 10, 2017

Feed-forward neural networks

Deep Neural Networks - Notes for Lesson 2
Notes for Deep learning focusing on Perceptrons
Oren Bochman
Sunday, July 16, 2017

the true weights

Deep Neural Networks - Notes for Lesson 3
Notes on Deep leaning and ML from Neural Networks for Machine Learning by Geoffrey Hintonon on Coursera
Oren Bochman
Tuesday, August 1, 2017

relational information

Deep Neural Networks - Notes for Lesson 4
Notes on Deep leaning and ML from Neural Networks for Machine Learning by Geoffrey Hinton on Coursera
Oren Bochman
Thursday, August 10, 2017

Deep Neural Networks - Notes for Lesson 5
Notes on Deep leaning and ML from Neural Networks for Machine Learning by Geoffrey Hintonon on Coursera
Oren Bochman
Thursday, August 17, 2017
 
Deep Neural Networks - Notes for Lesson 6
We delve into mini-batch gradient descent as well as discuss adaptive learning rates.
Oren Bochman
Wednesday, August 23, 2017

Memoryless models

Deep Neural Networks - Notes for Lesson 7
This module explores training recurrent neural networks
Oren Bochman
Friday, September 1, 2017

Memoryless models

Deep Neural Networks - Notes for Lesson 7a
Modeling sequences — A brief overview
Oren Bochman
Saturday, September 2, 2017

Deep Neural Networks - Notes for Lesson 7b
Training RNNs with back propagation
Oren Bochman
Sunday, September 3, 2017

toy RNN problem

Deep Neural Networks - Notes for Lesson 7c
A toy example of training an RNN
Oren Bochman
Monday, September 4, 2017

Deep Neural Networks - Notes for Lesson 7d
Why it is difficult to train an RNN
Oren Bochman
Tuesday, September 5, 2017

Deep Neural Networks - Notes for Lesson 7e
Training RNNs with back propagation
Oren Bochman
Wednesday, September 6, 2017

Curvature Matrices

Deep Neural Networks - Notes for Lesson 8
We continue our look at recurrent neural networks
Oren Bochman
Monday, September 11, 2017

Deep Neural Networks - Notes for Lesson 9
We discuss strategies to make neural networks generalize better
Oren Bochman
Thursday, September 21, 2017

Drosophila melanogaster Proboscis

Deep Neural Networks - Notes for lecture 1a
Why do we need machine learning?
Oren Bochman
Sunday, July 2, 2017
 
Deep Neural Networks - Notes for lecture 1b
Notes for Deep learning focusing on why do we need machine learning?
Oren Bochman
Monday, July 3, 2017

linear activation function

Deep Neural Networks - Notes for lecture 1c
Notes for Deep learning focusing on simple models of neurons
Oren Bochman
Tuesday, July 4, 2017
 
Deep Neural Networks - Notes for lecture 1d
Notes for Deep learning focusing on a simple example of learning learning?
Oren Bochman
Wednesday, July 5, 2017
 
Deep Neural Networks - Notes for lecture 1e
Notes for Deep learning focusing on the three types of learning
Oren Bochman
Thursday, July 6, 2017

Deep Neural Networks - Notes for lecture 2a
Notes for Deep learning focusing on types of neural network architectures
Oren Bochman
Monday, July 17, 2017

The standard Perceptron architecture

Deep Neural Networks - Notes for lecture 2b
Notes for Deep learning focusing on Perceptrons, the first generation of neural networks.
Oren Bochman
Tuesday, July 18, 2017

Weight-space

Deep Neural Networks - Notes for lecture 2c
Notes for Deep learning focusing on a geometrical view of perceptrons
Oren Bochman
Wednesday, July 19, 2017

idea of the proof

Deep Neural Networks - Notes for lecture 2d
Notes for Deep learning focusing on why the learning works?
Oren Bochman
Thursday, July 20, 2017

Impossible to satisfy

Deep Neural Networks - Notes for lecture 2e
Notes for Deep learning focusing What Perceptrons can not do
Oren Bochman
Friday, July 21, 2017

the true weights

Deep Neural Networks - Notes for lecture 3a
Focusing on Learning the weights of a linear neuron
Oren Bochman
Wednesday, August 2, 2017

error surface of a linear neuron

Deep Neural Networks - Notes for lecture 3b
The error surface for a linear neuron
Oren Bochman
Thursday, August 3, 2017

logistic activation function

Deep Neural Networks - Notes for lecture 3c
Learning the weights of a logistic output neuron
Oren Bochman
Friday, August 4, 2017

back proogations of errors

Deep Neural Networks - Notes for lecture 3d
The back-propagation algorithm
Oren Bochman
Saturday, August 5, 2017

overfitting

Deep Neural Networks - Notes for lecture 3e
Using the derivatives computed by backpropagation
Oren Bochman
Sunday, August 6, 2017

relational information

Deep Neural Networks - Notes for lecture 4a
Learning to predict the next word
Oren Bochman
Friday, August 11, 2017
 
Deep Neural Networks - Notes for lecture 4b
A brief diversion into cognitive science
Oren Bochman
Saturday, August 12, 2017

Deep Neural Networks - Notes for lecture 4c
Another diversion — The Softmax output function
Oren Bochman
Sunday, August 13, 2017
 
Deep Neural Networks - Notes for lecture 4d
Neuro-probabilistic language models
Oren Bochman
Monday, August 14, 2017

A serial architecture

Deep Neural Networks - Notes for lecture 4e
Ways to deal with the large number of possible outputs
Oren Bochman
Tuesday, August 15, 2017

Deep Neural Networks - Notes for lecture 5a
Why object recognition is difficult
Oren Bochman
Friday, August 18, 2017
 
Deep Neural Networks - Notes for lecture 5b
Ways to achieve viewpoint invariance
Oren Bochman
Saturday, August 19, 2017

Deep Neural Networks - Notes for lecture 5c
Convolutional neural networks for hand-written digit recognition
Oren Bochman
Sunday, August 20, 2017

Examples from the test set

Deep Neural Networks - Notes for lecture 5d
Why object recognition is difficult
Oren Bochman
Monday, August 21, 2017

error surface

Deep Neural Networks - Notes for lecture 6a
Overview of mini-batch gradient descent
Oren Bochman
Thursday, August 24, 2017

Deep Neural Networks - Notes for lecture 6b
A bag of tricks for mini-batch gradient descent
Oren Bochman
Friday, August 25, 2017

Momentum intution

Deep Neural Networks - Notes for lecture 6c
The momentum method
Oren Bochman
Saturday, August 26, 2017

Deep Neural Networks - Notes for lecture 6d
Adaptive learning rates for each connection
Oren Bochman
Sunday, August 27, 2017

some learning rate algorithms

Deep Neural Networks - Notes for lecture 6e
rmsprop - divide the gradient by a running average of its recent magnitude
Oren Bochman
Monday, August 28, 2017
 
Deep Neural Networks - Some Questions
Unresolved questions on Deep learning.
Oren Bochman
Thursday, December 21, 2017

ensemble architecture

Deep Neural Networks — Readings I for Lesson 10
Review & summary of — Evaluation of Adaptive Mixtures of Competing Experts
Oren Bochman
Monday, October 2, 2017
 
Deep Neural Networks — Readings II for Lesson 10
Review & summary of — Improving neural networks by preventing co-adaptation of feature detectors
Oren Bochman
Tuesday, October 3, 2017
 
Glossary of terms for Deep Neural Networks
Glossary of terms in Deep leaning and ML from Neural Networks for Machine Learning by Geoffrey Hintonon on Coursera
Oren Bochman
Sunday, August 6, 2017
No matching items