Introduction to neural networks

Timetable

Lecture 1.5 h per week

Laboratory (Exercise) 1.5 h per week

Module aim (aims)

The aim of the course is to provide basic theories of
(1) neural networks
(especially deep learning) as the basis for today's artificial intelligence
and
(2) quantum computation as a related theme.

Pre-requisites in terms of knowledge, skills and social competences (where relevant)

Very basic knowledge of linear algebra, differential calculus, probability. A bit of fmiliarity
with programming languages like Python will be helpful. 

Syllabus

Week 1. McCulloch-Pitts model of neurons. Linear separability.

Weak 2. Perceptron and learning. Theorem of Minsky and Papert.

Weak 3. Deep neural network (model of Rosenblatt).

Weak 4. Linear model of association (model of Nakano, Nagumo, Amari, Kohonen).                                        Hebbian rule. Nonlinear model of association.
Attention mechanism. Transformer.

Weak 5. Gradient descent. Stochastic gradient descent.

Weak 6. Back propagation. Deep learning. Autoencoder.

Weak 7. Convolutional Neural Network (CNN). Residual Neural Networks (ResNets).

Weak 8. Reinforcement learning. Monte Carlo Tree Search (MCTS).

Weak 9. AlphaGo.

Weak 10. Transformer. Recurrent Neural Network (RNN). Application to natural language processing.

Weak 11. Nakano-Nagumo-Amari-Kohonen-Hopfield network. Application to traveling salesperson problem (TSP).

Weak 12. Boltzmann machine and learning. Deep Boltzmann machine. Deep belief network.

Weak 13. Qubit. Entanglement and tensor product.

Weak 14. Discrete Fourier transform. Shor's algorithm for factorization.

Weak 15. Quantum annealing.

Reading list

1. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, The MIT Press, 2017.
2. www.deeplearningbook.org
3. B. Ramsundar, R. Bosagh Zadeh, TensorFlow for Deep Learning: From Linear Regression to Reinforcement Learning, 1st edition, O'Reilly Media, 2018.