Introduction to neural networks

Timetable

Lecture 1.5 h per week

Laboratory (Exercise) 1.5 per week

Module aim (aims)

The aim of the course is to provide basic theories of
(1) neural networks (especially deep learning) as the basis for today's artificial intelligence and
(2) quantum computation as a related theme.

Pre-requisites in terms of knowledge, skills and social competences (where relevant)

Basic knowledge of linear algebra, differential calculus, probability. Familiarity with programming
languages like Python, C++ and basic skills of algorithm implementation

Syllabus

Week 1. McCulloch-Pitts model of neurons. Linear separability.
Weak 2. Perceptron and learning. Theorem of Minsky and Papert.
Weak 3. Deep neural network (model of Rosenblatt).
Weak 4. Linear model of association. Hebbian rule.
Nonlinear model of association (model of Nakano-Nagumo-Amari-Kohonen). Attention mechanism. Transformer.

Weak 5. Gradient descent. Stochastic gradient descent.

Weak 6. Back propagation. Deep learning. Autoencoder.
Weak 7. Convolutional Neural Network (CNN).
Weak 8. Reinforcement learning. Monte Carlo Tree Search (MCTS).
Weak 9. AlphaGo.
Weak 10. Transformer. Recurrent Neural Network (RNN). Application to natural language processing.
Weak 11. Nakano-Nagumo-Amari-Kohonen-Hopfield network. Application to traveling salesperson problem (TSP).
Weak 12. Boltzmann machine and learning. Deep Boltzmann machine. Deep belief network.
Weak 13. Qubit. Entanglement and tensor product.
Weak 14. Discrete Fourier transform. Shor's algorithm for factorization.
Weak 15. Quantum annealing.

Reading list

1. I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, The MIT Press, 2017.
2. www.deeplearningbook.org.
3. B. Ramsundar, R. Bosagh Zadeh, TensorFlow for Deep Learning: From Linear Regression
to Reinforcement Learning, 1st edition, O'Reilly Media, 2018.