Stochastic Processes and Optimization in Machine Learning

Postgraduate Program “Data Science and Machine Learning”

Spring Semester 2026: Room 001, Tuesday 8:45 – 10:30 and laboratory exercise 10:30 – 11:30 (Starting Tuesday 17/2/2026)

Instructor: Professor Emeritus Vasilis Maglaris

Support: Dr Eng. Mary Grammatikou, PhD Candidates Dimitris Pantazatos and Nikos Bazotis

Course Outline

Basic definitions of Machine Learning (ML) & Artificial Intelligence (AI). Training, Validation & Testing Datasets. Review of Opimization Algorithms in ML: Supervised, Unsupervised, Reinforcement Learning. Discriminative & Generative Models, LLMs. Linear & Logistic Regression

Neural Networks, Hebb’s rule, parameter setting via Supervised Learning, Rosenblatt’s Perceptron,Back-Propagation Algorithm

Unsupervised Learning: K-Means Clustering, Principal Components Analysis (PCA), Self-Organizing Maps (SOM), Autoencoders

Machine Learning & Statistical Mechanics Concepts: Markov Chains & state classification. Chapman – Kolmogorov equations, asymptotic behavior, irreducibility, recurrence, ergodicity, invariant probabilities

Markov Chain Monte Carlo (MCMC) methods, Metropolis-Hastings algorithm, Simulated Annealing, Gibbs Sampling. Generative models, Boltzmann Machine, Restricted Boltzmann Machine (RBM), Deep Belief Networks (DBN)

Reinforcement Learning and Dynamic Programming: Markov Decision Processes, Bellman’s Optimality Criterion, optimization algorithms – Value Iteration & Policy Improvement. Approximate dynamic programming methods, Temporal-Difference (TD) & Q-Learning

Reinforcement Learning & Internet Routing: Bellman – Ford algorithm, Border Gateway Protocols (BGP)

Kernel Classification Methods: Pattern Separability – Cover’s Theorem. Applications in Radial-Basis Function (RBF) Networks, Hybrid Learning and Support Vector Machines (SVM)

Non-parametric Classifiers, based on K-Nearest Neighbors (KNN) labelled classification

Statistical evaluation of binary classification, Confusion Matrix, Receiver Operating Characteristics (ROC) & Area Under the Curve (AUC). Parametric Probabilistic Classification, Bayes rule, approximate methods, Naive Bayes Classifiers

Decision Trees: Binary Splitting, Classification and Regression Trees (CART), Gini Index, Random Forests, Bagging (Bootstrap & aggregating) algorithms

Recurrent Neural Nets (RNN) & Neurodynamic Models: Associative Memory – Content Addressable Memory (CAM), Hopfield Networks. Sequence modeling RNNs based on time/character series & speech processing datasets, Long-Short Term Memory (LSTM) nets

Explainability in ML/AI – XAI (eXplainable AI): Definitions, Intrinsic & Model Agnostic XAI methods, PI (Permutation Feature Importance), SHAP (Shapley Additive exPlanations), LIME (Local Interpretable Model Agnostic Explanation)

Lab Exercises:

Room 002 of the ECE School (New Buildings) using Python tools – libraries.

Suggested Bibliography:

1. Simon Haykin, “Neural Networks and Learning Machines”, Third Edition, Pearson Education, 2009
2. Bernhard Mehlig, “Machine learning with neural networks”, Cambridge Univ. Press 2021 https://arxiv.org/pdf/1901.05639.pdf  
3. Kevin P. Murphy, “Machine Learning: A Probabilistic Perspective”, MIT Press, 2012
4. Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, MIT Press, 2016 https://www.deeplearningbook.org/ 
5. Daniel Jurafsky and James H. Martin, “Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition”, Third Edition draft, https://web.stanford.edu/~jurafsky/slp3/, 2025
6. Andrew Ng, “CS229 Lecture Notes“, Stanford University, Fall 2018 https://see.stanford.edu/materials/aimlcs229/cs229-notes1.pdf
7. James Gareth, Daniela Witten, Trevor Hastie and Robert Tibshirani, “An Introduction to Statistical Learning with Applications in R“, Second Edition, Springer 2021, https://hastie.su.domains/ISLR2/ISLRv2_website.pdf
8. Richard Sutton and Andrew Barto, “Reinforcement Learning: An Introduction“, Second Edition, MIT Press, 2018
9. Dimitri P. Bertsekas and John Tsitsiklis, “Neuro-Dynamic Programming”, Athena Scientific, Belmont MA, 1996
10. Christopher Bishop, “Pattern Recognition and Machine Learning”, Springer 2006, https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf 
11. Tom Mitchell, “Machine Learning”, McGraw Hill 1997 http://www.cs.cmu.edu/~tom/mlbook.html
12. Charu C. Aggarwal, “Outlier Analysis”, Springer 2013 https://link.springer.com/chapter/10.1007/978-1-4614-6396-2_1
13. Christoph Molnar, “Interpretable Machine Learning“, Second Edition, Christoph Molnar, Munich, 2022 https://christophm.github.io/interpretable-ml-book/
14. Leonida Gianfagna and Antonio Di Cecco, “Explainable AI with Python”, Springer 2021 https://link.springer.com/book/10.1007/978-3-030-68640-6
15. Robert Hogg, Joseph McKean and Allen Craig, “Introduction to Mathematical Statistics“, Eighth Edition, Pearson Education, 2020
16. Frank Kelly, “Reversibility and Stochastic Networks“, Wiley, 1979 http://www.statslab.cam.ac.uk/~frank/BOOKS/book/whole.pdf
17. Sheldon Ross, “Applied Probability Models with Optimization Applications“, Dover, 1992
18. “Data Engineering Foundations: Core Techniques for Data Analysis with Pandas, NumPy, and Scikit-Learn“, Cuantum Technologies, 2025
19. “Feature Engineering for Modern Machine Learning with Scikit-Learn: Advanced Data Science and Practical Applications“, Cuantum Technologies, 2025
20. Chip Huyen, “AI Engineering: Building Applications with Foundation Models“, O’Reilly, 2025

Lectures 2026

Date	Lecture	Presentation File(s)
17/02/2026	Stochastic Processes and Optimization in Machine Learning: i) Machine Learning (ML) & Artificial Intelligence (AI), ii) Definitions of Datasets, iii) Discriminative & Generative AI Models, iv) Supervised Learning, Linear & Logistic Regression – Video	File
24/02/2026	Stochastic Processes and Optimization in Machine Learning: i) Overview of Neural Networks, ii) Hebb’s Rule, iii) Weight Tuning via Supervised Learning, iv) Back-Propagation Algorithm – Video	File
03/03/2026	Stochastic Processes and Optimization in Machine Learning: i) Unsupervised Learning, ii) 𝑲-Means Clustering, iii) Principal Component Analysis (PCA), iv) Autoencoders – Video	File
10/03/2026	Stochastic Processes and Optimization in Machine Learning: i) Stochastic Methods Rooted in Statistical Mechanics, ii) Markov Chains: Definitions of States & Transitions, iii) Stationarity & Ergodic Probabilities, Iv) Global & Detailed Balance Equations – Video	File
17/03/2026	Random Sampling Methods: i) Metropolis-Hastings Algorithm, ii) Gibbs Sampling, iii) Markov Random Fields, Ising Model, iv) Simulated Annealing – Video	File
24/03/2026	Stochastic Processes and Optimization in Machine Learning: i) Boltzmann Machine, ii) Gibbs Sampling & Bayesian Statistics, iii) Boltzmann Learning Rule, Maximum Likelihood Principle, iv) Generative Models, Generative Adversarial Network (GAN) – Video	File