Elements of Deep Learning

This textbook offers a comprehensive introduction to deep learning and neural networks, integrating core foundations with the latest advances. It begins with essential machine learning concepts and classic neural network architectures before progressing through convolutional models, backpropagation, regularization, generalization theory, PAC learning, and Boltzmann machines. Advanced chapters cover sequence models including recurrent networks, LSTMs, attention, Transformers, state-space models, and large language models alongside deep generative approaches such as VAEs, GANs, and diffusion models. Emerging topics include graph neural networks, self-supervised learning, metric learning, reinforcement learning, meta-learning, model compression, and knowledge distillation.

Balancing mathematical rigor with hands-on practice, Elements of Deep Learning emphasizes both theoretical depth and real-world application. Different theories are introduced with PyTorch-based code examples, helping readers to translate theory into implementation. Organized into five sectionsfundamentals, sequence models, generative models, emerging topics, and practicethe text provides a unified roadmap for mastering modern deep learning.

Designed for advanced undergraduates, graduate students, instructors, and professionals in engineering, computer science, mathematics, and related fields, this book serves both as a primary course text and a reliable reference. With minimal prerequisites in linear algebra and calculus, it offers accessible explanations while equipping readers with practical tools for applications in vision, language, signal processing, healthcare, and beyond.

Demystifies deep learning with clear, step-by-step explanations grounded in theory and practice Packed with practical tips, design insights, and PyTorch code to build real-world deep learning systems Ideal for researchers, professionals, and advanced students seeking depth, clarity, and lasting value

Autorentext

Benyamin Ghojogh received the B.Sc. degree in electrical engineering from the Amirkabir University of Technology, Tehran, Iran, in 2015, the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2017, and Ph.D. in electrical and computer engineering (in the area of pattern analysis and machine intelligence) from the University of Waterloo, Waterloo, ON, Canada, in 2021. He was a postdoctoral fellow, focusing on machine learning, at the University of Waterloo, in 2021. He is the co-author of Elements of Dimensionality Reduction and Manifold Learning , published by Springer. His research interests include machine learning, deep learning, dimensionality reduction, data science, and computer vision.

Ali Ghodsi is a Professor of Statistics and Computer Science at the University of Waterloo, Director of the Data Science Lab, and a Faculty Affiliate at the Vector Institute for Artificial Intelligence. His research focuses on the theoretical foundations and algorithmic development of machine learning and artificial intelligence, with applications in natural language processing, bioinformatics, and computer vision.

He is the co-author of Elements of Dimensionality Reduction and Manifold Learning (Springer). His widely viewed online lectures including a popular deep learning course make advanced AI topics accessible to a global audience.

Inhalt

Chapter 1: Introduction.- Part 1: Fundamentals of Deep Learning.- Chapter 2: Feed-forward Neural Network.- Chapter 3: Regularization.- Chapter 4: Convolutional Networks.- Chapter 5: Restricted Boltzmann Machine (RBM) and deep belief network.- Part 2: Sequence Modeling.- Chapter 6: RNN and LSTM.- Chapter 7: Attention Mechanism, Transformers, BERT, and GPT.- Chapter 8: Large Language Models.- Part 3: Generative Models.- Chapter 9: Variational Models.- Chapter 10: Generative Moment Matching.- Chapter 11: Generative Adversarial Networks.- Chapter 12: Diffusion Models.- Part 4: Emerging Topics in Deep Learning.- Chapter 13: Graph Neural Networks.- Chapter 14: Deep Reinforcement Learning.- Chapter 15: Few-shot Learning and Meta-learning.- Chapter 16: Network Compression.- Chapter 17: Federated Learning.- Chapter 18: Explainable AI.- Chapter 19: Self-supervised Learning.- Part 5: Theory of Neural Networks.- Chapter 20: Theory of Neural Networks.- Part 6: Deep Learning in Practice.- Chapter 21: Deep Learning Tuning.