Markov Decision Processes and Reinforcement Learning for Timely UAV-IoT Data Collection Applications

This book offers a structured exploration of how Markov Decision Processes (MDPs) and Deep Reinforcement Learning (DRL) can be used to model and optimize UAV-assisted Internet of Things (IoT) networks, with a focus on minimizing the Age of Information (AoI) during data collection. Adopting a tutorial-style approach, it bridges theoretical models and practical algorithms for real-time decision-making in tasks like UAV trajectory planning, sensor transmission scheduling, and energy-efficient data gathering. Applications span precision agriculture, environmental monitoring, smart cities, and emergency response, showcasing the adaptability of DRL in UAV-based IoT systems. Designed as a foundational reference, it is ideal for researchers and engineers aiming to deepen their understanding of adaptive UAV planning across diverse IoT applications.

Introduces the concept of the Age of Information (AoI) Presents the MDP framework for various problems and offers their mathematical representations Discusses several algorithms designed to solve issues related to fresh data collection

Inhalt

Introduction to AoI in UAV-assisted Sensor and IoT Systems.- AoI aware UAV IoT Modeling using MDPs.- Reinforcement Learning and DRL for AoI aware UAV IoT.- Challenges and Future Considerations.