Mathfoundationrationr / Book-Mathematical-of – Freincement-Full-Learning: This is the homepage in a new book entitled “Mathematical Foundation of Learning to Strengthen.”

This book has received 5,000+ stars! Pleased it will help many readers.

My English Open Curse online now. You can click the upper number or the link here to jump on our YouTube channel. You can also click the following links to point to specific lecture videos.

You are hospitable to check English videos to see if they can help!

This book refers to give a mathematics but friendly Introduction to basic concepts, major problems, and classic build algorithms. Some important features of this book are highlighted as follows.

• The book introduces learning to learn from a mathematical point. Hope readers don’t just know the way an algorithm but also understands why it is designed in the first place and why it is effective to work.

• The depths of mathematics is well controlled by an adequate level. Math is also presented in a well-designed way to make sure the book is friendly. Readers can choose to read materials presented in gray boxes according to their interest.

• Many meaningful examples are given to help readers understand the subjects better. All the examples of this book are based on a world-like plan, which can be easily understood and helpful for the description of concepts and algorithms.

• If an algorithm is introduced, the book refers to separate the core idea from the complications that can be distracted. In this way, readers can understand the core idea of ​​an algorithm.

• The book’s contents are organizations. Each chapter is built based on the previous chapter and puts a necessary foundation for the next.

Subjects addressed to the book are shown in the number below. This book contains ten chapters, which can be classified into two parts: the first part is about basic tools, and the second part about algorithms. Ten chapters are very much better. Overall, the first chapters should study before the latter.

This book is designed for senior undergraduate students, graduate students, researchers, and trainers who are interested in learning strengthening.

No readers need a background to build up to learn because it starts by introducing the most basic concepts. If the reader has some backgrounds to come up, I believe that the book helps them understand some subjects more deeply or give different views.

This book is, however, requires the reader with the knowledge of probability theory and linear algebra. Some basics in required mathematics are also included by the appendix in this book.

By combining the book with videos in my lectures, I believe you can be better.

• Chinese Likit Videos: You can check the Visitor Channel or the YouTube Channel. Lecture videos received 1,300,000+ views Until February 2025 on the Internet and received very good feedback!

• English Lesson Videos: English lecture videos are uploaded to YouTube. Please refer to links and details on another side of this document.

You can see my information on my homepage https://www.shiyuzhao.net/ (Googuleite) and my research group website https://shiyuzhao.westlake.edu.cn

I have taught a graduate level course to come up with the reinforcement since 2019. Teaching, I prepared this book as lecture notes for my students.

I am honest this book helps readers smoothly enter the exciting field of learning to strengthen.

@book{zhao2025RLBook,
  title={Mathematical Foundations of Reinforcement Learning},
  author={S. Zhao},
  year={2025},
  publisher={Springer Nature Press and Tsinghua University Press}
}

Many enthusiastic readers send me the source code or note they have developed in their study of this book. If you create any course-based materials, you are accepted in writing an email. I’m glad to share here with links here and hope they can help other readers. I have to mean I didn’t confirm the code. If you have any questions, you can contact developers.

CODE

Python:

R

C ++:

Study Notes

English:

Chinese:

There are also many other notes made by many other internet readers. I can’t do them all over here. You are hospitable to recommend me if you see a good one.

Videos of Basiibi made based on my course

(Dec 2024) 4,000+ stars
This book has received 4,000+ stars! Pleased it will help many readers.

(October 2024) Cover in the book

The design of the book cover is ended. The book is officially published in Springer early next year. It was published in Tsinghua University Press.

(Sep 2024) Minusa update before printing in Springer

I have changed some small places that can not find readers. This is the last version before printing in Springer.

(Aug 2024) 3000 stars and more code

The book received 3000+ stars, which was a great success to me. Thanks for everyone. Hope this helped you.

I also received additional code implementation from enthusiastic readers. For example, this GitHub Page Python enforcement is given to almost all examples of my book. On one side, I’m glad to see that. On the other hand, I’m a little worried that my students in my offline office can use the code to do their homework :-). Overall, I am happy because it indicates that the book and open course will really help readers; If not, they do not suffer from improving the code themselves 🙂

(Jun 2024) Minor update before print

This is the fourth version of the draft of the book. This can be the end before the book is officially published. Specifically, if proofread the book manuscript, I see some small issues. Along with some reported by enthusiastic readers, they are re-revised in this version.

(Apr 2024) Code for World World

We’ve added the code for the Grid-World around my book. Interested readers can develop and test their own algorithms in this environment. Given both versions of Python and Matlab.

Please note that we do not provide the code to all algorithms involved in the book. That’s because they are homework for Offline Teaching Students: Students should develop their own algorithms using the given environment. However, there are third party implementations in some algorithms. Interested readers can check links to the home page book.

I need to thank my PHD students, Iize Mi and Jianan Li, who also helps teach my education. They have contributed closely to the code.

You are happy to give any feedback about code like bugs if seen.

(Mar 2024) 2k star

The book has received 2k star. I also received many positive book evaluations from many readers. It is very pleased that it helps.

(Mar 2024) Minor update

The third version of the draft of the book is online now.

As compared to the second version, the third version is enhanced by the definition that some smaller tysos are corrected. Here, I want to thank readers who sent me their feedback.

(Sep 2023) 1000+ stars

The book receives 1000+ stars! Thanks for everyone!

(Aug 2023) Major Update – Second version

The second version of the draft of the book is online now !!

Compared with the first version, online one year ago, the second version is enhanced in different ways. For example, we repeated most of the numbers, also organized some content to clear them, correcting some typos, not included in the first version.

I put the first draft of this book online on August 2022. Now, I received important feedback from many readers around the world. I want to express my gratitude to readers.

(Nov 2022) will be further published

This book will be published Springer Nature and Tsinghua University Press. It can be printed in the second half of 2023.

I received some comments and suggestions about this book from some readers. Thank you so much, and I appreciate it. I am still collecting feedback and maybe redeem the draft again for several months. Your feedback can make this book more helpful to other readers!

(October 2022) Lesson Notes and Vidoes

the Slides in lectures Uploaded to the “lecture lecture.” folder

the Lecture videos (in Chinese) is online. Please check our Blaibili Channel https://space.bilibili.com/2044042934 or the youtube channel https://www.youtube.com/channel/ucztgts5yyinv8x3pjlgment

(Aug 2022) first draft

The first draft of the book is online.