Targeted at graduate students, researchers and practitioners in the field of science and engineering, this book gives a self-contained introduction to a measure-theoretic framework in laying out the definitions and basic concepts of random variables and stochastic diffusion processes. It then continues to weave into a framework of several practical tools and applications involving stochastic dynamical systems. These include tools for the numerical integration of such dynamical systems, nonlinear stochastic filtering and generalized Bayesian update theories for solving inverse problems and a new stochastic search technique for treating a broad class of non-convex optimization problems. MATLAB codes for all the applications are uploaded on the companion website.