Table of content:

Micro-Syllabus of Unit 4 : The Least-Mean-Square Algorithm (5 Hrs.) 5 marks fix

Introduction, Filtering Structure of the LMS Algorithm, Unconstrained Optimization: A Review, The Wiener Filter, The Least-Mean-Square Algorithm, Markov Model Portraying the Deviation of the LMS Algorithm from the Wiener Filter, The Langevin Equation: Characterization of Brownian Motion, Kushner‟s Direct-Averaging Method, Statistical LMS Learning Theory for Small Learning-Rate Parameter, Virtues and Limitations of the LMS Algorithm, Learning-Rate Annealing Schedules

🗒️Note:→

# Introduction to Least-Mean-Square Algorithm :

              The Least Mean Square algorithm was developed by Widrow and Hoff in 1960. It was the first linear adaptive filtering algorithm (inspired by the perceptron) for solving problems such as prediction.

LMS algorithm uses the estimates of the gradient vector from the available data.

LMS incorporates an iterative procedure that makes successive corrections to the weight vector in the direction of the negative of the gradient vector which eventually leads to the minimum mean square error.

Compared to other algorithms, LMS algorithm is relatively simple to implement, computationally efficient, and robust with respect to external disturbances.