# Widrow-Hoff

The WIDROW-HOFF (a.k.a. LEAST MEAN SQUARED (LMS) ) algorithm is the most well-understood and principled approach to training a linear system to minimize this squared error loss [Widrow60] . It does this by making a small move (scaled by the parameter $\eta$) in the direction of the gradient of error. This gradient is defined exactly by the derivative of Equation (FOAref) with respect to the document vector: \Delta \mathbf{q} = - 2 \eta (\mathbf{q} \cdot \mathbf{d} - R_{\mathbf{d}}) \mathbf{d}

It is also important to remember that changes made to a single document in response to a single query can make no guarantees about improved performance with respect to other documents and other queries. For example, two documents might both be moved closer to a query (as proposed by Brauen/Roccio) while their relative rankings are not changed at all!

FOA © R. K. Belew - 00-09-21