Incremental Risk Minimization Algorithm Incremental Regression with Polynomials ↑ Incremental (or on-line) learning regression is the process of adapting a model one example at a time without accumulating a batch of data. It has the advantages of allowing continuous adaptation to non-stationary environments, easily handling big data through stream processing, and a fixed low computation and memory demand. The easiest solution is to perform a gradient descent on a squared error metric with each new training example. But this solution does not work well for complex model structures. Especially, the influence of a non-linear transformation of the inputs through a fixed model structure has long been an open problem. During my PhD I worked on an approach which is able to deal with a broad class of non-linear model structures. Its emphasis is on minimizing the effect of local training examples on changes of the global model. Thus, it yields a robust behavior by preventing overfitting on sparse data as well as fatal forgetting.