We start by demonstrating that an elementary learning task—learning a linear filter
from training data by means of regression—can be solved very efficiently for feature
spaces of very high dimensionality. In a second step, firstly, acknowledging that such
high-dimensional learning tasks typically benefit from some form of regularization and,
secondly, arguing that the problem of scale has not been taken care of in a very satis-
factory manner, we come to a combined resolution of both of these shortcomings by
proposing a technique that we coin scale regularization. This regularization problem can
also be solved relatively efficient. All in all, the idea is to properly control the scale of a
trained filter, which we solve by introducing a specific regularization term into the overall
objective function. We demonstrate, on an artificial filter learning problem, the capabil-
ities of our basic filter. In particular, we demonstrate that it clearly outperforms the de
facto standard Tikhonov regularization, which is the one employed in ridge regression or
Wiener filtering.