In analytical chemistry, multivariate calibration is applied when substituting a time-consuming reference measurement (based on e.g. chromatography) with a high-throughput measurement (based on e.g. vibrational spectroscopy). An average error term, of the response variable, is often used to evaluate the performance of a calibration model. However, indirect relationships, between the response and explanatory variables, may be used for calibration. In such cases, model validity cannot necessarily be determined solely by the average error term. One should also consider the use of the models, as well as the validity of the indirect relationships in future samples. If the analyte of interest is partly quantified from signals of interfering compounds, then these interfering compounds will play a hidden role in the calibration. This hidden role may affect future use of the calibration model as strong covariance relationships between analyte estimates and interfering compounds may be imposed. Hence, such model cannot detect changes in the relationship between the analyte and interfering compounds. The problem is called the cage of covariance. This paper discusses the concept cage of covariance and possible consequences of applying models exposed to this issue.