Increasing awareness of the ability to transform data into knowledge has steered more focus on data science within the educational system as well as the development of machine learning methods capable of handling complex problems with minimal or no human interaction. In principle, this raises the question on where human-computer interaction is superior in building good models in contrast to fully automated algorithms. In this study, we investigated modeling performance by using bachelor students, master students, and a fully automated procedure on three near-infrared (NIR) calibration tasks of increasing complexity. From a total of 107 student and +5000 automated models, it is evident that simple calibration tasks can be automated to achieve similar or better performance, whereas for the more complicated tasks, the human-computer interaction is superior. Indeed, teaching data science and chemometrics should focus on tools for fundamental data understanding and emphasize the use of domain knowledge and critical thinking in the analysis of data.