It is tempting to base (eco-)toxicological assay evaluation solely on statistical significance tests. The approach is stringent, objective and facilitates binary decisions. However, tests according to null hypothesis statistical testing (NHST) are thought experiments that rely heavily on assumptions. The generic and unreflected application of statistical tests has been called “mindless” by Gigerenzer. While statistical tests have an appropriate application domain, the present work investigates how unreflected testing may affect toxicological assessments. Dunnett multiple-comparison and Williams trend testing and their compatibility intervals are compared with dose-response-modelling in case studies, where data do not follow textbook behavior, nor behave as expected from a toxicological point of view. In such cases, toxicological assessments based only on p-values may be biased and biological evaluations based on plausibility may be prioritized. If confidence in a negative assay outcome cannot be established, further data may be needed for a robust toxicological assessment.