Background: Students' knowledge of scientific principles of evolution is often inadequate, despite its recognized importance for understanding biology. Moreover, difficulties associated with underlying abstract concepts such as randomness and probability can hinder successful learning of evolutionary concepts. Studies show that visualizations, particularly simulations together with appropriate instructional support, facilitate the learning of abstract concepts. Therefore, we have developed interactive, web-based simulation software called EvoSketch in efforts to help learners grasp the nature and importance of random and probabilistic processes in evolutionary contexts. We applied EvoSketch in an intervention study comparing four self-directed study conditions: learning with EvoSketch (1) alone, (2) combined with interpretative support, (3) combined with reflective support, and (4) using texts about randomness and probability instead of EvoSketch. All conditions received no support from any instructors. Knowledge about evolution as well as randomness and probability in the context of evolution, time-on-task, and perceived cognitive load were measured. A sample of 269 German secondary school students (M _age = 15.6 years, SD = 0.6 years) participated in the study. Results: Learners using EvoSketch without additional support obtained higher follow-up test scores regarding their knowledge of randomness and probability than those using the text-based approach. However, use of the simulations together with given instructional support (interpretative or reflective) did not increase students' performance, relative to the text-based approach. In addition, no significant between-intervention differences were found concerning the knowledge of evolution, while significant differences between the groups were detected concerning students' perceived cognitive load and time-on-task. Conclusions: From our findings, we conclude that EvoSketch seems to have a very small positive effect on students' understanding of randomness and probability. Contrary to our expectations, additional self-directed instructional support did not improve students' understanding, probably because it was not necessary to understand EvoSketch simulations. When using EvoSketch in the classroom, we recommend increasing the intervention timeframe to several sessions and a variety of evolutionary examples for which EvoSketch serves as an underlying framework.