Ultra diffuse galaxies (UDGs) are a type of large low surface brightness (LSB) galaxies with particularly large effective radii (r(eff)> 1.5 kpc) that are now routinely studied in the Local (z<0.1) Universe. While they are found to be abundant in clusters, groups, and in the field, their formation mechanisms remain elusive and comprise an active topic of debate. New insights may be found by studying their counterparts at higher redshifts (z> 1.0), even though cosmological surface brightness dimming makes them particularly difficult to detect and study in this channel. In this work, we use the deepest Hubble Space Telescope (HST) imaging stacks of z> 1 clusters, namely, SPT-CL J2106-5844 and MOO J1014+0038. These two clusters, at z=1.13 and z=1.23, respectively, were monitored as part of the HST See-Change programme. In making a comparison with the Hubble Extreme Deep Field as the reference field, we find statistical over-densities of large LSB galaxies in both clusters. Based on stellar-population modelling and assuming no size evolution, we find that the faintest sources we can detect are about as bright as expected for the progenitors of the brightest local UDGs. We find that the LSBs we detect in SPT-CL J2106-5844 and MOO J1014-5844 already have old stellar populations that place them on the red sequence. In correcting for incompleteness and based on an extrapolation of local scaling relations, we estimate that distant UDGs are relatively under-abundant, as compared to local UDGs, by a factor similar to 3. A plausible explanation for the implied increase over time would be the significant growth of these galaxies over the last similar to 8 Gyr, as also suggested by hydrodynamical simulations.