Members of the P4 subfamily of P-type ATPases are thought to help create asymmetry in lipid bilayers by flipping specific lipids between the leaflets of a membrane. This asymmetry is believed to be central to the formation of vesicles in the secretory and endocytic pathways. In Arabidopsis thaliana, a P4-ATPase associated with the trans-Golgi network (ALA3) was previously reported to be important for vegetative growth and reproductive success. Here we show that multiple phenotypes for ala3 knockouts are sensitive to growth conditions. For example, ala3 rosette size was observed to be dependent upon both temperature and soil, and varied between 40% and 80% that of wild-type under different conditions. We also demonstrate that ala3 mutants have reduced fecundity resulting from a combination of decreased ovule production and pollen tube growth defects. In-vitro pollen tube growth assays showed that ala3 pollen germinated ~2 h slower than wild-type and had approximately 2-fold reductions in both maximal growth rate and overall length. In genetic crosses under conditions of hot days and cold nights, pollen fitness was reduced by at least 90-fold; from ~18% transmission efficiency (unstressed) to less than 0.2% (stressed). Together, these results support a model in which ALA3 functions to modify endomembranes in multiple cell types, enabling structural changes, or signaling functions that are critical in plants for normal development and adaptation to varied growth environments.