The consumption of foods with intact cellular structure is recognized to lower the glycemic index. Studies have focused on the gelatinization and digestibility of starch in intact cells. However, the effects of intact cells on starch retrogradation and digestibility are still unclear. Mild acid and alkali soaking were used to isolate intact potato cells and hydrothermal treatment (73 °C for 30 min) followed by retrogradation (4 °C for 0–15 days). The retrogradation period of 0–15 days increased the crystallinity and enthalpy change, while decreased the digestion rate and extent in potato cell and starch ghost samples. Furthermore, the existence of cell wall (potato cell) lowered the starch digestion rate and extent than potato starch ghost. It was concluded that the ghost structure is the main factor that controls the starch retrogradation, whereas the intact cell wall structure provides the barrier for starch digestion and protects the ghost structure.