Acinetobacter baumannii has emerged as one of the leading pathogens causing hospital-acquired infection. The success of A. baumannii as a pathogen has to a large extent been attributed to its capacity to remodel its genome. Several major epidemic clonal complexes of A. baumannii spread across different health care facilities around the world, each of which contains a subset of diversified strains. However, little is known about the population dynamics during colonization of A. baumannii within hosts. Here, whole-genome sequencing was used to analyze population dynamics of A. baumannii strains isolated from a group of patients at different time points as well as from different sites of a particular patient. Seven out of nine of the sampled A. baumannii strains belonged to the international clone II (CC92 clonal complex). While the A. baumannii strains were found to be stable in three patients, there was a change of A. baumannii strains in one patient. Comparative genomic analysis revealed that the accessory genome of these strains contained a large set of virulence-encoding genes and these virulence factors might play a role in determining population dynamics. Microscale genome modification has been revealed by analysis of single nucleotide polymorphisms (SNPs) between A. baumannii strains isolated from the same patient. Parallel evolutionary traits have been observed during genome diversification when A. baumannii colonize in different patients. Our study suggested that both antibiotic usage and host environment might impose selective forces that drive the rapid adaptive evolution in colonizing A. baumannii.