Human papillomavirus (HPV) infection is the cause of the majority of cervical cancers and head and neck cancers worldwide. Although prophylactic vaccines and cervical cancer screening programs have shown efficacy in preventing HPV-associated cervical cancer, cervical cancer is still a major cause of morbidity and mortality, especially in third world countries. Furthermore, head and neck cancer cases caused by HPV infection and associated mortality are increasing. The need for better therapy is clear, and therapeutic vaccination generating cytotoxic T cells against HPV proteins is a promising strategy. This review covers the current scene of HPV therapeutic vaccines in clinical development and discusses relevant considerations for the design of future HPV therapeutic vaccines and clinical trials, such as HPV protein expression patterns, immunogenicity, and exhaustion in relation to the different stages and types of HPV-associated lesions and cancers. Ultimately, while the majority of the HPV therapeutic vaccines currently in clinical testing target the two HPV oncoproteins E6 and E7, we suggest that there is a need to include more HPV antigens in future HPV therapeutic vaccines to increase efficacy and find that especially E1 and E2 could be promising novel targets.