Objective: Immediate direct-to-implant breast reconstruction with acellular dermal matrix (ADM) is the method of choice for many plastic surgeons and patients, but the use of ADM remains a controversial subject in the literature. This study aimed to investigate complications, reconstructive failure and possible risk factors in direct-to-implant breast reconstruction with ADM (primarily Strattice™). Methods: We retrospectively examined all patients undergoing immediate direct-to-implant breast reconstruction with ADM, during a five-year period (2014–2019) at a university clinic. Study outcomes were all complications and explantations. Complications were stratified within and after 6 months postoperatively and subcategorized by type of intervention. Explantations were subcategorized into loss of implant or salvage with immediate insertion of a tissue expander, the same or a new implant. Results: We included 154 patients and 232 breasts. Complications within 6 months per patient included hematoma (4%), seroma (8%), infection (9%), necrosis, wound dehiscence and delayed wound healing (19%). The total complication rate per patient was 34%. Explantation occurred in 20 patients (13%) of which 9 (6% of all) had implant loss. Preoperative radiotherapy was a significant predictor of explantation (adjusted OR 4.9, 95% confidence interval (CI), 1.0–23.5; p = 0.045), and smoking was also associated with risk of explantation, although only borderline significant (adjusted OR 4.0, 95% CI, 1.0–15.8; p = 0.050). Conclusion: This study demonstrates acceptable rates of re-operations and implant loss compared to other studies but highlights the importance of proper patient selection with regards to risk factors to minimize complications.