The genetic background of familial, late-onset colorectal cancer (CRC) (i.e., onset > age 50 years) has not been studied as thoroughly as other subgroups of familial CRC, and the proportion of families with a germline genetic predisposition to CRC remains to be defined. To define the contribution of known or suggested CRC predisposition genes to familial late-onset CRC, we analyzed 32 well-established or candidate CRC predisposition genes in 75 families with late-onset CRC. We identified pathogenic or likely pathogenic variants in five patients in MSH6 (n = 1), MUTYH (monoallelic; n = 2) and NTHL1 (monoallelic; n = 2). In addition, we identified a number of variants of unknown significance in particular in the lower penetrant Lynch syndrome-associated mismatch repair (MMR) gene MSH6 (n = 6). In conclusion, screening using a comprehensive cancer gene panel in families with accumulation of late-onset CRC appears not to have a significant clinical value due to the low level of high-risk pathogenic variants detected. Our data suggest that only patients with abnormal MMR immunohistochemistry (IHC) or microsatellite instability (MSI) analyses, suggestive of Lynch syndrome, or a family history indicating another cancer predisposition syndrome should be prioritized for such genetic evaluations. Variants in MSH6 and MUTYH have previously been proposed to be involved in digenic or oligogenic hereditary predisposition to CRC. Accumulation of variants in MSH6 and monoallelic, pathogenic variants in MUTYH in our study indicates that digenic or oligogenic inheritance might be involved in late-onset CRC and warrants further studies of complex types of inheritance.