The recent identification and rise to dominance of the P.1 and B.1.351 SARS-CoV-2 variants have brought international concern because they may confer fitness advantages. The same three positions in the receptor-binding domain (RBD) are affected in both variants, but where the 417 substitution differs, the E484K/N501Y have co-evolved by convergent evolution. Here we characterize the functional and immune evasive consequences of the P.1 and B.1.351 RBD mutations. E484K and N501Y result in gain-of-function with two different outcomes: The N501Y confers a ten-fold affinity increase towards ACE-2, but a modest antibody evasion potential of plasma from convalescent or vaccinated individuals, whereas the E484K displays a significant antibody evasion capacity without a major impact on affinity. On the other hand, the two different 417 substitutions severely impair the RBD/ACE-2 affinity, but in the combined P.1 and B.1.351 RBD variants, this effect is partly counterbalanced by the effect of the E484K and N501Y. Our results suggest that the combination of these three mutations is a two-step forward and one step back in terms of viral fitness.