OBJECTIVE: To validate an MRI reference criterion for a positive SIJ MRI based on the level of confidence in classification of spondyloarthritis (SpA) by expert MRI readers. METHODS: Four readers assessed SIJ MRI in two inception cohorts (A/B) of 157 consecutive back pain patients ≤50 years, and in 20 healthy controls. Patients were classified according to clinical examination and pelvic radiography as having non-radiographic axial SpA (n=51), ankylosing spondylitis (n=34), or non-specific back pain (n=72). Readers recorded their level of confidence in the classification of SpA on a 0-10 scale (0=definitely not; 10=definite). The MRI reference criterion was pre-specified as the majority of readers recording a confidence of 8-10; absence of SpA required all readers to record Non-SpA (confidence 0-4). We calculated inter-reader reliability and agreement between MRI-based and clinical classification using kappa statistics. We estimated cut-off values for MRI lesions attaining specificity `0.90 for SpA. RESULTS: 76.4%/71.6% of subjects in cohorts A/B met the MRI criterion. Kappa values for inter-reader agreement were 0.76/0.80, and between MRI-based and clinical assessment 0.93/0.57. Using this MRI reference criterion, the cut-off for number of affected SIJ quadrants needed to reach a pre-defined specificity `0.90 was `2/`2 for BME and `1/`1 for erosion in cohorts A/B, and both lesions BME and/or erosion increased sensitivity without reducing specificity. CONCLUSION: This data-driven study using two inception cohorts and comparing clinical and MRI-based classification supports the case for including both erosion and BME to define a positive SIJ MRI for the classification of axial SpA. © 2012 by the American College of Rheumatology.