A rigorous process of question analysis is undertaken by the Assessment Committee following an administration of the CSAO in order to ensure that the questions included in the examination are unambiguous and discriminating. Questions that fail standard measures of their integrity are removed from scoring for all candidates.

Candidate's individual assessment experiences are also considered. If a question was not administered appropriately to an individual applicant, the item may be deleted from the scoring of that applicant alone. For example, if an event within the administration of a Skills component prevented a candidate from obtaining particular results, then the affected questions are deleted from that candidate's scoring.

Assessor’s responses to questions are reviewed to ensure accuracy by comparing the expected clinical results to ca candidate’s results. Inaccuracies in assessor’s responses are corrected prior to scoring.

Before publishing results, the Assessment Committee considers the overall reliability of scores generated. Particular attention is paid to ensure that item analysis determinations, generally and individually, have been applied to each candidate's assessment.

When a candidate appears to have failed a component, a further analysis is undertaken prior to publishing the results to ensure that the score resulted from that candidate's performance and not from administrative factors.

These procedures are employed to ensure that candidates receive a fair assessment of their performance. These mechanisms require significant effort and time and are the primary cause of the time interval between administration and reporting of scores.