More than 500 additional candidates should have passed the examination of the Order of Nurses of Quebec last September. A poorly designed test and a pass mark that is too high lead the Admissions Commissioner to recommend a review of the results.
• Read also – Angry nurses: massive failure in professional exam
• Read also – Nurses Examination: A Serious Injustice
The Commissioner has just unveiled his second progress report in the investigation into the massive failure of the exam of the Order of Nurses of Quebec (OIIQ) in September 2022.
Of the 2,376 candidates who applied for the first time, the success rate reached only 51.4%.
A simulation conducted by the Commissioner shows that “had it not been for the high pass mark maintained by the Order, more than 500 candidates would have passed rather than failed the September 2022 exam”.
“It is as many future nurses who could have had their license to practice in the fall of 2022 and contribute to the health network, writes Ms.e Andre Gariepy. The commissioner calls for the September 2022 results to be recalculated based on a revised and fully justified pass mark.”
A failing exam
Two observations lead the Commissioner to this conclusion.
First, the review has several flaws. The documents used to write the questions are not up to date. So the Exam blueprint dates from 1999. The Skill Mosaic of the Order, it has not been revised since 2009.
Additionally, “the College does not have an exam construction-specific nursing job analysis,” the report notes.
To compensate for these shortcomings, the OIIQ chose to add a “measurement error” which added 4% to the pass mark. “Maintaining a high passing score on the September 2022 exam, without sufficient justification and documentation, resulted in a significant and unusual increase in the failure rate,” writes the commissioner.
Passing grade
Moreover, the way of determining the pass mark is itself criticized. Called the Angoff method, it consists of selecting a group of experts who assign a value to each question by evaluating the ability of a minimally competent student to answer it.
But the group selected by the OIIQ was too small, with only 10 members, and included only people with a bachelor’s degree, while the exam is also intended for female CEGEP students. In addition, some of the members were absent during the determination of the pass mark for the September 2022 session, thereby reducing the diversity of opinions sought to establish the standard.
With the Angoff method and measurement error, the pass mark was increased to 53.8%. The whole thing was then adjusted linearly to make it a “fixed pass mark of 55% for public communication”.
Unclear questions
Finally, the questions themselves were problematic: 12% of them “did not perform well statistically.” For example, they could be ambiguous or have several correct answers.
Moreover, a survey of candidates for the exam reveals that 85% of them “considered the level of difficulty of the exam unfavorably”. Similarly, “74-75% of respondents rated the clarity of questions and answer choices unfavorably.”