MCC readies a new, computerized medical qualifying exam

Canada's new doctors can look forward to writing qualifying exams that take half as long and are offered twice as frequently because of a new, computerized examination system introduced by the Medical Council of Canada (MCC). The Part 1 exam currently takes 2 days to write and is offered only once a year due to the complexity and expense of administering it. Beginning in the fall of 2000, however, all 16 medical schools will offer a 1-day, 7.5-hour computerized exam twice annually. The exam, which will be delivered from the council's Ottawa office via the Internet, will continue to cost $600. FIGURE

Figure.

The new computer-based MCC Qualifying Examination Part 1 eliminates the problems and costs associated with printing and distributing the paper exam. "With no physical delivery, it's much easier to administer," says David R. Miller, the council's manager of computer-based testing. "It's also better for students."

Candidates can schedule their own exam by selecting from a menu of dates at their chosen locations. Most important, though, the new exam is shorter due to an innovative, adaptive format.

Miller explains that whereas non-adaptive exams cover all disciplines and all levels of difficulty, "the adaptive exam is tailored to your ability. We don't waste time asking easy questions that you already know and we don't waste time asking really hard questions that will frustrate you."

As a result, two-thirds of the multiple-choice section of the exam will be eliminated. With the current exam, the multiple-choice sections take a day-and-a-half to complete. The new computerized exam reduces this to a half day. Adaptive tests have proven to be as reliable as fixed-length tests (for more information see http://ericae.net/scripts/cat). How can the examination be fair if everybody gets a different one? The MCC says everyone will get an equal chance to demonstrate his or her abilities.

"If you are unable to answer questions in some areas at a particular ability level, the computer will adjust the difficulty level accordingly," says the council. "The computer will always give you the opportunity to better yourself as you continue with the examination."

The exam doesn't produce an accurate estimate of how good a student is, but that's not the goal; it's only intended to indicate a pass or fail. "Our mandate is to screen out the relatively small percentage of people who would like to practise in Canada but don't have the skills to do so," explains Miller. "We set the bar. This isn't an assessment tool as such."

The MCC will continue to provide aggregate exam data to universities so that they know where their students' strengths and weaknesses lie and can adjust curricula accordingly.

Pilot sessions for the new Part 1 exam were run at 7 centres last month. Part 2 of the exam, which measures human skills such as examination techniques and patient communication, will continue to be offered as a series of real-life interactions. All 3 steps of the United States Medical Licensing Examinations have been administered by computer since May 1999.

The council admits that the move away from pencil and paper to computers has the potential to create problems. For instance, if a student's computer crashes during an examination, some responses may be lost. In the "rare event" that a network crashes, exams would be rescheduled as soon as possible. The MCC compares this to a power outage or building evacuation during a pencil-and-paper test and notes that a new exam "will be much easier to reschedule."

Candidates who don't use a computer regularly can familiarize themselves with the look and feel of the exam at the MCC's Web site (www.mcc.ca), and practice disks will also be available. - Barbara Sibbald, sibbab{at}cma.ca