{"id":15134,"date":"2023-07-04T14:25:35","date_gmt":"2023-07-04T14:25:35","guid":{"rendered":"https:\/\/cpatacabamcdev.wpengine.com\/?p=15134"},"modified":"2024-07-29T14:25:29","modified_gmt":"2024-07-29T14:25:29","slug":"cpatas-registration-committee-establishes-standard-setting-methods-to-determine-pass-marks-for-the-new-qualifying-examinations-starting-in-2024","status":"publish","type":"post","link":"https:\/\/cpata-cabamc.ca\/en\/cpatas-registration-committee-establishes-standard-setting-methods-to-determine-pass-marks-for-the-new-qualifying-examinations-starting-in-2024\/","title":{"rendered":"CPATA\u2019s Registration Committee Establishes Standard-Setting Methods to Determine Standards for the New Qualifying Examinations Starting in 2024"},"content":{"rendered":"

Edits were made to this article for clarity on July 29, 2024<\/b>.<\/p>\n

At its July 4, 2023, meeting, CPATA\u2019s Registration Committee made decisions regarding how the standards for demonstrating competency will be determined for the new qualifying examinations. Starting in 2024, standards for demonstrating competency on the exams will be determined after the tests are written, based on both subject matter expertise and candidate performance data.<\/p>\n

Through the competency initiative and the exam redevelopment process, CPATA aims to:<\/p>\n

    \n
  1. Establish what knowledge and skills entry-level practitioners require in order to be competent practitioners<\/li>\n
  2. Create a fair qualifying examination process that tests that these competencies have been acquired<\/li>\n<\/ol>\n

    We do not aim to make the exams \u2018easier\u2019 but rather to ensure that they measure the right thing, in the right way.<\/p>\n

    The changes we have been making to the licensure process are likely to improve candidates\u2019 ability to learn the competencies necessary for entry level practice, and to study and prepare for the examinations. They promote public protection and fairness by allowing for a more precise determination of who is ready to be fully licensed\u00a0based on the technical competency profiles.\u00a0An increased pass rate may be a consequence, but is not a goal, of the exam redevelopment process.<\/p>\n

    Specifically, the Modified Angoff method will be used to determine the standard for demonstrating competency on the knowledge examinations and the Borderline Regression method will be used to determine the standards for demonstrating competency on the skills examinations.<\/span>\u00a0<\/span>These standard-setting processes are considered best practice for high-stakes examinations and are more precise and fairer because they reflect the difficulty of each exam administration.\u00a0<\/span>\u00a0<\/span><\/p>\n

     <\/p>\n

    What standard will candidates need to meet to be successful on the new qualifying examinations?<\/span><\/b>\u00a0<\/span><\/h4>\n

    Based on the standard setting analyses conducted by psychometric experts, CPATA\u2019s Registration Committee will approve the standard for demonstrating competency for each exam administration, after the exams are written.\u00a0 Starting in 2024, exam performance will be determined after the tests are written, based on both subject matter expertise and candidate performance data. CPATA will use both test-centred and candidate-centred methods to determine the appropriate fair standard to be successful on each particular administration of the exam.<\/span> These standard-setting processes are considered best practice for high-stakes examinations and are more precise and fairer because they reflect the difficulty of each exam administration.<\/span>\u00a0<\/span><\/p>\n

    \u00a0<\/span>\u00a0<\/span><\/p>\n

    What are the Modified Angoff method and Borderline Regression methods?\u00a0<\/span><\/b>\u00a0<\/span><\/h4>\n

    Modified Angoff method<\/span><\/b>: In this test-centred method, a panel of subject matter experts (SME panelists, in this case, practising licensees) estimates the probability that a minimally competent candidate would answer each item on the test correctly. Several rounds occur, with panelists able to adjust their estimates given different information. The aggregate of these estimates informs the overall minimum standard. The specific standard used to determine \u201ccompetent\/not-yet-competent\u201d decisions would be approved following the exam administrations. This ensures that the standards for demonstrating competency are set with the benefit of the statistical performance of the items and the test overall.<\/span>\u00a0<\/span><\/p>\n

    Borderline Regression Method<\/span><\/b>: In this candidate-centred method, SME panelists (in this case, the examination advisory group members who are practising licensees) categorize candidates into 3 or more categories, of which one is a \u201cborderline\u201d category (e.g., below the standard, borderline, above the standard) but incorporates a linear regression approach. The specific standards used to determine \u201ccompetent\/not-yet-competent\u201d decisions would be approved following the exam administrations. This is because the standard can only be determined using the data obtained during the examination process. Because the specific content of the skills examinations changes with every administration, resulting in potential changes to the level of difficulty, Borderline Regression analysis will be conducted following each exam administration.<\/span>\u00a0<\/span><\/p>\n

    \u00a0<\/span><\/p>\n

    Why were these methods selected for the new examinations?\u00a0<\/span><\/b>\u00a0<\/span><\/h4>\n

    There is no single method to determine a defensible entry to practice standard for all tests or purposes; different methods are suited to different circumstances. As a licensing body, CPATA needed to select an approach to standard setting based on the purpose of the examinations, which is to assess entry level competencies, as well as how closely the method aligned with assessment goals. <\/span>\u00a0<\/span><\/p>\n

    For a high-stakes licensing examination (in other words, a test with important consequences), the selected standard-setting approach should relate to the purpose of the examination, be appropriate to the level of complexity of the competencies to be assessed by the examination, be appropriate to the specific format of examination (e.g., multiple-choice, performance-based), be supported by scholarly research supporting its use, and (for practical reasons) should reflect the availability of resources (e.g., money, time, expertise in statistical analysis, availability of experts).<\/span>\u00a0<\/span><\/p>\n

    The Modified Angoff method: <\/span>\u00a0<\/span><\/p>\n