Exam Questions - Guarantee for a High-Quality Certificate

Exam Questions - Guarantee for a High-Quality Certificate?

Fairness, professionalism, quality, and reputation.?

At completion of the training to become a Certified Professional for Requirements Engineering (CPRE), an examination is taken, after which the CPRE certificate is awarded. This exam must be balanced, fair and of high quality – the only way to guarantee the high reputation of the CPRE certificate.?

Questionnaires with multiple choice (MC) exam questions are used for the exam. The exam questions address the learning objectives as defined in the CPRE syllabi. For these, the highest content standards exist to meet our goal of advancing professionalization in RE.?

The questionnaires themselves must first and foremost be fair.?

• Fair in terms of ensuring that all learning objectives are included in all questionnaires with the correct weighting in terms of teaching time.?
• Fair in terms of equality - all questionnaires should have the same level of difficulty.?
• Fair in terms of wording - no quibbles in wording, but scientifically sound questions and answer choices.?
• Fair in terms of timeliness - changes and additions to learning objectives must be 100% reflected in all questionnaires used.?
• Fair in terms of time frames - sufficient time to answer, but still challenging.?

In addition, the questionnaires must guarantee the value of the certificate. The difficulty must be appropriate to the reputation of the certificate. This results in two essential aspects:?

• A large part of the candidates should pass with appropriate preparation.?
• The composition of the questions in a questionnaire must be designed in such a way that?
• very good candidates pass with a high score,?
• good candidates pass well,?
• teetering candidates are on the borderline of passing the test,?
• and bad candidates do not pass.?

How is all this ensured by IREB and what quality assurance measures are included in the questionnaires? In the following, we will illustrate how the quality of the questionnaires is substantiated in terms of content, MC test theory and methodology, and what role the University of Bern plays in this process.?

Important role of the University of Bern?

IREB develops the questionnaires based on the research results of the University of Bern and thus benefits from the latest findings in the fields of exam research and MC test theory.?

The members of the IREB examination group were instructed in a workshop by the University of Bern so that question structure, number of answer options, wording, scoring and other aspects of the multiple-choice questions follow strict scientific criteria.?

In addition, existing exam questions were reviewed by the University of Bern and potential for improvement was identified.?

Subject matter expertise?

The exam questions are initially formulated by members of the IREB working groups who have developed the content for the respective education. They ensure content correctness and alignment with the syllabus and glossary.?

Test-theoretical aspects?

The IREB examination group examines each question from a scientific point of view. They ensure that the phrases are not complicated or confusing or even contain solution hints. Their goal is to provide examinees with questionnaires that allow for quick and easy comprehension of questions and MC answers by adhering to criteria of MC test theory. MC test theory also calls for the use of so-called "distractors", i.e., information that at first glance looks like clues to correct solutions. IREB also uses this element in the exam questions. Well-prepared candidates recognize such "distractors".?

Methodological aspects?

There is continuous monitoring of the questionnaires by IREB. The Certification Bodies, which are contracted by IREB to conduct the certification exams, regularly provide anonymized information on which of the MC answers were selected for each exam question. Based on this data, an analysis tool allows us to determine the following two key metrics:?

1. item difficulty - "How hard is a question?":?Ideally, the good candidates score well, and the poorer candidates do not. If all candidates score poorly or well, the question is either too hard or too easy.?
2. selectivity - "How well does a question separate the good candidates from the poorer ones?":?Ideally, the good candidates answer a question correctly and the bad candidates answer it incorrectly.?

In the case of one or two conspicuous key figures, the question is analyzed and, on the basis of the answers given, it is identified where answers are correct or incorrect more often than average, or also where the good candidates may be misguided.?

Such evaluations via the analysis tool ensure that the exam questions are of high quality and that the difficulty levels of the questionnaires do not differ.?

For each syllabus (CPRE Foundation Level, CPRE Advanced Level RE@ Agile, etc.) there are several questionnaires with disjunctive questions. In addition, the questions are distributed in so-called pools, from which they are randomly drawn according to predefined rules, in order to be able to offer many different questionnaires for online exams.?

Translations?

In order to ensure the comprehensibility and consistency of the questionnaires even in the different languages offered, IREB relies on the combination of native speakers and translation management tool.?

Each question in the questionnaires is translated, checked and, if necessary, corrected by native speakers. This is done in a professional translation management tool, which performs a cross-check with the glossary and also includes a so-called translation memory, so that texts in all formats and publications are always translated in the same way.?

In the case of updates to the syllabus or glossary, the translation tool can be used to identify all the sections of the questionnaires that are affected by the update - for all languages!?

Fairness, professionalism, quality and high value are our goals.??

As an examinee you can be sure that - regardless of the language - the questions are professionally sound and the exams are fair in every respect.?

IREB is a non-profit organization. All revenues are used for the promotion, content and quality assurance of the documents and translations, with the aim of advancing professionalization in the areas supported by IREB and keeping the education at an excellent level.