Audit no. I/24/003/LPO
In 2021–2024, the system of assessing and grading matura exams did not function as it should. That is why, the exam grades were not always objective or fair. Every fifth examiner from the District Examination Board (DEB) in Poznań made content errors when assessing the matura exam papers, and 10-18% of examiners made technical mistakes. In the DEB in Poznań 40% of requests to view the matura exam paper ended in a higher grade and the need to change the matura certificate. There were 25% of such cases nationwide. For many secondary-school graduates lowering grades may be a lost opportunity for higher education. Although the two examination boards covered by the audit were organisationally prepared to do their tasks, their efforts did not help eliminate or at least significantly minimise the error rate in exam results. Also the examiners training system failed as it did not prepare them to correctly assess and grade exam papers.
Having analysed numerous press releases about errors in assessing matura exams, interventions of the Ombudsman, parliamentary questions and complaints filed with NIK, the Polish SAI considered it justified to investigate this issue.
Examiner training system needs improvements
The system of training examiners in grading principles did not ensure their full preparation for proper assessment and grading of exam papers. Despite the procedures implemented in 2021-2024, more and more examiners in the DEB in Poznań made content errors when grading assignments.
One day before checking exam papers, the District Examination Board in Poznań carried out a training for examiners which dealt with applying the principles of assessing assignments being part of exam papers in a given subject. At the end the training participants assessed example solutions, as part of a control exercise. The results of those exercises revealed that the examiners found it difficult to properly apply the rules of assessing tasks as part of the matura exams. For instance, in 2023 in two biology assignments, being part of the training, their solutions were incorrectly graded by 59% and 56% examiners.
Review of grading of 10% of papers
In 2021-2024, as part of the ongoing monitoring of the grading quality, every tenth exam paper was double checked in the course of examiners’ work (in case an error was identified, the exam result was changed). Although the mechanism made it possible to identify and correct a significant number of erroneous grades even before announcing the results, it did not help reduce the scale of incorrectly graded papers, revealed only following the review requested by examinees.
To limit the scale of errors while grading matura exams, an examiner/ double-checker was appointed in each team.
In 2021-2024, not more than 10% of all papers graded before were reviewed by the examiner double-checker, in line with effective procedures.
The adopted verification threshold enabled identifying and correcting – before announcing the exam results - at least some of the mistakes made by examiners. It did not result in reducing the scale and scope of erroneous grades, though.
In the District Examination Board in Poznań the double-check of every tenth exam paper helped identify errors in its grading. As a consequence, the exam result had to be corrected in many cases. In case of biology it was even in 54% of the reviewed papers. As for geography and chemistry it was 27% and 26%, correspondingly.
Ineffective technical verification
In the DEB in Poznań the number of examiners who made technical mistakes resulting in incorrect grading of papers in 2021-2024 totalled: 411, 391, 630 and 322, which made up from 10% to 18% of all examiners grading exam papers. Errors were identified and explained after the examiners team completed their work and before the exam results were announced.
The technical errors resulting in erroneous grades occurred, although – in line with the grading procedure – examiners were obliged to double check the papers’ review in technical terms.
Matura exams in Poland – every fourth paper with erroneous grade
From January 2021 to November 2024, the number of requests to view the exam papers (from 32 thousand to 53 thousand) and requests to review the score (from 9 thousand to 16 thousand) almost doubled.
The examinees’ requests for review showed that every fourth paper nationwide was incorrectly graded and so the matura certificate had to be replaced. It is worth noting that the data on errors concern only the reviewed papers, i.e. about 1% of all papers in various subjects. It may mean that the real scale of unidentified errors in exam results is much broader.
The biggest number of mistakes, ending in the result improvement by over 20 points, concerned 52 papers, mostly in the Polish language (30 papers). As for the total score changes, most cases were related to the exam in biology (777 papers on average), chemistry (540 papers on average) and the Polish language (742 papers on average).
Poznań with worse results than Poland’s average
The NIK audit revealed that the statistics in the District Examination Board in Poznań concerning papers which required changing the matura certificate due to incorrect grading are much worse in Poznań as compared with Poland’s average. In the audited period approx. 40% of requests to review the exam result in all subjects were approved (in 2023 it was as much as 45%). It means that nearly in every second case the request to change the exam result was justified.
The percentage of approved requests concerning biology and chemistry was particularly high. It is worth noting that high score in these subjects is important in the context of recruitment to some studies, e.g. medicine or dentistry, and lowering the exam result by one or two points may mean that the examinee will not be admitted to the major of his or her choice.
Even every fifth examiner made content errors
In 2021-2024, in the DEB in Poznań the share of examiners making content errors when grading assignments doubled (from 10% in 2021 to 20% in 2024). Especially, among biology examiners, 60%-75% of them made at least one mistake and in case of chemistry it was 46%-78%.
Despite the large scale of errors, nearly 98% of examiners covered by the survey claimed that they did not find it difficult to grade assignments from exam papers in terms of their content and according to 90% of them the review principles were comprehensible, unambiguous as well as easy to interpret and use.
The problem of content errors in exam grading intensified, despite taking measures defined in the procedures of the District Examination Board in Poznań, including striking off an examiner’s name from the examiners register. In 2021-2024, nine entries were struck off in that way. According to the DEB experts, those examiners erroneously graded from 55% to 94% of exam papers covered by the review.
It needs to be pointed out that the current legal order allows changing an incorrect exam result exclusively upon the examinee’s request.
Unfavourable dates for viewing exam papers
Despite the surge in interest in gaining access to exam papers (from 32 thousand to 53 thousand), the Director of the Central Examination Board did not establish any rules to make sure that as many interested parties as possible could see their works in an optimal time. He also did not oblige the DEB heads to streamline the process of making the papers available to examinees. Meanwhile, the date of viewing the exam paper is of key importance for them.
The growing number of exam viewing requests, while applying the existing procedures, could have disrupted the recruitment process for candidates for studies. For example, in 2021-2024, at the Pomeranian Medical University in Szczecin, a total of 960 persons applied for admission to studies after the result of the matura exam was increased due to the disclosure of an error in grading, and at the Medical University of Poznań it was 780 persons.
Complementary recruitment will not level the candidates’ studying opportunities
Complementary recruitment for people whose result of the matura exam was increased following the score review or appeal – i.e. the solution adopted in the Act on Higher Education and Science – in practice did not fully protect examinees against adverse effects of mistakes made by examiners.
Growing outlays and worsening quality of grading
In 2021-2024, expenditures on the functioning of the matura exam system increased from PLN 75.5 million to PLN 105.7 million, including expenditures related to the process of checking and grading papers: PLN 61.5 million in 2021 and PLN 87.3 million in 2024. The largest amounts were spent on examiners' pays – in 2021-2024, they totalled PLN 54.2 million, PLN 56.7 million, PLN 59.8 million and PLN 77 million, correspondingly.
The NIK audit findings showed that the situation was getting worse and worse in 2021-2024. Even the examiners’ 30% pay rise in 2024 did not help. However, more than half of the examiners surveyed indicated that it is the low remuneration rates that affect the exam grading quality.
Recommendations
To the Director of the Central Examination Board to:
- strengthen supervision over the implementation of tasks related to the assessment and grading of matura exams by district examination boards, including more effective elimination of errors in the grading of papers before announcing the exam results.
To the Director of the District Examination Board in Poznań:
- conduct a reliable analysis of the risks associated with the process of checking and grading exam papers, in particular in terms of their level and taking actions to effectively reduce the scale and scope of errors made by examiners at the DEB in Poznań.
