A proposal for Examination Practice and Evaluation Reform

Abstract

The examination system in the educational institutions of Pakistan is facing a grave crisis. The aim of this article is to highlight a serious problem confronting the examination system in our country and propose some solutions to it. The problem is of the non-existence of any procedure for ensuring comparability in standards across the different educational boards and universities of this country and across the years.

Evolving an examination system so that the results truly reflect a student’s academic ability relative to his contemporaries and predecessors irrespective of the examining board is a science, which we have neglected in detriment to our education system and its international standing.

This problem is the very root of the examination crisis being faced by the educational institutions of Pakistan today. Because, as explained in the article, it not only hampers any efforts for improving the examination/education system it also encourages cheating by providing a spurious justification for the use of unfair means in exams.

While formulating the proposals care has been taken that these are in consonance with our secio-political environment and that if these proposals are adopted the resulting change in the system are smooth.

The problem

The examination system in the educational institutions of Pakistan is facing a grave crisis. Most people will agree that educational standards in this country are gradually deteriorating. However, if we look at the examination results of different boards and universities we find that more and more students are passing their exams with higher and higher marks. In the post only a small percentage of people passed the exams in first division but now a much higher proportion of students do. The explanation is, in my opinion, that for various reasons it has become much easier to obtain good marks in examinations. It is well known that even the students obtaining very high positions in their respective boards or universities have manifestly little knowledge and understanding of their subject. The marks obtained by students are clearly not a reliable indicator education system virtually meaningless.

Cause of the Problem

The well-known cause with are considered to be responsible for the present situation are:

1. Use of unfair means in examinations.
2. Students can get full marks by cramming just a part of ever course syllabus instead of studying the bulk of it.
3. Marking is sometimes overgenerous.

The step taken so far

The more pressing and topical of the above is the serious epidemic of the use of unfair means in examinations. Authorities are making strenuous efforts to tackle this problem but without much success. At present they are concerned with the more general aim of improving law and order can never eradicate the problem or even make worthwhile improvements in the situation. The other two reasons are somewhat over shadowed by the first. Nevertheless, there is a consensus that our examination system needs a thorough reviewing. But no serious, systematic efforts have ever been made for it.

The root cause

The aim of this article is to highlight a serious problem confronting the examination system and try to present some possible solutions to it. It is my conviction that this problem is the root cause of the rampant use of unfair means in examinations, as well as the abrupt deterioration of education standards in the country. The problem of the THE NON-EXISTENCE OF ANY PROCEDURE FOR ENSURING COMPARABILITY IN STANDARDS ACROSS THE DIFFERENT EDUCATIONAL BOARDS AND UNIVERSITIES OF THIS COUNTRY AND THE LACK OF ANY POLICY FOR ENFORCING THIS.

At the time of independence there were only two universities in the western part of the country, which responsible for conducting examinations at all levels. Now there are more than twenty educational boards and nearly twice as many public and private universities in the country. The number of students studying at different schools, colleges and universities has increased by probably even a higher proportion. This increase in boards and universities has created some new problems. One problem is simply the question of whether students passing exams of different boards or universities who also score equal marks can be considered to be of the same scholastic standard. Different boards can certainly have different examination standards. Therefore it is imperative that a procedure is adopted for maintaining comparable standards in examination from different boards/universities.

Presently marks obtained in the exams, irrespective of the board or university, are the sole indicator of the scholastic ability and constitute the academic criteria for job selections or admissions to higher classes at universities. Using these criteria it is possible, at least in theory, that a student securing even first position in the intermediate examination from some board may still be unable to gain admission to an educational institute of his/her choice because his examination board had a higher standard in examinations than others, resulting in lower marks. On the other hand, many students of much lesser ability who passed their exams from another board gain admission to colleges or universities of their choice or land jobs of their choice simply because their examination board could or would not maintain good examination standards and procedures: perhaps easy papers were set, or marking was overgenerous or there were no proper arrangements for preventing candidates using unfair means in the exams.

This problem is not only of the comparability of results of different boards. The lack of harmony in examination standards and results from year to year even of the same board contribute this problem. An allied problem is the following: if students passing exams during different years from a board or university score equal marks then can we reasonably consider them to have the same scholastic standard? The havoc which problem can cause became very apparent when the semester system was introduced in the country in the seventies. All those who passed their exams under that system got much higher marks. Since there was no way to compare the results under the two systems, they had undue advantage over those who passed before the system was introduced and after it was abolished. Clearly non-existence of any procedure for comparability of standards across the years makes it very risky to experiment with any changes in the education system.

There has been awareness of this problem for a long time. However its gravity and resulting loss to the educating system is perhaps not fully appreciated. Probably this is why no serious efforts have ever been made to solve the problem. Before proposing a possible solution I feel it necessary to throw some light on the loss it causes to our educational system.

Why it is root cause?

There is a pressing need for reviewing the examination system so that it is a more reliable and accurate indicator of a student’s ability, across the whole examination board system. For example, steps should be taken to constrain students to study the bulk of every course syllabus by making it difficult to get full marks by studying only a part of it or by answering only a few questions in examinations. The rigidity of the syllabus and choice of set book for each course exacerbates the problem for it greatly limits the type of questions that can be set. Questions in some instances are therefore almost predictable. But no board or university is willing to make its examination or examination procedure more difficult than the others. For otherwise, its students will obtain lower marks than students of some other boards. It will therefore be disadvantaging its students who will be competing such students with less rigorous boards/universities. It would be almost impossible to persuade all the boards and universities to agree on concurrent joint reforms of their education systems. Universities and boards are to a great extent independent in forming their examination rules and regulations, and it is accepted that in the academic interest this independence should be maintained. The present situation is creating a kind of inertia since no educational institution is willing to risk taking any steps which might jeopardize the future of its students.

This situation encourages cheating and makes it difficult to curb the use of unfair means in examinations, a problem that has now taken on epidemic proportions. If students are prevented from copying or cheating in any way at one place, they will feel aggrieved and victimized. For they will know that at other places students will get much higher marks by using unfair means. It seems that some institutions/examination centers, instead of providing better education for the students, concentrate their efforts on providing facilities for cheating in the exams. We sometimes hear disturbing stories that at such and such a center questions were solved on blackboards or that photocopying machines were brought into the examination halls and solutions distributed. It is difficult to say whether any part of the country is safe from this epidemic of cheating. In crisis, Booty Mafia is very well organized and is resisting all efforts by authorities to curb its spread. Indeed, in some remote areas there are complaints that authorities deliberately turn a blink eye to it. Some authorities even blatantly provide facilities for cheating in exams, in the knowledge that it goes on elsewhere, so that their students are not disadvantaged in comparison with others.

I hope that it is now clear that how this problem encourages cheating by providing a spurious justification for the use of unfair means in exams. In my opinion this is the very root of the crisis and must be tackled first before even attempting to deal with the other problems. Any other strategy would be futile. Thus it is imperative that we have an equitable procedure for comparing results of different boards and universities to curb the menace of the use of unfair means in exams and to improve the standard of education in the country.

What others are doing?

Before proposing a solution for our country it would be appropriate to look into what is being done elsewhere. Perhaps the British example would be more relevant because our educational system is historically modeled on the British System.

The UK has a system of A level GCEs, which are necessary for university entrance and are set by various boards. Nevertheless, within and outside the UK, the A level has an enviable reputation for consistency and reliability which is achieved by a well-tried system for ensuring (within accepted statistical margins of error) parity of results from year to year and between the various examining boards.

The system is reasonably simple. Every student is awarded a certain grade in each subject depending upon his performance in the exams in that subject. Basically it is assumed that the percentage of students getting a certain grade each year for each board is roughly constant. Thus the marks of candidates do not directly come into the computation of their grade.

The grades are from 1 t0 6 for a pass, with 1 the top grade. Then grades 7 to 9 for fails plus an unclassified U grade, which signifies, total failure. It is assumed that approximately 10% of candidates will get grade 1, 20% grade 2,20% grade, 3, then 10% for each of grades 4,5,6 and the remaining 20% fail.

All borderline cases are examined by the Chief Examiner who also takes a sample to mark from each marker and then can adjust marks accordingly. There are also roving national Chief Examiner Committees who visit each board’s office time to time and sample marked papers from the present and previous sittings.

The important points, which I would like to emphasize, as they special relevance to the proposal that I am going to put forward shortly, are:

1. Students have different grades in different subjects.
2. Percentage of students getting a certain grade each year for each board is roughly constant.

A possible solution for us

While proposing a solution to the problem one must kept in mind that it must be in consonance with our socio-political environment and that any resulting change in the system must be smooth. Now I come to my proposal.

"When there is a need for comparison between the results of two students, who have passed their exams from different boards, then instead of making the comparison on the basis of marks obtained by them we should examine their rank (which I shall define later) in their respective boards. An ordering of students is obtained through the idea of rank. Students obtaining first position (in rank ordering) from different boards should be deemed equal in scholastic ability even if they have obtained different marks. Then those who obtained second position will be regarded equal, similarly for those obtaining third position and so on."

This is the basic idea of my proposal in a rather simple and crude form. But before giving it an implementable shape several details will have to be sorted out.

1. For determining the rank of any student we will have to know the number of students in his board who obtained the same marks and how many obtained higher marks. If these numbers are M and N, respectively, then the rank of each of the M students obtaining equal marks will be (M+1)/2+N. To illustrate this formula suppose that in Lahore board’s intermediate examination 1998, there were three students who scored 830 marks and there were 75 students who scored more than 830 marks. Then the rank of each of the three students scoring 830 marks will be (3+1)/2+75, that is 77. It is to be noted that there will be no student in the board with rank 76 or 78 in this case. It may also be noted that rank will be in fractions if M is even in the above formula. It should be clear that a student with higher marks than another student would receive a lower rank number. Thus the smaller the rank number the better the marks. This, of course, agrees with what is normally suggested by rank: for instance, rank 1 is regarded as better than rank 2.
2. The number of students appearing for an exam is never the same for different boards. Two questions arise from this observation. First, if there are 7000 of one board and 35000 of another, then is first position in one board the same as in the other? Second, which rank in the former board is equivalent to, say the rank 20000 in the latter? These questions make is essential to use the concept of Ranking Quotient instead of rank when comparing results of students from different boards or from different years. By ranking quotient (RQ) of a student is meant the ratio of his own rank to the total number of students of the board. Thus if a student’s rank is 5000 from a board in which 7000 students appeared for the exam then his RQ will be 5000/7000 (=(5/7), and this student will be treated as having the same merit as the one whose rank is 25000 from a board in which 35000 students appeared for the exam. Again a student with smaller RQ than another will be regarded as the better student.
3. Students can opt for different courses in their exams. For example, some students may opt for Physics, Chemistry and General Mathematics as elective subject in B.Sc. and another group of students can opt for Mathematics A & B and Statistics. Then there will be different numbers of students for different courses. Also there will be some students who will pass their exams by parts. For these reasons it will become impracticable to determine rank and RQ of students using their total marks and by using the total number of students appearing in the exams. Instead we will have to find a student’s rank for each course he has passed and then his RQ by considering the total number of students who attended that course’s examination.
4. RQs of a student for different subjects will have the same significance as that presently given to marks for that subject. Accordingly, we shall also have to adopt a new measure to replace (or supplement) the total marks obtained. We shall call this measure the Percentile Cumulative Rank (PCR) and it will be the average of a student’s RQs over the different subjects passed by him multiplied by 100. PCR of a student will be a number greater than zero and less than 100. If a student’s PCR is 9.57 then he would be considered to be roughly among the top 10% students and his performance in the exam could be regarded to be better than 90.43% students.

   If these proposals are accepted then certificates or result cards for each student, in addition to having columns for marks obtained and total marks in each subject passed, will also have columns for his rank and the total number of students appearing in each subject, as well as the Percentile Cumulative Rank together with the total marks obtained by him. In the following table a sketch is given for such a certificate.

For certain reasons it will be necessary to have columns for the individual subject marks obtained in the certificate. One reason is that it will enable comparison of results of students who passed their exams before and after the implementation of this scheme and will thus guard smoothness of the change in system. Another reason is that marks obtained are in themselves, in certain instances, a useful measure of a student’s abilities. However, there is a need for improving the usefulness of this measure and my next proposal aims at this end.

Second proposal

Lack of harmony in the examination system among various examining boards and universities becomes very evident when we are told that the pass rate in, say, the intermediate examinations conducted by one of the boards in the country was 28% and in another it was 21%. It would be wrong to assert from this difference in pass rates that students of one board were more hardworking or intelligent than of the other. The only likely reason for the difference is that the standard of examination was different in the two boards. It is essential for the sake of harmony and fairness that this disparity should never be allowed. This can be achieved using the following policy.

A committee of chairmen of different boards should decide before declaring the results of every examination what should be the pass rate in the exam. This decision can, for instance, be based partly on the country’s academic needs in particular subjects. In time, sophisticated statistical methods may be used to help arrive at a pass rate taking into account the data of earlier years.

Once a pass rate is set, every board should either upscale or downscale the marks of their students to achieve this pass rate. For example, if it is decided that the pass rate for a particular examination in a certain year is to be 40% and in some board 40% of students secure 30% or more marks, then every student of this board will have his marks suitably scaled up so that the marks of those students achieving at least 30% become at least 33% (33% being the pass percentage). Similarly, if in some board 40% of students secure 38% or more marks then each student will have his marks suitably scaled down so that the marks of those students with less than 38% become less than 33%. We will again have to resolve some details before implementing this proposal.

For the some reasons (item 3, page 5) which make it necessary to evaluate rank for each subject on the basis of marks obtained in that subject, it would not be possible for us to set a pass rate for the whole of examination. Instead we will have to decide the pass rate for each subject. This pass rate can be the same for each subject, but in practice may be different for different subjects. For example, for some year it can be decided that the pass rates in the intermediate examination will be 60% for Urdu, 55% for Islamiat, 50% for English, 45% for Chemistry, and 40% for Mathematics and Physics etc.

A method for scaling marks

While up-scaling or downscaling marks for achieving the desired pass rate, a formula needs to be adopted which does not change the ranking of students. If in a certain subject the required pass rate is 40% and there are 40% of candidates who have secured 30% or more marks, then we will have to add 3% marks to those who have obtained 30% marks. It would be nice if we could proportionately increase marks for every student appearing in the exam for that paper. That is add 4% to those who have obtained 40% marks, 5% to those who have obtained 50%, etc. However, this is not possible because in this way we may end up giving more than 100% marks to some students. The following is a possible formula.

Suppose that the pass rate of students in a certain course is agreed to be r%, and suppose that in a certain board this percentage of students secures a minimum of p% marks. Then every student whose raw marks, i.e. marks before scaling, are m% will have (22p) (100m) (100p) percentage marks added to his raw marks.

1. It is assumed that the maximum marks in the course are 100, and pass percentage is 33. The formula can be modified accordingly if these values are different.
2. The factor (33p) (100p) in the formula is the increment-scaling factor. It will be positive if p is less than 33 and negative otherwise. Thus the marks will be scaled up or down accordingly.
3. The increments obtained using this formula will be usually infractions. This number may be rounded to obtain a final value in integers.

The propels given in this article will, if accepted, have to be implemented using a computer. Therefore it may be in order to present the above formula in computer pseudo code as well.

sm: = m + round (33p)*(100m)(100p)

Here sm denotes the scaled marks.

Epilogue

This is essence concludes my proposals. For the sake of brevity and clarity I have not gone into some minor details, which need to be decided before implementation. For instance, what should be considered to be the total number of students for a particular examination: should it only count those who sat the examination or count all who registered for it, or how many significant digits after the decimal may be retained in the Percentile Cumulative Rank (PCR).

Moderating examination marks so that they truly reflect a student’s academic ability relative to his contemporaries and predecessors, irrespective of the examining board, is a science that we have neglected in detriment to our education system and its international standing. As I have explained before, the priority is to formulate and implement regulations for parity in our examination system in consonance with our socio-political environment. Only then can we stem and eradicate the epidemic of unfair practices and only then can our education system be amenable to further improvements leading ultimately to the ideal of an examination system we can be proud of and that is respected internationally. Punjab University being the pioneers certainly has a responsibility to take the lead in this regard.

The author wishes to express his gratitude to Dr. V. C. Mavron, from University of Wales, for providing information about the British examination system and for going through the first draft of this article and for providing very useful comments and suggestions.

| Home | Cycling the Babusar Pass |
| Allama Iqbal Society for Promotion of Education |
| A proposal for Examination Practice and Evaluation Reform |
| Operations Research and its Prospects in Pakistan |
| From Abbotabad to Thandyani and Beyond …. |