Results from tests or assessments can be used to make different types of decisions. The specific decision we want to make based on a test result will determine how we interpret the test results.
Sometimes we want to compare students with their peers or rank-order them. When we use tests to make this type of decision we say we are making a norm-referenced or normative interpretation of test results.
Some tests are used to evaluate a person's mastery of a subject, usually by using cut-scores to define proficiency or pass/fail decisions. When tests are used to decide if a student meets a certain pre-defined standard or criteria, we say we are making a criterion-referenced interpretation of test results.
In this interactive module we will use an explorable example to learn the basic concepts and uses of both norm-referenced and criterion-referenced interpretations of test results.
Mary is a 9th grade student at ABC High School and she just took the final Math I test.
The results from this test will be used to:
How can we use Mary's test score to decide if she can move on to Math II without remediation? How can we use that same score to decide if she will receive a prize for being a top performer in comparison to her schoolmates?
Let's find out!
Some details about our example:
Mr. Smith, Mary's math teacher, computes the Math I test score by counting the number of correct answers, with no penalty for incorrect responses. We call the number correct a raw score because it is based solely on the number of correctly answered items on the test.
A raw score might be difficult to interpret and compare since tests can have different numbers of questions and total numbers of points. A simple way to make sense of a raw score is to put it on a percent scale. When we transform the raw score into percent correct we produce a scaled score.
If a student got questions right out of 50 on the test, the student would score % correct on the test.
In our example, let's assume Mary got correct questions (raw score) or % correct (scaled score) on the test.
Does this score tell you if she won any of the Math prizes?
No! We don't know if Mary won a prize because her raw and percent correct scores tell us nothing about how her score compares to her schoolmates.
In order to know how one student's score compares to other students who took the same test we must have a norm-referenced score.
A norm-referenced score typically indicates the examinee's relative position in a group of test takers that we want to compare the examinee's score with. Once we have information about this group we are able to answer the questions above.
A basic ranking in terms of raw or scaled scores can start to give us a better picture of how Mary's performance compares to that of her classmates.
Activity 1: You may sort the values in the graph below to see how Mary ranked in her class in terms of both raw and percent correct scores. Is Mary in the top 10% of her class?
There are 20 students in Mary's class, so to place in the top 10% she would need to be one of the top 2 students in her class. Looking at the sorted chart above we can see that Mary did not rank in the top 10% of her class.
Can Mary still win a Math I prize?
To answer this question we first need to define Mary's norm group. A norm group is any group we wish to make comparisons against.
In order to quickly understand how a particular score compares to other scores in a norm group we usually use a type of norm-referenced score called percentile rank. If a raw score of 15 correspond to a percentile of 30, it means that 30% of the norm group had raw scores lower than 15. Similarly, when we say that a student's score is at the 80th percentile we are saying that the student's performance on the test was equal to or better than 80% of the students in the norm group.
Percentile ranks are different from percent correct scores! The percentile rank indicates a student's relative standing in a group. The percent correct score indicates what percent of the total possible points a student got on a test.
Activity 2: See how Mary's percentile rank changes when you change the norm group. Did Mary win a Math I prize?Norm group:
Yes! Mary won a Math I prize since she placed in the top 10% of all 9th graders at ABC High even though she was not in the top 10% of her class.
Can you tell if she will be able to move on to Math II without remediation?
Placement, mastery or pass/fail decisions are usually made by comparing the student's performance on a test against a fixed set of predetermined criteria or learning standards.
Test performance is summarized as the student's test score, and the predetermined criteria is usually defined as a minimum cut-score that the student must obtain to demonstrate proficiency in the subject matter.
When we compare test scores against a cut-score to make placement, mastery or pass/fail decisions we are making a criterion-referenced interpretation.
Activity 3: In our example, the minimum criteria for moving on to Math II without remediation is a cut-score of 35. Use the chart below to set this cut-score. Will Mary be able move on to Math II without remediation? How many students in her class will need remediation before being eligible to move on to Math II?
Yes! Since Mary scored above the cut-score of 35, we assume she has demonstrated sufficient mastery of Math I and can move on to Math II without remediation. However, 11 students in Mary's class did not meet the predefined criteria and will need remediation before being eligible to enroll in Math II.
Generally speaking, it is not the test but the interpretation of its results that is norm-referenced or criterion-referenced. A test might have been built to provide criterion-referenced interpretation of a student's mastery of a subject but after many administrations it could be used to make a norm-referenced interpretation, that is, to compare the student's performance to that of other test-takers.
Additionally, we can, and often do, interpret a single test score both in terms of norm- and criterion-referencing.
Activity 4: Now we will look at some sample questions that use both criterion-referenced and norm-referenced interpretation of test scores to apply the knowledge you have gained in this module.
Does the question selected require a norm-referenced, criterion-referenced or combined interpretation of the test results?
In our example, we could say that we made a norm-referenced interpretation of Mary's test results when:
We made a criterion-referenced interpretation of Mary's test results when:
We made a combined norm- and criterion-referenced interpretation of Mary's test results when:
This work by Luciana Cançado is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.