Schools and credentials
Two applicants can perform the same task, but only one holds the certificate requested by an employer. A school can respond that its certificate represents a broader preparation than the task alone reveals. The other applicant can respond that the requirement excludes competence acquired elsewhere. Both claims deserve examination. To understand education as an institution, we must separate learning, demonstrating learning, gaining admission to instruction, and acquiring a credential that other organizations recognize. These processes connect, but none is interchangeable with all the others.
A school has more than one consequential output
Education can develop knowledge, practical skill and habits of inquiry. Schools also allocate grades, credentials and recommendations, organize peer relationships and establish expectations about legitimate achievement. These activities can reinforce one another, but they can also diverge. A student may learn a great deal without completing a qualification, or obtain a high grade by mastering a narrow assessment strategy.
Consider an invented six-week course in equipment maintenance. Its stated purpose is to prepare learners to inspect a machine, explain a fault and document a safe handoff. Its certificate requires attendance at every evening session and a written examination. The assessment might capture conceptual understanding while missing practical inspection. The attendance rule might encourage participation while excluding a competent learner whose shift changed unexpectedly.
Neither observation proves that attendance and written work are useless. It identifies an alignment question: how closely do the requirements represent the educational purpose? An institution needs observable criteria, but those criteria should be examined as choices. The criterion can be convenient to administer without being the best available indicator of the capacity the certificate claims to represent.
Now add an employer who uses the certificate to screen applicants. The school's decision travels into another organization. It becomes part of a hiring rule even though the employer did not observe the student's work. Credentials are powerful partly because they make information portable. Their influence therefore depends on relationships among schools, employers, professional bodies and learners, not just classroom teaching.
Admission is a process before it is a decision
Imagine that the maintenance course has enough places for every qualified applicant. Access could still be unequal. Prospective students must discover the course, understand its prerequisites, obtain documents, arrange time and complete registration. A person who never reaches the decision stage cannot benefit from a fair decision at that stage. An admissions audit that begins only with complete applications has already excluded part of the process.
A fictional funnel makes this visible. Of one hundred interested people, eighty open the application, sixty submit it, fifty meet the prerequisite and forty enroll. The enrollment rate is 40 percent of the original interested group, 50 percent of those opening the form and 80 percent of those meeting the prerequisite. Each denominator answers a different question. Calling all three figures “the acceptance rate” would produce confusion.
Suppose an improved form raises submissions from sixty to seventy-five while the number of enrolled students remains forty. The form may have removed a barrier, but another constraint now matters: scheduling, places, prerequisites or financing. A reform can succeed at the step it changes without achieving the ultimate outcome. Conversely, failure to change enrollment does not establish that the form had no effect whatsoever.
This sequential view changes how we interpret motivation. Not completing enrollment could reflect a considered decision, a missed deadline, an unaffordable schedule or an incorrect belief about eligibility. The behavior alone does not reveal which. Interviews can clarify meanings, while process records can establish timing and points of withdrawal. The two kinds of evidence answer complementary questions.
Assistance and information are different interventions
Eric Bettinger, Bridget Terry Long, Philip Oreopoulos and Lisa Sanbonmatsu studied assistance with the United States financial-aid application in a randomized field experiment launched in 2008 through H&R Block. The intervention combined help completing the FAFSA with personalized aid information; another arm supplied information without that help. Their January 2012 paper reports increased application and educational participation from combined assistance. Among dependent students, two consecutive years of enrollment rose from about 28 to 36 percent. Information alone did not produce a comparable measurable improvement. This is evidence about a historical intervention, not a statement about today's application form. Read the authors' paper, introduction and experimental design.
The setting matters. The program used tax-preparation encounters, where relevant financial information was already available. J-PAL's account identifies 156 offices in Ohio and the Charlotte, North Carolina area, and describes administrative records used to track outcomes. This was not a random sample of everyone who might consider college. See the study implementation and outcome account.
The What Works Clearinghouse's 2013 review rated the evidence with reservations: the initial subgroup information did not permit calculation of attrition for the reported subgroup comparisons. That limitation qualifies the interpretation even though assignment was randomized. See the review's methods explanation and footnotes.
The key conceptual distinction is between changing what a person knows and changing what completing a task requires. An information sheet can make an opportunity clearer while leaving the same time-consuming steps intact. Assistance can reduce those steps, help resolve uncertainty and connect the task to an encounter already underway. The combined intervention does not isolate each of these possible mechanisms separately.
Work through the inference rather than just the headline
Using the rounded figures above, the difference between 36 and 28 percent is eight percentage points. Relative to the comparison rate, eight divided by twenty-eight is approximately 29 percent. Those descriptions are mathematically compatible, but “eight percent” is ambiguous unless we specify whether we mean points or a proportional change. The outcome concerns consecutive enrollment, not a demonstrated increase in lifetime earnings or a completed degree for everyone.
An experiment can reveal that a package changes behavior while leaving open why it works. Perhaps help removes a difficult step. Perhaps completing the task immediately reduces postponement. Perhaps an interaction with a knowledgeable person changes expectations. These mechanisms can coexist. To isolate one, a new design would need a comparison that varies that component while holding other relevant features sufficiently similar.
Information alone showing no statistically detectable effect does not establish that information is universally irrelevant. A result depends on the information supplied, the people reached, the comparison condition, implementation and measurement precision. Different information at another decision point might matter. The defensible claim is about the intervention actually evaluated, with uncertainty appropriate to its design and follow-up.
There is also a scale question. A service delivered during an existing tax encounter may be harder to reproduce where no comparable encounter occurs. Expanding assistance can require trained staff, reliable software and coordination with another organization. The experimental result gives a reason to investigate those conditions. It does not eliminate the need to establish them in a new setting.
What does a grade measure?
Assessment validity concerns the justification for interpreting a performance as evidence of a particular capacity. If an examination tests memorized definitions, it can support an inference about recall under those conditions. It may not support an equally strong inference about repairing an unfamiliar machine. Calling an assessment rigorous tells us little until we know which interpretation the evidence is meant to justify.
Reliability concerns consistency of measurement. Two examiners might agree closely on a score, yet both use a rubric that omits a central skill. The assessment is consistent but incomplete for the intended purpose. Conversely, a rich practical task can reveal relevant capacities while producing inconsistent scoring unless standards and training improve. Validity and reliability identify different problems rather than competing labels for quality.
In our fictional maintenance course, suppose the written examination yields scores of 88 and 76 for learners A and B. A practical inspection rubric yields 60 and 92. Neither ranking can be declared the complete truth about ability without examining the tasks. If the certificate promises both conceptual explanation and practical inspection, a single score may conceal a meaningful difference in preparation.
A better assessment might require minimum performance in both domains, with a route to improve a deficient component. That design has costs: more examiner time, equipment and opportunities for disagreement. It may also be more informative to employers. The institutional choice concerns how much evidence is needed, who bears the cost of producing it and how learners can challenge a mistaken judgment.
Credentials can indicate skill and restrict opportunity
A credential can act as a signal: an observable indicator used when another quality is difficult to observe directly. An employer may interpret completion as evidence of knowledge, persistence or familiarity with workplace routines. The signal can be useful without being perfectly accurate. Its usefulness depends on what completion actually requires and how closely those requirements relate to the work.
A credential can also become a gate. If employers refuse every applicant lacking it, the requirement limits competition to recognized holders. Sometimes a gate protects a consequential standard. Sometimes it excludes relevant competence acquired through other routes. Distinguishing those possibilities requires evidence about job demands, assessment quality and alternatives, rather than assuming that every requirement is either necessary protection or empty status.
Imagine a warehouse role involving accurate inventory records. An employer requires a four-year degree but never tests record accuracy. The mismatch is a hypothesis, not yet proof of an unjustifiable rule. Perhaps the role includes analytical tasks omitted from the advertisement. A useful inquiry obtains an actual task description, compares the requirement with performance evidence and examines whether another assessment could meet the same need.
The consequences extend backward. If a credential becomes a gate, people may pursue it partly to remain eligible, even when its teaching is only loosely connected to the work. Schools may adjust programs to employer expectations; employers may rely more heavily on school classifications. This feedback can sustain a requirement beyond any individual decision-maker's original intention.
Informal expectations shape the formal task
A syllabus may explain deadlines while leaving unstated how to ask for help, interpret feedback or challenge a grade. Learners who have encountered similar institutions may know these practices. Others may treat a teacher's invitation to visit office hours as a sign of failure or may not recognize that draft feedback is available. Such differences concern access to practical institutional knowledge, not an absence of intelligence.
In a supplied example, two students misunderstand an assignment. One emails a draft and receives clarification; the other follows the ambiguous wording and loses marks. The difference could be reduced by making the clarification available to everyone. That does not require predicting each learner's background. It requires recognizing that a private exchange has altered the effective instructions for only part of the class.
A serious objection is that no institution can make every expectation explicit. Some learning involves initiative, judgment and navigating uncertainty. That is true, but it does not establish that every hidden requirement serves a learning goal. The question is whether the uncertainty is part of the capacity being taught or merely an incidental obstacle. A course assessing practical inspection need not secretly assess familiarity with academic email conventions.
We can therefore distinguish a demanding standard from an opaque route to that standard. Making assessment criteria clear does not require lowering them. Offering a correction route does not require granting every appeal. The important institutional work is to make the relationship among purpose, preparation, evidence and decision more intelligible and more open to justified challenge.
Put the pieces together
Return to the two applicants with equal performance on one task but different certificates. We now need several questions. What broader capacities does the credential assess? How reliable is that assessment? Who can access the preparation and examination? Can someone demonstrate equivalent competence through another route? What does the employer actually need? The apparent conflict cannot be resolved by praising education or dismissing credentials in general.
The school also faces constraints. It may need a scalable assessment, a predictable calendar and a qualification other organizations recognize. Learners need meaningful instruction and a fair opportunity to demonstrate it. Employers need usable information. These interests overlap without being identical. An institutional explanation makes their connections visible rather than assigning the whole outcome to the student's effort or the teacher's goodwill.
Our final reform memo will apply this logic to a smaller process. For now, the educational case teaches a durable distinction: a stated opportunity, a usable route into it, a learning experience and a recognized outcome are separate achievements. Evidence about one should not silently stand in for evidence about the others.
Check your understanding: A form-completion intervention increases aid applications but does not isolate reassurance from reduced paperwork. What can you infer, and what additional comparison would clarify the mechanism?
Expected answer: The evaluated package can affect the measured outcome, subject to its design and follow-up limits. It does not identify each component's contribution. A design comparing equivalent information and personal contact with versus without task-completion assistance could help separate mechanisms, provided implementation and outcome measurement are comparable.
Application
Allow twenty-five minutes. Create a one-page audit of the fictional maintenance certificate. State its promised capacities, admission steps, two assessment methods and one alternative route for demonstrating competence. Identify one requirement that serves the purpose and one that might merely obstruct access.
Then explain the 28-to-36-percent comparison in two mathematically correct ways. Name the outcome, historical setting and follow-up qualification. Do not turn consecutive enrollment into a claim about graduation or earnings.
A strong audit gives a reason for each requirement, considers measurement consistency and relevant capacity separately, and preserves a meaningful standard while making the route to it clearer.