Education and opportunity

A fictional employer receives two applications for a technical role. Both applicants complete the same practical exercise equally well, but only one holds the employer's preferred qualification. The employer chooses the credentialed applicant. Has education improved productivity, supplied useful information, restricted entry or simply marked a difference that began before either applicant enrolled? The hiring decision alone cannot answer. It is compatible with several mechanisms that require different evidence.

Education can change what people know, certify what they have achieved, connect them to others and organize access to later opportunities. These functions can coexist. The task is not to select one as the universal truth about schooling. It is to ask which function explains a particular outcome, how the institution distributes access to it and what comparison would reveal its contribution.

Learning changes capacity

Begin with a simple case. Before instruction, a learner cannot interpret a wiring diagram. After guided practice and feedback, the learner can identify a fault in a new diagram. If the assessment actually measures the intended ability, this is evidence of learning. It does not yet establish how much of the change came from this particular course rather than independent practice, prior exposure or another experience during the same period.

The distinction between demonstrating change and attributing change matters even when the improvement is welcome. A before-and-after comparison records development. A causal claim about the program asks what would have happened without it. A comparison group, assignment rule or other credible design helps address that second question. The appropriate design depends on the program and the outcome, not on a desire to label every activity an experiment.

Useful learning can be narrow or transferable. Memorizing the solution to one familiar exercise may improve a test score without improving performance on a new problem. A course intended to develop transferable skill should therefore assess a genuinely new task that depends on the same underlying understanding. The assessment must avoid introducing unrelated obstacles that would obscure the skill being tested.

This concern connects education to inequality. If an examination rewards prior familiarity with an example that some students encountered at home, the score may combine school learning and earlier opportunities. That does not make the score meaningless. It means that interpreting it as a pure measure of what the school produced would require an argument the score itself cannot supply.

Credentials carry information, but what information?

A credential is an institutionally recognized record of completion, qualification or attainment. A selector may use it because directly observing all relevant skills is costly. This creates a signaling possibility: the credential conveys information about a person that the selector cannot readily obtain another way. The information could concern acquired knowledge, persistence, prior selection or some combination.

Learning and signaling are not mutually exclusive. A training program can improve ability and award a certificate that helps others recognize the improvement. Conversely, a credential might influence selection even when the final requirement adds little learning. To distinguish the effects, we need variation that separates instruction from possession of the credential, or evidence about how employers respond to demonstrated performance and certification.

Return to the opening applicants. Equal scores on one practical exercise do not guarantee equality on every job-relevant dimension. The employer may believe the credential predicts something the exercise misses. That belief could be accurate, mistaken or a convenient justification for a familiar rule. An analysis must investigate its performance rather than infer validity from the fact that an employer uses it.

We should also distinguish a signal from a formal requirement. If an occupation legally or organizationally requires a qualification, an applicant may be excluded regardless of what a selector believes about their ability. The requirement's justification and effects then become part of the explanation. We are identifying possible arrangements here, not asserting the current rules for any specific occupation or jurisdiction.

Selection begins before the classroom

People who enter a program may already differ from those who do not. Prior preparation, resources, information, expected benefits and competing obligations can affect enrollment. If graduates later earn more than nonparticipants, some of the difference could reflect those starting conditions. Comparing the two groups without addressing selection cannot isolate what the program added.

Completion creates another selection stage. Those who finish may differ from those who leave, and leaving can occur for reasons unrelated to ability. A student might perform well but lose housing or need to provide care. Measuring outcomes only among graduates can make a program appear more successful by omitting participants for whom its demands proved difficult to sustain.

Consider a deliberately invented example. One program admits one hundred people and graduates forty; another admits one hundred and graduates eighty. If we report only the average earnings of graduates, we leave the completion difference out of view. That average may be relevant to a narrow question, but it cannot describe the experience of everyone admitted. A useful evaluation tracks the stages and states which population each result covers.

Application can itself be selective. A fee waiver might change who applies, altering the composition of admitted students even if the selection rule stays fixed. Comparing average outcomes before and after the waiver could then combine a policy effect with a changed population. The evaluation needs to ask whether it is studying applicants, admitted students, enrolled students or all eligible people.

A study that separates a diploma from nearby achievement

Clark and Martorell's 2014 article, The Signaling Value of a High School Diploma, uses the difference between students who barely pass and barely fail high-school exit examinations to investigate diploma signaling. Its published abstract reports little evidence of a signaling effect in that setting. The assigned reading is the publisher's abstract; we do not treat an earlier preliminary draft as the final article's complete methods or numerical results. Published study abstract.

The design's general logic deserves careful attention. Near a pass threshold, students on opposite sides may have similar underlying achievement while facing a difference in qualification. If the relevant assumptions hold, the threshold can help separate the consequence of the credential from a smooth relationship between scores and later outcomes. This is a regression-discontinuity strategy: investigate a jump associated with a rule at a boundary.

The boundary does not perform magic. Researchers must consider whether people can precisely manipulate their position around it, whether other rules change there and whether the outcome would otherwise vary smoothly. They must also establish how the threshold changes actual receipt of the credential. These are general requirements of the strategy, not claims that this chapter has independently reanalyzed the study's records.

The scope of the conclusion is local. Evidence near a particular exit-exam margin does not establish that university degrees, professional qualifications or all schooling have no value. It does not measure every possible benefit of learning. Nor does little evidence of an effect prove that the true effect is exactly zero. The useful lesson is how a carefully chosen comparison can target one mechanism more directly than a broad graduate–nongraduate earnings gap.

Resources can change participation and instruction

School resources are another possible mechanism, but the term needs unpacking. A smaller class, a stable teaching team, usable equipment, more instructional time and assistance outside class are different inputs. Increasing one does not guarantee that another improves. A budget total cannot by itself establish how resources reached learners or what changed in their experience.

Imagine a fictional evening course that purchases new equipment while keeping its laboratory open only during standard working hours. The equipment may be excellent, yet employed learners may have little opportunity to use it. Extending supervised access could matter more for them than another purchase. The mechanism runs through available practice time, not simply the monetary value of the equipment inventory.

Now suppose the course adds a tutor. The effect could depend on whether students know the tutor is available, can attend, receive appropriate help and use it on later tasks. Each step can interrupt the pathway. Recording the appointment and the final examination result leaves the middle of the process unobserved. An implementation account identifies whether the intended resource actually became usable instruction.

This is also why a disappointing result should not immediately become a verdict on the underlying idea. A program may fail because its theory was wrong, because it reached the wrong stage of the problem or because delivery differed from the plan. Conversely, faithful implementation does not excuse an ineffective theory. Evaluation needs evidence about both the process and the outcome.

Sorting can reorganize opportunities

Schools and training systems often sort people into classes, levels, subjects or routes. In a hypothetical system, students with similar current preparation might receive different assignments because one route has limited places. The assignments could then affect instruction, peers, expectations and later eligibility. Sorting can respond to prior differences while also contributing to subsequent ones.

The key causal question is what happens because of the assignment. A higher average outcome in an advanced class is not enough, since its students may have entered with stronger preparation. A credible comparison needs to address that initial difference. A lottery among eligible applicants, a threshold or a carefully justified alternative design might help, depending on how places are allocated.

Expectations introduce another possible pathway. If a label changes the difficulty of tasks offered or the feedback a learner receives, it can affect opportunities to develop. But observing that teachers expect more from high-scoring students does not establish that expectations caused their later scores. Prior performance could explain both. The pathway needs evidence about the response to the label beyond the information that led to it.

We should avoid assuming that all differentiation is harmful or all common instruction is fair. Different learners may benefit from different support, and a single route can conceal unequal access within it. The useful standard is whether the arrangement provides appropriate learning opportunities, how movement between routes works and whether the stated selection criteria match the educational purpose.

Denominators can reverse the apparent result

Here is a stipulated evaluation exercise, not reported research. Two hundred eligible applicants enter a fair lottery. One hundred receive an offer of additional training and one hundred do not. Eighty offered applicants enroll; forty in the other group obtain similar training elsewhere. Later, forty people in the offer group and thirty-two in the comparison group reach the specified employment outcome.

Using everyone assigned to each group, the outcome rates are forty percent and thirty-two percent: an eight-percentage-point difference. Under the lottery's assumptions, this comparison estimates the effect of being offered the program for this applicant population. It includes people who do not use the offer, because assignment rather than participation created the comparable groups.

Suppose instead we divide forty by eighty and thirty-two by forty and announce fifty percent versus eighty percent. The calculation silently assumes that every recorded success belongs to an enrollee, which our original information did not establish. Even if additional records confirmed that assumption, comparing only enrollees would select different subsets after assignment. It would no longer preserve the lottery's straightforward comparison.

This example exposes two errors at once: assigning outcomes to a subgroup without evidence, and conditioning on participation that the intervention can change. A per-participant question may be legitimate, but it needs additional assumptions and methods. Dividing by whichever denominator makes the result look impressive is not an evaluation strategy.

The eight-percentage-point estimate would also have uncertainty in a real finite experiment. It should not be presented as an exact population effect merely because the arithmetic is exact. We have stipulated counts to teach the comparison; estimating uncertainty and investigating missing outcomes would be necessary before making a real policy claim.

Costs need equally careful denominators. Suppose the fictional offer costs the program 100,000 units in total. Dividing that figure by forty successful outcomes gives 2,500 units per observed success, but many successes might have occurred without the offer. Using the stipulated difference of eight additional outcomes gives 12,500 units per additional outcome instead. This second calculation targets a different question and still omits uncertainty, participant costs and any other benefits. It should not become a general cost-effectiveness claim. The arithmetic demonstrates why the number of observed successes and the number attributable to an intervention must be kept distinct when judging what a program purchased.

Choosing outcomes changes the judgment

An education program can affect knowledge, completion, employment, earnings, confidence, social connections and participation in public life. These outcomes need not move together. A narrow employment measure may be appropriate for a job-training question, while remaining inadequate for judging every purpose of education. The evaluation should declare the outcomes that matter before selecting the most favorable result.

Timing matters too. A demanding course could temporarily reduce paid work while increasing later opportunities. A short follow-up might capture the cost and miss a later benefit; a long follow-up might lose track of participants or encounter changes in the surrounding economy. Neither horizon is automatically correct. The choice should follow the proposed mechanism and include plausible costs as well as benefits.

Distribution also matters. An average improvement can coexist with little benefit for learners facing the greatest constraints. Alternatively, a program may produce a modest overall average while substantially helping a defined group. Subgroup analysis can investigate this, but searching many groups after seeing the results creates opportunities for accidental patterns. A careful account distinguishes planned comparisons from exploratory findings.

The opening employer now looks less simple. A credential may reflect learning, selection, recognition and access, while its use can shape future opportunities. To explain the decision, we need evidence about what the qualification represents and how the selection rule operates. To judge the arrangement, we also need a standard: useful competence, equal access, efficient information or some combination. Evidence informs that judgment without choosing its values for us.

Application

Design a one-page evaluation of a fictional evening training course. Specify its intended learning, the population, an outcome, a comparison and two stages where participation could fail. Use the lottery example to explain why an offer effect differs from a comparison among participants. Then write a careful two-sentence account of what the diploma study does and does not establish.

Check your understanding: Graduates earn more than people who never enrolled. Does that difference identify the amount by which the program increased earnings, and does weak evidence of diploma signaling establish that education has no value?

Expected answer: Neither conclusion follows. Enrollment and completion can select people with different starting conditions, so the observed gap does not isolate the program's contribution. A local diploma-signaling result addresses one mechanism near a particular margin; it does not settle the effects of learning, other qualifications or all educational outcomes.