Part 10: Assessment Design Is Decision Design

This is Part 10 of the 'Better Assessment Design Series:
Why Good Tests Are About More Than Questions and Scores.'
Click here for more in the series.

Assessment design is often treated as a technical task.

Write the questions.
Choose the format.
Set the pass mark.
Publish the test.
Report the score.

But assessment design is really decision design.

Every assessment result is used, formally or informally, to make a decision.

Someone has passed or failed. Someone is ready or not ready. Someone has understood or has not. Someone needs support, progression, certification, remediation or further review.

That means the quality of the assessment affects the quality of the decision.

Good assessment design protects the decision by making sure the evidence is valid, reliable, readable, fair, aligned and useful.

A weak assessment produces a score

A score can be useful.

But a score alone is not enough.

A person might score 80%, but what does that mean?

Did they understand the content?

Could they apply it?

Did they miss critical questions?

Were they weak in one area?

Did they guess several answers correctly?

Were the questions valid, reliable, readable and fair?

Without good assessment design, the score may create confidence that is not deserved.

That is a risk.

A strong assessment produces evidence

A strong assessment does more than produce a pass or fail result.

It provides evidence.

It shows what the person knows, what they understand, what they can apply, where they are strong, and where they may need further development.

That requires careful design.

These are not separate ideas.

They work together.

Bloom’s Taxonomy helps ensure the assessment tests the right level of thinking.

Readability helps ensure the person understands what is being asked.

Validity helps ensure the assessment measures what it claims to measure.

Reliability helps ensure the result is consistent.

Cognitive load helps ensure the difficulty comes from the concept or task, not confusion.

Item design helps ensure each question provides useful evidence.

Fairness helps ensure people are not disadvantaged by irrelevant barriers.

Alignment connects learning, assessment, evidence and action.

Data design ensures results can be interpreted and used.

Together, these elements create assessment quality.

Why this matters

Poor assessment design can lead to:

• people passing when they have not demonstrated the required standard
• people failing when they do understand the content
• learning gaps being missed
• unfair outcomes
• unreliable certification or credentialing
• lack of confidence in results
• difficulty defending decisions
• wasted investment in learning and assessment

Good assessment design does the opposite.

It helps identify what people know, what they understand, what they can apply, and where they need support.

It supports better decisions.

It also helps improve learning over time.

The pass mark matters too

A pass mark should not be arbitrary.

It should reflect the decision being made, the risk involved, and the required standard of performance.

A low-risk knowledge check may tolerate a different threshold from a high-stakes licensing, certification or safety assessment.

The more serious the consequence of getting it wrong, the more care should go into standard setting, item quality and evidence review.

Assessment design is therefore not just a testing activity.

It is a quality, evidence and decision-making activity.

Final thought

A weak assessment produces a score.

A strong assessment produces evidence.

And evidence is what we need if assessment results are going to support meaningful decisions.

Assessment design is not just about better questions.

It is about better decisions.

If the assessment is weak, the decision is weak.

But if the assessment is valid, reliable, readable, fair, aligned and well designed, it becomes more than a test.

It becomes a defensible measure of learning, capability, readiness or performance.

Rudy Crous is a corporate psychologist and the co-founder and CEO of Compono, a people and culture platform that brings hiring, engagement, learning and credentialling together in one place, adding behavioural science to the processes most HR tools only track.