Part 2: Readability: If People Cannot Understand the Question, You Cannot Trust What the Answer Means

This is Part 2 of the 'Better Assessment Design Series:
Why Good Tests Are About More Than Questions and Scores.'
Click here for more in the series.

Most assessments produce scores.

But a score is only useful if the person understood what they were being asked.
That sounds obvious. Yet readability is one of the most overlooked parts of assessment design.

A poorly written question can make an assessment look more difficult, more technical, or more rigorous. But in reality, it may simply be harder to read.

That matters because when someone gets a question wrong, we need to know why.
Did they lack the knowledge?
Did they misunderstand the concept?
Or did the wording get in the way?

Good assessment design is not just about writing questions. It is about designing evidence. And evidence becomes weaker when the question is unclear.

Readability is a validity issue

Readability is often treated as a writing preference.
It is much more than that.
In assessment design, readability is a validity issue.

If the question is hard to understand, the result becomes harder to interpret. The assessment may no longer be measuring the intended knowledge, skill or judgement. It may be measuring reading ability, language confidence, test-taking experience, or familiarity with complex wording.

That creates noise in the result.

A simple example

Consider this question:

In circumstances where operational non-compliance has occurred in proximity to a controlled pedestrian environment, what would constitute the most appropriate behavioural response?

That may sound official. It may even sound professional.

But it is poor assessment language.

A clearer version would be:

You are driving near a school and see a stop sign. Why is it important to come to a complete stop?

The second version is easier to understand, but it can still test the same underlying idea. It asks the person to connect the rule with the reason behind it.

Clear language does not lower the standard.

It sharpens the assessment.

The person can focus on the concept, not the sentence.

Simple language can test complex thinking

There is a false belief that complex wording makes an assessment more rigorous.

It does not.

An assessment should be difficult because the concept, task or judgement is difficult. It should not be difficult because the question is badly written.

A question can use simple language and still test sophisticated thinking.

For example:
A patient agrees to a procedure but does not seem to understand the risks. What should happen next?

That is not a complicated sentence. But depending on the answer options, it can test consent, ethics, communication, judgement and procedural understanding.

The challenge is in the decision, not the wording.

That is good assessment design.

Practical advice: check readability before publishing

A practical way to improve assessment quality is to check readability before an assessment goes live.

Two useful measures are:

1. Flesch-Kincaid Grade Level

   This estimates the school grade level required to understand the text.

2. Flesch Reading Ease

   This gives a score from 0 to 100, where higher scores are easier to read.

These tools are not perfect. They do not understand context, technical meaning, or whether a question is valid. But they are useful warning lights.

For broad assessment audiences, a useful target is usually:

For Flesch Reading Ease, a broad assessment should generally aim for 60 or above, and ideally 70 or above when technical language is not part of the assessment construct.

A better question to ask

Before publishing an item, ask:

Could someone who understands the content still get this wrong because of the wording?

If the answer is yes, the question needs work.

Also check:

• Is the sentence too long? 

• Are there unnecessary technical terms? 

• Are there double negatives? 

• Is there more than one idea in the question? 

• Are the answer options easy to compare? 

• Could the wording be simpler without changing the standard? 

Good readability does not make an assessment easier.

It makes the evidence cleaner.

The person should spend their effort answering the question, not decoding it.

That is why readability matters.

Rudy Crous is a corporate psychologist and the co-founder and CEO of Compono, a people and culture platform that brings hiring, engagement, learning and credentialling together in one place, adding behavioural science to the processes most HR tools only track. 

Read the next article in the series.