Get out of the way.

Melanie Helmy
Jun 16
2 min read

The impact of open verbatim questions within research is a hotly researched topic, especially in the field of behavioural psychology. Most of this work however focuses on the actual value, or lack of as the research has consistently proven, of the response itself. Psychologists have studied the topic for decades and all agree that humans are poorly equipped to explain their decisions. In what is widely accepted as post-rationalisation bias, the effects do vary around this phenomena with three main areas explaining this inability of the human race.

As interesting as this all is, it doesn’t seem to ‘cut the mustard’ from a commercial research perspective. Sure, we know that the open verbatim is going to be a fanciful story with very limited introspection and at times, an out-right guess on the consumers part in answering the question but what harm is that in the scheme of things?

Our hypothesis that we set out to test was that there is in fact some harm in these seemingly innocuous open verbatim questions. We created a test designed to research the research with a clear objective of "does including an open verbatim question within a survey negatively impact the data obtained on subsequent questions within the survey design?" As with so many research projects, the clear objective, morphed into a secondary objective of "is there a difference between spontaneous feedback and post-rationalisation verbatim responses?"

Utilising two different research methodologies, we tested our hypothesis and analysed the resultant data across three key criteria: data integrity & reproducibility, data quality & answering patterns and respondent enjoyment & engagement.

The short version response (ideal for closing statements in a blog post) is that both the spontaneous feedback and post-rationalisation verbatim responses had a detrimental impact on the resultant data. The long version response? That is within our latest white paper available for download below. This is a preview to what will be presented at The Research Society conference in September 2025.