Why Bother Asking Open-Ended Questions?

This week's newsletter is based on a true story that the Sopact Team encountered during a meeting with a large organization that runs multiple social programs. We wanted to share the topic with you because it is relatable to all of us who run surveys. We demonstrated our product's AI capability to analyze open-ended feedback and turn it into meaningful insights.

(Check the image below that's exactly what we were demonstrating) And suddenly, we were asked:

"Why would you bother asking open ended questions? It's so hard to analyze and you could just ask option-based question and get it over with."

For a long time, even we shied away from asking open-ended questions as analyzing them was a difficult, manual, and labor-intensive process. We used to recommend that our users go for standard option-based responses only to make our lives easier at the analysis stage.

However, we quickly realized that the problem was not asking open-ended questions but the manual and labor-intensive process of deriving insights from these responses.?

Today, we thought let's take the opportunity to explain to a wider audience why asking open ended questions is not only important but almost mandatory and how, in this day and age, no one has to dread the manual effort to derive insights anymore.

Let's take a hypothetical example: Say you conduct a resume-building workshop for people in your skill-building program so they can get better jobs and improve their quality of life as a result. To know your workshop's impact, maybe you ask a couple of questions on a survey:

To which you might get:

20% rate it with 1,

30% rate it with 3,

and 50% rate it with 5.

Yes, you have some numbers in nice charts. But:

1. You didn't get any insights.
2. You cannot use it to persuade other participants why they should join your resume-building workshop.
3. You cannot use it to improve your program and its workshops.
4. You didn't get any direction at all.

This is where open-ended queries outpace close-ended or option-based questions.?

Imagine you ask this instead:

Can you give us feedback on the resume building workshop and the impact it had on your job search and interview calls?

And you might get these kinds of responses:

The first response clearly states that the student benefitted long after graduating from the program with successful job switches. The second one states that it helped boost the student's confidence, which, as we know, can be the difference between someone succeeding and someone not. This nuance is impossible to capture in an option-based question, but this is the depth needed to truly understand the impact and help you make a program better, raise funds to continue running the program, or whatever the motivation.?

Now, by analyzing post-program feedback, we can see real insights. What were their experiences like after attending the workshop? Did their confidence improve? Did they land more or fewer interviews? What about the ones who did not attend?

If I have 100+ responses, how am I supposed to do this analysis? Our answer: This is what AI should be used for social good! For eight years now, our purpose at Sopact has been to keep up with technological advancements that can be implemented to make things easier for the social impact industry. Some good and some not. We know now that AI is here to stay, so why not put it to good use?? 100?responses or 1000, our AI-generated Thematic Analysis does what used to seem impossible (or, put more objectively, endless). Our new product, Sopact Sense, uses AI to identify patterns from these open-ended responses, and all you see are the insights you need to understand your program's impact better than before. Many of our first users gave this a go for their one-year survey for their undergrad scholarship program. They dared to switch from the typical option-based questions:?

"Select if you have experienced the following."

←To →

"In your own words, tell us about the impact of the scholarship in your life?"

And their discovery was huge after AI thematic analysis:

Not only were their participants mentioning?"financial relief",?"possibility to enroll", and?"possibility to graduate", which was expected. But 72% of them were also mentioning?"avoidance of debt". This came as a true surprise to them, as they had never anticipated that some of the students would be able to graduate free of debt. Which, in their own words, "is more impact than we had imagined". You can, too, make the switch and open yourself to open-ended questions and AI-thematic analysis.

Go to? Sopact Sense?to get started!

Get Started with Sopact Sense Click Below or Sopact Sense

Sopact Sense allows you to answer all of your questions before funders, directors, or partners ask them. Use your data and turn it into the insights you need to keep your programs relevant, useful, and impactful.