A year of hard work, going through millions of questions and trying to find a logic and a read thread is finally coming to an end with the launch of the Question Library, available in English (live as of 6th July).
The Question Library is the result of natural language programming (NLP) analysis of 90,000 previously run projects in Ipsos, with 570,000 unique questions in English. The library is part of the Ipsos.Digital FastFacts product which enables the user to create questions from scratch or tap into the Question Library and choose the relevant questions. The questions are correlated based on the topic of interest expressed by the client in the set-up phase and previous correlation between questions analysed on the 90,000 projects.
Anders Arpteg, Chief Scientist at Peltarion, a Swedish AI company we have been collaborating with on this endeavour, explained the complexity in the task at hands:
"The beauty of working with Ipsos is that you have loads of more or less structured data. In this case we were exposed to 500 GB or raw text data, with limited common denominators. We used BERT from Google to first of all find patterns in the questions, then to extract information that can be piped in, such as the action verb, brand name, the category or the time span used in the question. Based on the meaning of these questions, we created a Similarity algorithm, where the questions
- How many cups of coffee do you drink per day - is similar to
- Did you have a warm drink yesterday? - even if not a single word is the same. "
In order to validate the algorithm and improve the performance, we took the help of 64 Ipsos researchers around the world, each validating the suggestions given by the algorithm thus allowing the algorithm to self-improve. We thank all these researchers deeply!
"The next step in the Question Library development is to expand it to many more languages and we are now already engaging with clients to also build proprietary question library based on their specific phrasings, adapting the algorithm to the questions asked by some of our largest accounts." says Andrei Postoaca, CEO Ipsos.Digital
Question Library will become a strategic piece for Ipsos as it will add more features and enable users to:
- choose the best wording for specific questions
- harmonize questions' wording across different groups
- breaking down each question into its components (action verb, category, brand, timeframe, etc.) and enable instant adaption to your survey: "How often do you buy yogurt in a month?" to "How often do you drink coffee in a day?"
- automate custom survey creation through easy access to questions based on predefined taxonomies and question tags
- build predefined reporting templates by question type