Engagement and Efficacy

One fairly straightforward way to predict whether or not someone will learn a new skill is to measure how much he or she has practiced it. We see this with skills like playing sports or musical instruments and speaking new languages. Following this logic, if we wanted to establish the efficacy of a language learning program, we would expect to see that learners who completed more activities would improve their proficiency more, on average, than learners who completed fewer activities.

One metric that Voxy uses to measure the efficacy of its product is the Voxy Proficiency Assessment (VPA®), which is an objective, in-product proficiency instrument designed to measure learners’ general level of English. Each test assesses a learner’s grammatical knowledge as well as his or her reading and listening comprehension skills. Taken before learners begin using Voxy, the VPA® is used as a benchmark tool to determine each learner’s current level; after product usage, the VPA™ measures how much the learner has improved his or her proficiency. There are multiple forms of the VPA®, and depending on how learners self-assess their proficiency, they receive tests that are harder, easier, or for 0-beginners.

After a recent analysis of the engagement and performance of 870 learners, Voxy confirmed that increased Voxy usage is correlated with improved VPA® scores. The VPA® is scored from 0 to 1680, and it places learners into seven different proficiency categories, from beginner to advanced. On average, learners who only engaged minimally with the product improved their scores just 3.4 points. Learners who engaged in medium usage, which was determined by the median number of activities completed, improved their scores by 48.4 points, whereas learners who engaged the most improved their scores by 67.1 points.

On average, the VPA® scores for learners who completed at least 276 activities in a three-month period increased 11%. For learners who used the product for an additional three months, proficiency scores increased by 19% total. These data confirm the efficacy of Voxy’s language learning activities as well as Voxy’s earlier findings that completing at least 276 activities between VPA®s leads to consistent, improved proficiency.