Dr. Katie Nielson delivers a talk on “Measuring the ROI of a Workplace Language Training Program” at ATD 2019.
A couple of years ago, we shared the framework Voxy developed to measure the success of our English courses along with real data illustrating that our approach is effective.
Since then, we’ve continued to use these criteria to demonstrate the success of our programs, and we recently attended the Association of Talent Development (ATD) 2019 International Conference & Expo where I was delighted to give a talk that explained the science behind how we measure results.
Here’s a quick recap:
Measure 1—Time-on-Task: Because learning a language is learning a skill, the single biggest predictor of workplace language training success is the amount of time learners spend practicing. Time-on-task is the total time a learner spends actively engaging with our learning materials—simply keeping the application open and doing other tasks in the background does not count!
Measure 2—Mastery of Language & Content: In addition to measuring how much time learners spend practicing, we also need to make sure they are learning. A successful workplace language program will improve learners’ overall proficiency and also give them the language they need to complete their job tasks. Measuring these things is key to demonstrating success.
Measure 3—360° Satisfaction: A great way to make sure a language learning program is working is by asking stakeholders, including learners, their teachers, and their supervisors. By asking stakeholders specific questions, we can measure exactly what is working and what might need improvement.
Measure 4—Real-World Success: Fundamentally, language is a tool we use to accomplish real-world tasks, so we should make sure that our learners are able to meet their goals. The true measure of a workplace program’s success is the number of employees who are able to complete their job tasks in English.
These last two metrics, stakeholder feedback and measuring real-world success, were how I concluded the talk at ATD. In June 2018, Voxy was able to survey 500 Brazilian learners affiliated with a large, multinational company. These employees had been taking English classes with Voxy for six months, and 57% of respondents reported that English was required for their jobs, with 87% indicating that they spoke English at work several times per week. See below for details on how much time employees were able to save as a result of English training.
Setting up a workplace language training program requires investment from learners, supervisors, and other stakeholders, so it goes without saying that it should work. By making sure that the program can report clear outcomes and that those outcomes can be used to measure ROI, talent development professionals are setting themselves—and their learners—up for success.
Katie is Voxy’s Chief Education Officer, which means she leads the teams ensuring that learners are getting the most efficient and effective educational experience possible. She has a PhD in Second Language Acquisition and years of experience teaching languages, building language courses, and evaluating the effectiveness of language training as a research scientist. She lectures and writes about all things related to language learning and educational technology.