Do companies need a language testing assessment partner, or can they develop and administer their own assessments in-house? What are the repercussions to attempting a homegrown assessment? If a company does choose to hire a language assessment partner, how would that company know they are choosing the right vendor? At Language Testing International (LTI), these are common questions we address with our clients. However, instead of taking our word for it, let’s hear from someone who actually has to make these decisions and who has chosen to work with LTI.
On a recent episode of LTIs “Language Is Your Superpower” podcast, special guest, Vinay Patel, PhD, discussed what he and his team at AT&T look for when choosing a language testing partner, and offered advice from his experience as to what companies may want to avoid.
To start, Vinay mentioned a common pitfall that many companies run into, which is to try to develop and administer language assessments in-house. “For somebody that doesn’t have the background,” Vinay explained, to determine “in what actually is proficient and what’s not proficient, if they’re not using standardized criteria, you could really be getting yourself into quite a bit of trouble. You could be assessing things that seem like they’re assessing language, but it could be full of bias. So really, the advice that I would give is to be careful with homegrown assessments. Really, if you’re trying to measure language proficiency, do your research. So, go online, see what language proficiency is, how it’s defined, what’s required within your company, all that kind of stuff. There’s a lot of things to look at here before you just dive in and make homegrown assessments.”
Vinay continues to say that “the good thing about some vendors is that they’ve done that work. It’s their bread and butter. They use standard measures of what proficiency is. They’ve done a lot of the research on the back end with different languages. So, it [working with a language testing provider] might be a lot easier than trying to develop something in-house.”
At AT&T, Vinay outsources their language assessments, but explains that not all vendors are operating at the same level of experience. He goes on to detail what he looks for when helping to choose AT&T’s language assessment partners. “First and foremost, I want to make sure that the tests they are using or have developed are reliable and valid. I want to make sure that data has been collected on how good they are at doing what they are supposed to be doing. So, really, they have to be demonstrated. The second step would be that there needs to be room for validating these assessments locally. So, I’d want data from the people that are applying at this company and see how the test functions, and monitor that as we move along.”
Another factor in selecting a language proficiency testing provider is the cost. “The other thing that is something to consider is cost,” Vinay says. “You want to make sure that the test ultimately is providing utility. So, if the costs are right where they are supposed to be and you are getting enough juice for the squeeze, that’s something that I would consider when choosing an assessment partner.”
Vinay concluded with the third criteria he looks for: “Time-to-hired is a very important metric. We want to make sure that our candidate experience is as positive as possible. I want to make sure that the scores are turned around quickly, so the amount of time it takes for us to know if a candidate is qualified or not qualified in a particular language has to be fairly quick.”
Additionally, Vinay mentions the employee benefit and experience, which is detailed in “the way that the candidate interacts with it and walks away from the assessment. I would want data around that; if they viewed it as a positive experience or if they viewed it as something that was unfair and not job-related.”
As the exclusive licensee of ACTFL, Language Testing International (LTI) works very closely with and administers ACTFL assessments. We know first-hand that it requires years and years of research, subject matter expertise, norming tests, and confirming they are valid and reliable. Having served over 5 million test-takers over the last 30 years, we know it’s not an easy process, but ultimately is an extremely important process to continue to try and get right. It is important not just for the companies who hire us as their language assessment partner, like AT&T, but also for the experience of the job candidates and employees who are taking those assessments.
