{"id":311,"date":"2014-02-22T19:25:05","date_gmt":"2014-02-22T19:25:05","guid":{"rendered":"https:\/\/www.languagetesting.com\/blog\/?p=311"},"modified":"2023-12-28T16:32:10","modified_gmt":"2023-12-28T16:32:10","slug":"using-real-world-tasks-to-assess-student-performance","status":"publish","type":"post","link":"https:\/\/www.languagetesting.com\/blog\/using-real-world-tasks-to-assess-student-performance\/","title":{"rendered":"Using Real-World Tasks to Assess Student Performance"},"content":{"rendered":"
\n
\n

\"\"Aimee Mack came to a quick realization last spring as she and colleagues pored over Connecticut\u2019s new teacher evaluation system: Their jobs would depend on being able to share evidence of their language learners\u2019 growth with the entire school community. \u201cWe knew that as a department we were going to be looking at data as part of our evaluation,\u201d says Mack, a French teacher and world languages team leader at Brookfield High School. \u201cThat made me wonder: Does a standardized test for world languages exist?\u201d She soon heard about something that seemed to fit the bill: the ACTFL Assessment of Performance toward Proficiency in Languages. Called AAPPL Measure or simply AAPPL, the online, performance-based test assesses students in the Interpersonal, Interpretive and Presentational modes of communication, with tasks that require learners to listen, read, speak, and write about topics commonly explored in language classrooms. Each task\u2014whether typing out an e-mail message, video chatting with a recorded native speaker, or making selections based on an understanding of something heard, read, or viewed\u2014occurs in the context of a Standards based classroom. Students can take one or all portions of the test; they then receive reports that classify their demonstrated level of performance and offer detailed suggestions for moving up. (Sample tests in Arabic, Chinese, French, German, Spanish, and Russian can be\u00a0found at\u00a0aappl.actfl.org\/demo<\/i><\/b>. More language\u00a0are being added, including English.) AAPPL\u2019s availability comes as states grapple with two interrelated, high-stakes challenges:<\/p>\n

<\/p>\n

(1) finding ways to evaluate teachers\u2019 effectiveness in the classroom and (2) identifying assessments that reliably gauge what students know and are able to do. This quest is especially critical given that many states weigh\u2014or soon will weigh\u2014students\u2019 standardized test scores heavily in teacher evaluations, and also because pressure has mounted to include traditionally non-tested subjects such as world languages and the arts in the assessment mix. \u201cWe\u2019re really seeing a confluence of things, from Race to the Top to the shift in teacher evaluations,\u201d says Craig Waterman, Assessment Coordinator for District-Determined Measures at the Department of Elementary and Secondary Education in Massachusetts, where at least three districts have begun using AAPPL. \u201cBut student learning can\u2019t just be (measured as) growth on traditional standardized tests. That\u2019s not the full range of what we do in schools. The majority of teachers are teaching in content areas that aren\u2019t currently assessed using standardized tests. Those teachers want feedback about what they\u2019re doing. The goal is to expand the range of what we\u2019re assessing, to cover the full range of learning that takes place in school.\u201d With such aims in mind, Mack piloted AAPPL last spring with 80 of her French students, who took a version of the test that assesses learners in the Novice-to-Intermediate range as described by the ACTFL Proficiency Guidelines 2012 and the ACTFL Performance Descriptors for Language Learners. (A second version targets Intermediate-to-Advanced learners.) Based on her positive experience, Mack and her colleagues had all their French and Spanish students complete the Interpersonal Speaking and Listening portion of the test this past fall. They plan to re-assess in the spring. Mack and instructors at other districts that have piloted AAPPL say the content and format of the test have fueled discussions about what needs to happen in their classrooms in order for students to achieve proficiency gains. AAPPL\u2019s proving grounds include Utah, where the state\u2019s dual language immersion students now take the test.<\/p>\n

\u201cWe know that instruction is always driven by what you test,\u201d says Gregg Roberts, the state\u2019s World Languages and Dual Immersion Specialist. \u201cStudents need proficiency targets so their language will be stretched. And we like having an external assessment because it\u2019s standardized and we can see how well our programs are doing.\u201d Language departments have experimented with a wide variety of assessments in recent years, from district-generated tests to commercial models. In 2001, ACTFL and other organizations worked with the Center for Applied Linguistics to create the framework for the Foreign Language National Assessment of Educational Progress (FL NAEP) which targeted 2003 as the launch year for a Spanish-language test for students in Grade 12. The design of the FL NAEP pushed the boundaries of large-scale testing of languages by including a simulated conversation and designing authentic tasks to assess Interpretive and Presentational modes of communication. Due to funding cuts, the FL NAEP was never officially administered. Out of that initiative came much of the research and thinking that ultimately shaped AAPPL, which was further enhanced by a FLAP grant and pilot testing in Glastonbury Public Schools and funding from several STARTALK projects. Language Testing International (ACTFL\u2019s official testing office) programmed a production version and built the Internet delivery, rating, and client reporting platforms.<\/p>\n

\u201cWe wanted to address the fundamental question of, \u2018What does language assessment look like?\u2019\u201d says ACTFL Director of Education Paul Sandrock. \u201cWe needed evidence of all three modes and it had to have an authentic feel. It was really the NAEP project that gave it a lot of push, that \u2018this is possible.\u2019\u201d After five years in development, AAPPL went live in January 2013. Since then, nearly 20,000 students have taken the test in approximately 130 school districts. As part of its effort to identify quality assessments for its own districts, Massachusetts hired San Francisco-based WestEd, a nonprofit focused on educational research and analysis since 1966. WestEd evaluated AAPPL as part of an examination of assessments in diverse subject areas to determine which aligned with state standards and could be considered for use by districts, says Carlos Camargo, Project Manager and Assessment Coordinator for WestEd. \u201cOne of the reasons that AAPPL came out ahead in reliability and validity is because ACTFL has strong assessment programs and has been testing this program with thousands of students across the country,\u201d Camargo says. \u201cThere were some new vendors that . . . lacked data on whether their assessments worked for different populations of students. There was also a great deal of information to support the interpretation coming out of the [AAPPL] test. At the end of the day, we think of assessment as an interpretation argument. It boils down to a number on a scale, and all the documentation has to support that number. AAPPL is a good measure that can say, \u2018This student has these proficiencies and competencies compared to other students.\u2019\u201d AAPPL provides information for the test taker, instructors, parents, and district leaders that helps all stakeholders see where individual and group learning stands, where gaps reside, and how to move students forward. That was an important consideration for school administrators in Minnetonka MN, where Spanish and Chinese immersion students in Grades 3\u20135 took the Interpretive Reading, Interpersonal Listening and Speaking, and Interpretive Listening portions of AAPPL Measure this past fall. The district\u2019s language teachers were familiar with the ACTFL Proficiency Guidelines, but they sought precision on their students\u2019 progress and district leaders wondered how students in different schools would fare, says Matt Rega, Director of Assessment. They were pleased, he says, when they read through the score reports a few days after tests were administered. \u201cWe found it was pretty consistent across buildings, with all (scores) falling within one sub-level on the proficiency scale,\u201d notes Rega. Minnetonka began using AAPPL this past fall as an alternative to homegrown assessments and a previous commercial test. From here on, the district plans to assess immersion students once a year in the spring as part of a districtwide drive to gauge progress toward grade-level proficiency targets. AAPPL offers a comprehensive, one-stop assessment that students say they enjoy and which allows teachers to use a common vocabulary of proficiency when discussing student work and progress,\u201d Rega says. \u201cTeachers have even created end-of-year targets where students can see if they\u2019re on track for achievement on [Advanced Placement] language exams. Students\u2014and their parents\u2014will know that if they continue to meet those targets that they\u2019re in good shape to score high on the AP exam by the end of ninth grade.\u201d Minnetonka\u2019s adoption of AAPPL reflects a growing wave of interest in language immersion programs in Minnesota and other parts of the country. Rega credits his own district\u2019s program with helping reverse a negative enrollment trend by attracting students from other districts. Seven years ago, he says, enrollment had dropped to about 6,600 students in Grades K\u20137. Now, six years after immersion came to Minnetonka, enrollment stands at 9,800, Rega says, with more than 2,200 students enrolled in the immersion program. In Utah\u2014a state that has become a national model of success for dual language immersion\u2014the use of AAPPL is an important component of students\u2019 proficiency-based learning, according to Roberts. Immersion students receive proficiency ratings on their report cards rather than grades, which parents have learned how to interpret at open house meetings. The state even brings in university professors trained in conducting modified oral proficiency interviews (OPIs) to check that students are on track to hit proficiency targets.<\/p>\n

Third-grade dual immersion students in French, Spanish, and Mandarin took the Interpersonal Speaking and Listening portion of AAPPL on a required basis for the first time last spring. \u201cOur elementary immersion program is driving our secondary program, as parents demand that the high school program incorporate proficiency targets as well,\u201d Roberts says. \u201cI always tell colleagues (in other states), if your state standards are proficiency-set, it\u2019s easier to get immersion schools in place and attract younger students. That gets the high school and secondary teachers to take notice, and they start to really create and design their curriculum around proficiency.\u201d For students and teachers, AAPPL hasn\u2019t produced the type of anxiety usually associated with standardized tests, Roberts says. \u201cTeachers see the benefits of having the (speaking) prompts as part of their instructions. And the kids love the test. They say, “Wow can we do that again?\u2019\u201d Roberts says. \u201cParents (love it), too. They say, \u2018Someone\u2019s finally telling me what my child can do with the language they\u2019re learning.\u2019 Many of Mack\u2019s students in Connecticut viewed AAPPL as \u201cjust something else we were doing in class,\u201d she says. Still, it was a different way of being evaluated. \u201cStudents had had a bit of one-on-one assessments with me, but the whole computer-based element was new to them. I like that the creators of the assessment go to lengths to make it look like (the native speaker) is listening. The kids are sometimes fooled: Is this person really seeing me?\u201d At a time of shrunken school budgets around the country, cost was a factor in Brookfield\u2019s decision to adopt a commercial test. AAPPL costs $20 per test taker for the complete, four-part assessment.<\/p>\n

The interpersonal portion alone, which is rated by certified evaluators, costs $10 per student and the presentational writing portion $5. The interpretive reading and listening portions are offered together for $5. Mack and her colleagues were asked to present\u2014and justify\u2014their wishes. For Mack, the sole French teacher at her high school, a key argument was the difficulty in finding an independent rater to evaluate large numbers of speaking samples. After hearing this and other teachers\u2019 input, school board members approved the Interpersonal Speaking and Listening portion of the test for all French and Spanish students. Brookfield took that action as part of an overhaul of its foreign language program, undertaken with guidance from Glastonbury Public Schools\u2019 world languages department. Glastonbury, which offers six languages taught by more than 50 teachers, played an important role in piloting AAPPL through work funded by a five-year FLAP grant. The district piloted AAPPL with high school students, recently added it as a benchmark measure for eighth graders, and may try it out at the fifth grade level, says Rita Oleksak, Director of the district\u2019s foreign language department. \u201cWe\u2019re looking at what the results tell us,\u201d says Oleksak, a former ACTFL President who was recently named NADSFL Supervisor of the Year. \u201cIt presents a good opportunity for teachers to think through, analyze, and interpret the data. For instance, what if you have a student who\u2019s great in class but only comes out at Novice High? And then you have a quiet student who comes out at Intermediate Mid. A teacher might say of the first student, \u2018But he talked and talked!\u2019 Then we have to ask, \u2018But what did he really say?\u2019\u201d Already, feedback from students who have taken AAPPL has led some teachers to change their strategies and emphases in class. Some of Mack\u2019s students, for example, said they struggled to recall some of what they had learned in earlier years of study. \u201cMany upper-level students said they wished they had remembered some older vocabulary; for example, they hadn\u2019t really had school vocabulary since French I.\u201d That, she says, made her \u201cthink about spiraling and always going back to reinforce the most basic skills. It also has forced me to focus more on speaking and reading. Part of my evaluation is: How can I incorporate the things that are emphasized in the test in my everyday instruction?\u201d Mack also has taken time to explain to students the value of the score report and the strategies for improvement that each includes. \u201cWe\u2019ve talked about [the Novice ratings of N-1 and N-2] and what each means. We had never had that conversation before. Before, it was, \u2018You\u2019re getting an A or a B.\u2019 This isn\u2019t about a letter grade. It\u2019s about improving. It\u2019s the idea that everyone is at a different level and all they\u2019re trying to do is go up a level. If my students achieve that goal when we do the spring assessment, that\u2019s information I\u2019ll want to share with people.\u201d Performance assessments such as AAPPL, with the emphasis on measuring what students can do with language, go to the heart of effective communication, says Camargo of WestEd, a devout linguaphile who is currently teaching himself Korean. \u201cLanguage is about communication and exchange. It\u2019s not information alone, but using information in a particular context to move things or other people. What AAPPL does is measure the efficacy of that exchange, and that\u2019s very novel,\u201d says Camargo. What\u2019s more, he adds, \u201cit focuses on what it takes to move along that continuum from Novice to Advanced. It answers this very basic question: \u2018When it comes to communication, can this person get the job done?\u2019<\/p>\n

\n

By Douglass Crouse\u00a0(contributing writer to The Language Educator). He also teaches French at Sparta Middle School in Sparta, New Jersey.<\/em><\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Aimee Mack came to a quick realization last spring as she and colleagues pored over Connecticut\u2019s new teacher evaluation system: Their jobs would depend on being able to share evidence of their language learners\u2019 growth with the entire school community. \u201cWe knew that as a department we were going to be looking at data as […]<\/p>\n","protected":false},"author":25,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[4],"tags":[80,16,71,48,7,87],"class_list":["post-311","post","type-post","status-publish","format-standard","hentry","category-educational","tag-aappl","tag-actfl","tag-actfl-nabe-tesol-biliteracy-language-testing-high-school-lti-testing-academic","tag-bilingual","tag-language-testing","tag-school-academic"],"acf":[],"aioseo_notices":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/posts\/311","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/users\/25"}],"replies":[{"embeddable":true,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/comments?post=311"}],"version-history":[{"count":3,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/posts\/311\/revisions"}],"predecessor-version":[{"id":2959,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/posts\/311\/revisions\/2959"}],"wp:attachment":[{"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/media?parent=311"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/categories?post=311"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.languagetesting.com\/blog\/wp-json\/wp\/v2\/tags?post=311"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}