March 9, 2016
Problem of Practice: How can we use interim assessment* data to better provide teachers and students with personalized learning resources?
Practical Solution: At this year’s Data Champion Summit, I presented a case study to show one way we answered the question above. Ultimately, I hoped to expand the ways we use data, and address not only the question of “what works?” but also “how can we improve the way we work it?”
We have to use data to meet students where they are and improve our use of adaptive online resources so we can reach higher, faster and better than ever before. Here’s an example of the data we started with; it comes from the MAP assessments we administer in the fall
Even when visualized, the data from the example MAP report above wasn’t much better. At best, it told teachers something they already knew and at worst it misled teachers who might assume that top quartile students were at grade level when they may be 1-3 grade levels above.
Khan Academy has great missions that are adaptive but also grade-level specific and not optimal for the reality of the fifth grade math class that has students ready ready for 1st grade, 9th grade and every grade in between.
Last fall, Khan Academy and NWEA published a PDF that changed everything. This document correlates MAP® sub-goals and RIT ranges to Khan Academy® exercises. It provided us with a kind of route map and potentially help a teacher or student better tailor the instruction to ensure the student got from where they were to where they should be by the end of the academic year. Like a route map, this PDF gave us the means to “see” the routes students could take from where they were to where we wanted them to be. This was really, really helpful. Before then, we would rely on grade-level missions to provide students with “appropriate” exercises. These problems, however, were not tailored to what students began the year knowing how to do. Instead, the aligned to the standards that students are expected to cover over the course of 5th grade.
A MAP score report had no way of “speaking” to the PDF of suggested Khan Academy exercises. While these were the means for creating personalized playlists, using them to do so for 100+ students was impossible. Even with unconditional hope, a teacher trying to use these resources to create personalized playlists doesn’t have unlimited time. In fact, they may only have 60 minutes of prep time each day, so I saw it as my responsibility to automate the process of creating personalized KA playlists.
Using the Data to Work it
Here’s how teachers use the tool we made.
Step 1. highlight & copy data from MAP score report
Step 2. Paste data into tool we made & review auto-generated playlists for each student. Students names are highlighted to indicate the exercises they should begin with. All students begin with exercises that match their assigned color.
The first feedback we received was that it was amazing to see the grade level that students were ready for. This was a highly unanticipated benefit. Since Khan exercises are tied to standards and each standard is grade-specific we were able to easily translate MAP scores into grade-level equivalency** teachers could more easily see the grade level that students were ready to learn at. This playlist tool literally gave teachers a better picture of their students’ proficiency and a clearer sense of the fact that teaching fifth grade math and teaching math to fifth graders are not the same thing.
Data driven improvement =
evaluating what works + improving how we “work” it
* 3 big types of assessments:
- Summative: Tests used for end-of-year accountability and evaluation
- Formative: Educator resources that support measuring student learning in real-time during instruction
- Interim: Tests used for timely and periodic information based on local needs/goals
**Grade Equivalent scores range from K-12. They represent how a student’s test performance compares with that of other students nationally. For example, if a 5th-grade student has a GE of 7, the student’s score is equal to that of a typical 7th grader.