March 9, 2016
Problem of Practice: How can we use interim assessment* data to better provide teachers and students with personalized learning resources?
Practical Solution: At this year’s Data Champion Summit, I presented a case study to show one way we answered the question above. Ultimately, I hoped to expand the ways we use data, and address not only the question of “what works?” but also “how can we improve the way we work it?”
We have to use data to meet students where they are and improve our use of adaptive online resources so we can reach higher, faster and better than ever before. Here’s an example of the data we started with; it comes from the MAP assessments we administer in the fall
Even when visualized, the data from the example MAP report above wasn’t much better. At best, it told teachers something they already knew and at worst it misled teachers who might assume that top quartile students were at grade level when they may be 1-3 grade levels above.
Khan Academy has great missions that are adaptive but also grade-level specific and not optimal for the reality of the fifth grade math class that has students ready ready for 1st grade, 9th grade and every grade in between.
Last fall, Khan Academy and NWEA published a PDF that changed everything. This document correlates MAP® sub-goals and RIT ranges to Khan Academy® exercises. It provided us with a kind of route map and potentially help a teacher or student better tailor the instruction to ensure the student got from where they were to where they should be by the end of the academic year. Like a route map, this PDF gave us the means to “see” the routes students could take from where they were to where we wanted them to be. This was really, really helpful. Before then, we would rely on grade-level missions to provide students with “appropriate” exercises. These problems, however, were not tailored to what students began the year knowing how to do. Instead, the aligned to the standards that students are expected to cover over the course of 5th grade.
A MAP score report had no way of “speaking” to the PDF of suggested Khan Academy exercises. While these were the means for creating personalized playlists, using them to do so for 100+ students was impossible. Even with unconditional hope, a teacher trying to use these resources to create personalized playlists doesn’t have unlimited time. In fact, they may only have 60 minutes of prep time each day, so I saw it as my responsibility to automate the process of creating personalized KA playlists.
Using the Data to Work it
Here’s how teachers use the tool we made.
Step 1. highlight & copy data from MAP score report
Step 2. Paste data into tool we made & review auto-generated playlists for each student. Students names are highlighted to indicate the exercises they should begin with. All students begin with exercises that match their assigned color.
The first feedback we received was that it was amazing to see the grade level that students were ready for. This was a highly unanticipated benefit. Since Khan exercises are tied to standards and each standard is grade-specific we were able to easily translate MAP scores into grade-level equivalency** teachers could more easily see the grade level that students were ready to learn at. This playlist tool literally gave teachers a better picture of their students’ proficiency and a clearer sense of the fact that teaching fifth grade math and teaching math to fifth graders are not the same thing.
Data driven improvement =
evaluating what works + improving how we “work” it
* 3 big types of assessments:
- Summative: Tests used for end-of-year accountability and evaluation
- Formative: Educator resources that support measuring student learning in real-time during instruction
- Interim: Tests used for timely and periodic information based on local needs/goals
**Grade Equivalent scores range from K-12. They represent how a student’s test performance compares with that of other students nationally. For example, if a 5th-grade student has a GE of 7, the student’s score is equal to that of a typical 7th grader.
February 11, 2016
When it comes to adaptive learning resources, we should turn our focus from how they change. in response to student learning to how they change the student’s learning trajectory.
For students in grades 3-9, we use data from adaptive assessments to predict their 12th grade ACT scores. This helps us ensure they are on the path to and through college…
While our data team helps schools to use adaptive assessments to make predictions
my team, the innovation team, uses adaptive resources to break them…
There are several ways that we use adaptive assessment data to inform and improve adaptive content…
We used data from adaptive assessments to ask “What works?” and discover how our students and teachers worked it…We use adaptive assessment data to go beyond fidelity of use and define excellence in use to ensure all learners optimally leverage adaptive content. Most recently, we began using knowledge of adaptive content to designed a tool that could inform use of online content school-wide instructional design
February 11, 2016
“For every ounce of technology, you need 10 ounces of humanity.” ~ me
These we’re some of the advice I shared at last week’s SF EDTech Meetup, which was focused on adaptive learning. The link below goes to podcast of the ignite talks that I and six others gave.
For every ounce of technology, you’ll need 10 ounces of humanity. ~Me
February 11, 2016
The students entering our classrooms come from more diverse backgrounds and bring a wider set of needs and abilities than ever in history. At KIPP Bay Area Schools, we use assessment data to help us determine what our students know and what they are ready to learn.
Being a 5th grader, doesn’t mean you are ready to learn 5th grade math. The graphics below show the reality of trying to teach 5th grade math. On the left, you’ll see a graphic that shows the grade level they are ready to learn. On the right, is shows what will happen when you teach fifth grade math to the entire class-many will be underprepared, some will be ready and the rest will be very bored.
These graphics could represent a majority of our nation’s 5th grade classrooms, and illustrate a universal truth:
Planning to teach 5th grade math, and preparing to teach the 5th graders in your math class are not the same thing.
Planning to teach 5th grade math is like running a 5k… You do your best to stay on the path and at an optimal pace. Preparing to teacher every student in your 5th grade math class is like running a Tough Mudder.
April 9, 2014
The Promise of #MyData
On January 15th, the government announced a Request for Ideas (RFI) for a new FAFSA. The next iteration of the Federal Financial Aid Application could be an app that’s essentially Turbo Tax for filing financial aid to ensure that students get the maximum financial aid offer and help make sure there are no mistakes nor missed deadlines. The My Data initiative recognizes that schools, companies, and government agencies have databases containing the information a student needs to populate the blank spaces in a financial application form.
The number of students failing to file the FAFSA each year has remained flat for over a decade. Of the students who did not apply for financial aid from any source, almost all (95.3%) gave at least one of five reasons for not applying: thought ineligible (60.7%), no financial need (50.6%), did not want to take on the debt (40.2%), no information on how to apply (22.9%) and forms were too much work (18.9%). The first three reasons accounted for 92.2% of the non-applicants. Many of these students, however, would have qualified for financial aid. About a third of these students, for example, would have qualified for a Pell Grant and about a sixth for a full Pell Grant. At least 1.7 million students fail to file the FAFSA each year because they incorrectly believe themselves to be ineligible.
Currently, our financial aid application process is shameful injustice. We as a society are morally obligated recognize that students have a right to their data. Furthermore, they have the right to give others permission to use the information to automatically calculate or update their financial aid offer. I have seen how my students’ information is automatically used to notify them of their obligation to register for the draft, but I have longed to see it used to notify them of their financial aid eligibility.
December 27, 2013
Motivational focus affects how we approach life’s challenges and demands. While everyone is concerned at various times with both promotion and prevention, most of us have a dominant motivational focus, which affects what we pay attention to, what we value and how we feel when we succeed or fail. As educators, we have the pleasure of working with prevention and promotion-focused students, and those who successfully speak the motivational language of both have shown remarkable results. (1)
While the classroom management style prescribed in Doug Lemov’s Teach like a Champ is highly suitable for prevention-focused pupils, the promotion-focused will thrive when teachers have management styles that support creative solutions, long-term visions, and opportunities to shake things up. (2)
Many of my students had jobs and most of their parents had more than one. Those jobs, however, are insufficient means for providing promotion-focused students with inspiration. To clarify, a job and a career are not the same. A career is defined by the OED as an individual’s “course or progress through life.” The etymology of the term comes from the 16th century French word Carriere, which meant “road” or “racecourse.” In contrast, a job is “a regular activity performed in exchange for payment.” If you still don’t see the distinction, Chris Rock can elucidate further.
Some don’t have to leave the kitchen table to see people with fulfilling careers, these privileged individuals were not the students I worked with. Along the bus-ride or walk home from school each day, my students saw insufficient examples of successful careers. Regardless of their motivational focus, people will always dream about their future. But those dreams, and the will to realize them is limited in “dream deserts” where access and opportunity are unjustly stifled. (3)
“Have a Dream I said” – The language promotion-focused students need to hear
Lacking sufficient examples of career opportunities in the formal economy, my students were often vocal about their motivations to enter the informal economy. I took their insights seriously, and created an entire unit to address them. This unit began with an excerpt from Jay-Z’s Decoded titled “Have a Dream I Said”. It began with the a simple question: What’s the basic motivation for a hustler?
The kid in McDonald’s gets a check and that’s it. There’s no dream in fast food. Manager? That’s a promotion, not a dream…The truth is that most kids on the corner aren’t making big money–especially if you break their income down to an hourly wage. (footnote from freakonomics) But they’re getting rewarded in ways that go beyond dollars and cents. The kid on the streets is getting a shot at a dream.
This vignette spoke of my promotion-focused students’ profound capacity to channel their energy towards a compelling victory. Moreover, it undermined mythologies of the informal economy. As teachers, we must speak the language of promotion-focused students daily. We must provide them with more vivid, inspirational examples of success and a map that will lead them to it. Lastly, we must recognize the small wins that they make along the steep path to victory. These celebratory acts will make the ultimate dream seem even closer, worth more and more risks we will ask them to take.
(1) One study found that students who received instructions suited to their dominant motivational focus were about 50% more likely than others to turn in their reports.
(2) Prevention-focused students were found to thrive with strategies emphasize rules and standards, and under leaders who tend toward micromanagement, discourage errors, and focus on effectively reaching more-immediate goals.
(3) In the world of nutrition, urban communities are often described as “food deserts”. I’d like to encourage today’s geography majors to map “dream deserts” and identify such as any geographic area with unjustly low opportunities for meaningful careers.
December 22, 2013
It seems absurd that we have smartphones to help us navigate our way to a destination, yet nothing to provide students with turn-by-turn directions to get them from where they are to where they want to be. What if we had a way to clearly show them where they’re headed should they continue in the same direction? What if we could also show them how to get back on track should they get lost?
Granted, the problems in public education are not easily solved, if the were we wouldn’t still be looking at them. I will be one of the first to acknowledge that not every student has an equal privilege of choice. I know all too well the very real reasons that some of my students could not “choose” to show up to first period.
The systems of power and privilege are inequitable, but this fact can’t substantiate a delay in giving students the right to use information that could help them make the right choices. Students must be given the power to access, understand and use their own academic data. Students must be given the right to see their own data. With their own academic data, a student could utilize tools that tell her the probability of attending a 2 year or 4 year college. Moreover, it could give her directions to improve her chances of getting to and through post-secondary graduation. These tools would help a student visualize where she is headed and what it would take to arrive at a different destin
Student Rights in the Digital Age – The Right Choice
We need to upgrade the Students’ Rights Handbook. While the ACLU is rightly focused on a student’s right to privacy they aren’t fully articulating the equally important rights related to a student’s access to and control of his data. In February 2012, the White House issued a framework for protecting privacy and promoting innovation in the global digital economy, which clearly articulated guidance on what citizens should expect from those who handle their personal information, and set expectations for companies that use personal data. Rights related to access, transparency and control were of particular interest, and inspired me to appropriate the articles below:
Access: Consumers Students have a right to access their data in usable formats.
Individual Control: Consumers Students have a right to understand and exercise control over data that companies collect from them and how they use it.
Making the Right Choice
There are countless others committed to redefining the potential of our nation’s public education system. What if we came together to draft a blueprint for student rights in the digital age? Together we
could will create a dynamic model that offers strong protections and enables ongoing innovation in new education technologies. When the U.S. Government released weather and GPS data to the public, it fueled an industry that today is valued at tens of billions of dollars. What if we gave students a right to their own data? What if innovators used that right as an opportunity to build a tool for students? And what if this tool told every student not only where they are, but how to get where they want to go?
An app won’t fix our nation’s inequitable public education system. Giving students the right to their own information, however, is a good step towards giving them the power to make the right choice.
February 7, 2013
Over the past few years, the onus for technology purchase decisions has shifted from superintendent, to principal to committee and teachers themselves. Tonight, I propose we consider what the next step will look like, what it will mean to make these investments truly student centered. Since working at McNair Academy, here’s what I and other instructional technology teachers have developed:
1. The purchasing decisions should be based on observations and reports gathered during pilot studies with sub-groups of students who are targeted for support.
2. The introduction and instruction that accompanies district or school-wide roll out is created by students. Teachers are invited into the classrooms where sub-groups are piloting the programs. These teachers can actively participate in the pilot and learn/develop best practices for using new technology.
3. The technology of tomorrow is shaped during today’s lunch period. For realz! When students get chance to meet with entrepreneurs and beta test their programs and products over lunch its a positively transformative opportunity for everyone.
January 28, 2013
Kentaro Toyama is a researcher in the School of Information at the University of California, Berkeley. In “There Are No Technology Shortcuts to Good Education” , Toyama drew on the work of Larry Cuban and Todd Oppenheimer to draw attention to a “repetitive cycle of technology in education that goes through hype, investment, poor integration, and lack of educational outcomes.”
To escape the cycle, we must subject ourselves to fairly merciless self-examination that prompts reinvention of the methods by which we endeavor to achieve our goals. This relentless self-awareness is discussed in forthcoming book “The Art of Doing: How Superachievers Do What They Do and How They Do It So Well” by Camille Sweeney and Josh Gosfield. By engaging in this practice we can ascertain the limits of our current methodology and allow ourselves the opportunity to change course. Thusly, can we create space for iterative feedback necessary for effecting positive innovation in schools.
At the middle school where I work in East Palo Alto, we’ve created a student oriented cycle for the procurement, development and integration of instructional technology. Here’s the first of many lab reports to come out of McNair Middle School where we’re taking on mission of being an incubator of best practices for teaching with educational technology. The reports will put forth our objectives, assumptions, trials and lessons learned.
Objective: Reach the “reluctant learners”
Reluctant Learners: 5% of students who are not making progress with online reading and math programs that were adopted by the school.
Goal: adjust programs that the school currently subscribes to, or find new programs that would engage these students in learning.
1. The assessments used by programs that the school currently subscribes to provide inaccurate measurements of the students’ abilities.
2. The assessments used by programs that the school currently subscribes to provide correctly measured student’s abilities but prescribed curriculum that was “hella bootsy” (too slow in its presentation and progression).
3. The units or lessons were too lengthy and left students feeling frustrated that they’d put in so much effort but not been able to complete the lesson during class period. This frustration often carried over to the next day and made them reticent to re-engage with the program.
4. The “hints” or “lessons” provided were entirely text or altogether missing. Either way, the students were not able to learn what they needed to complete the assignment/level and left feeling “stupid”.
We need to try other programs!
Criteria for new programs we would pilot:
+ Must be fast paced units that were less than eight minutes in length.
+ Preferably, these programs would come with paper-based activities to complete by hand using information from online program. (While paper handouts are often criticized as being passe at worst and environmentally unfriendly at best, students felt more accomplished when they could hand in a piece of paper with completed work. Additionally, many of these students had difficultly sitting still for long periods of time and completing the handouts with pen or pencil provided opportunity for movement.)
+ Program can’t make these students feel like they are dumb or using something that is used by students in lower grades. No student should look at it and say “this thing’s for babies!” / “Do you think I’m dumb?!”
+ Gamification – I don’t care how you define “gamify” but the more it looks like a video game the less likely they were to hate it.
The result of the work was tremendously positive. By piloting different programs with these students we were able to find two that were tailored to their specific needs and capable of delivering lessons that enabled them to develop deep understanding of new content and skills. The information was passed along (bottom -> up) and informed district level decision makers of the need and importance of investing in programs that were working for these students. We were able to invite teachers into our classroom setting to observe how we introduced and used these programs with these students. In this setting, teachers were actively engaged in learning and able to see the positive value of adopting and introducing these programs.
January 13, 2013
See life’s a beautiful struggle, I record it / Hope it helps you maneuvering through yours
Last summer, the voices of teachers kicked down doors. From now on, you’ll need a teacher on board if you want to be taken seriously in the education innovation industry. This is an accomplishment, but not the ultimate victory for teachers who are on a mission to bridge the digital divide. Our accomplishments thus far are a fantastic starting point, and we should be proud of how far we’ve come.
Now, this is my job, I will not quit it / Pulled me out the depths when I thought that I was finished / Yeah I questioned if I could go the distance…
By the end of the summer, I seriously considered taking a sabbatical from teaching. But in the final round of interviews at an educational technology company, a maverick CEO inspired me to take the time I needed to figure out what I really wanted. Here’s the note I sent myself after that meeting:
In September, I moved out to the bay area where I’d have the best opportunity to teach and build the fellowship at the same time. I had no job waiting for me, no apartment to live in, and no family. But a teacher on a mission won’t be satisfied, so I guess that’s the sacrifice…
I was meant to be a warrior / Fight something amongst me, leave here victorious…
Despite having no job, apartment or family out here, I moved to the bay area because of an idea that I could create a fellowship. This fellowship would enable educators to be involved in high level decision making and as leaders in shaping innovation and relevant policy. I am here because I want to transform the “teacher approved technology” credential from a stamp of approval into a powerful and permanent platform from which the collective voice of a movement will be heard and headed. So, I’d start by making a plan and an MVP.
Speaking with leaders in education field helped me flesh out a first draft vision of the fellowship, but I was paralyzed whenever I attempted to create a business model for it. Don’t get me wrong, when it comes to identifying potential revenue streams, I can make it rain. In this case, my paralysis came from wanting to teach and develop an MVP at the same time. Luckily, this powerhouse network of teachers helped me find the teaching position in which I’ve had the opportunity to teach, beta test, and create a working MVP model of the fellowship over the past two months.
Make the money, don’t let the money make you / Change the game, don’t let the game change you / I’ll forever remain faithful…
So the journey continues. Inspired and informed by the past, I look towards everyday between now and next summer’s first iteration of the fellowship. I look forward to first period with my students tomorrow and the “office hours” I’ll be holding with entrepreneurs on the Caltrain as I make my way to and from school. Lastly, I look forward to having the time to write down the thanks that I owe to each and every person who’ve helped me get this far.
New Opportunity & New Office Hours;
An incubator for best practices;
I-Ready and Rosetta Stone, Meet me at camera 3!