Actionable Learning from Educational Research - Part 1

As a part of the professional development program at Khan Lab School, we spent a term in 2023-24 reading research papers related to our teaching and learning practice. The rule was that the papers needed to have been published within the last five years, and we needed to identify at least one actionable takeaway from each.

Articles:

The key characteristics of project-based learning: how teachers implement projects in K-12 science education (Anette Markula & Maija Aksela 2022)

Making Sense of K-12 Competency-Based Education: A Systematic Literature Review of Implementation and Outcome Research from 2000-20219 ( Evans et al. 2020)

The need to disentangle assessment and feedback in higher education (Naomi E. Winstone and David Boud 2021)

---

The key characteristics of project-based learning: how teachers implement projects in K-12 science education (Anette Markula & Maija Aksela 2022)

Major Takeaway: To create hands-on, authentic projects, a unifying theme can be helpful.

Outstanding Questions:

• How do our students interact with “projects” in classes, applied learning projects, etc?

• How do teachers interact with a central theme?

• How does the central theme become visible or relevant for students?

Actionable Learning: We should create a theme for next year (Our Local Community) and develop interdisciplinary projects in small professional learning community groups.

Notes: Things that often make PBL fail include poor articulation of the public product (in this paper, it’s called the “artifact”) and the driving question

• Things that teachers struggle with: managing long-term projects and assessing group work

• Learning from robotics:

  • Students are naturally motivated by the end goal of a competition

  • Students can identify the academic content ingrained in the process

• Driving question are difficult to develop because there’s a sense that it needs to cover a priori designed content. 

  • What makes a good driving question?

What is the purpose of PBL?

• To make learning fun? That should be a side effect!

• To make learning relevant - that's the important takeaway

---

Making Sense of K-12 Competency-Based Education: A Systematic Literature Review of Implementation and Outcome Research from 2000-20219 ( Evans et al. 2020)

Major Takeaway: The concept of “flexible pacing” that aligns with CBE implies a minimum pace (with reasonable deadlines).

Outstanding Questions: What makes some deadlines flexible and some rigid? What are the benefits (and drawbacks) of more rigid deadlines?

Actionable Learning: Develop a school-wide policy around late work that is manageable for teachers and mission-aligned.

Notes:

What surprised you about the paper? What stood out?

• 3 R’s (relevance to the organization and its people, readiness to undertake the innovation, and resources to implement) are necessary for success (pg. 17)

• Seems strange that CBE and mastery learning is both not that new, and not that well-studied. The paper makes it seem like implementation has been spotty and inconsistent, and therefore hard to reliably measure outcomes. This also makes it hard to evaluate the effectiveness of CBE, generally.

• What does proper implementation look like? What training do teachers need?

• It seems key to align assessment strategies to the teaching practices - but since the teaching practices focus on personalized learning, the assessment needed is by definition, not standardized. This makes evaluation difficult if not impossible!

• Let’s not throw the baby out with the bathwater - what is “traditional education” as defined by the paper, and what useful practices can be extracted from it?

• How many studies focus on STEM rather than humanities? What research is available that focuses on humanities courses and the implementation of CBE?

Where did you see connections to our context at KLS?

• The paper uses the phrase “flexible-pacing” instead of self-pacing. Self-pacing requires students to be the arbiters of the appropriate pace, which requires them to be able to self-diagnose. “Flexible pacing” implies a minimum pacing target that stretches to encompass most learners. Questions are left for the teachers about how stretchy that pace should be. 

• Stable definitions of terms are one of the keys to success - where and how should we document our definitions?

• As the school grows, we should remember that the flexibility we have to provide students with personalized learning plans requires small class sizes - so the faculty body has to grow alongside the student body. (question: (how) has CBE been implemented at larger public schools?)

What roads forward did the paper seem to suggest?

• They suggest developing a “continuum” that spans from “traditional teaching” to the most radical framework you could imagine. There are many useful frameworks on that continuum that we could pull from if we could define them and what teaching practices and administrative policies go with them. This sounds like a large research undertaking

• Definitions of terms and curriculum should not be tied to a teacher, but should live within the school and be easy to access.

• We should lean on our relationship with B21 to help develop a stable framework for assessment

• We should commit to an innovation for more than 1 year before measuring outcomes

• We should develop a mechanism for faculty to share what’s working and what’s not in their classrooms.

---

The need to disentangle assessment and feedback in higher education (Naomi E. Winstone and David Boud 2021)

Major Takeaway: Students don’t read feedback when it is delivered alongside assessment.

Outstanding Questions: How can we provide holistic, thorough, timely feedback without our teachers feeling underwater? How can peer learning be leveraged to provide quality feedback? How can self-reflection be integrated into the feedback process? What are the best practices for drafting and revision?

Actionable Learning: Create Learning Journals and build in time to the schedule for teachers to meet with students and talk through their work in one-on-one meetings. Consider and research portfolio grading systems.

Notes:

This paper focused on the ways in which feedback (directed dialogue with students in which they gain insights into their own work and a pathway to develop their skills further) is often conflated with assessment (a grade given at the end of a term or course). It made the argument that when assessments (grades) are given alongside feedback (e.g. narratives), students often ignore the feedback and focus on the grades. The delivery of this feedback at the end of a term also prevents students from using it in an actionable way. Teachers writing narratives often direct their writing towards the parents reading the reports, trying to avoid conflict by explaining the grade, rather than entering into a conversation with the student.

This article resonated with many of us, and we discussed potential solutions to the problem:

1) Soliciting Feedback: Could feedback be more meaningfully integrated into the coursework itself? The paper suggested having students write cover letters to major assignments in which they ask for feedback on a specific learning outcome. The focused feedback they receive from the teacher is then an answer to a question, rather than unsolicited and unwanted. This would also mimic the peer review process that research journals use - they require resubmissions to come with a cover letter explaining how feedback was incorporated into the revision.

2) Focusing Feedback: Feedback should be parsable for students - rather than redlining an entire paper or providing complex feedback on a wide variety of learning outcomes, feedback should focus on one thing at a time, ideally in response to student-defined goals.

3) Feedback as a Dialogue: We discussed the possibilty of co-writing narratives alongside students, so that they are the first people parents ask when they want to understand a grade. Following a mastery approach, students could present a portfolio of work to their teacher, and in a one-on-one conversation, they could decide what grade the work has earned, and craft the narrative together before the report card goes out.

We then talked a bit about what solution #3 would look like, and what logistics would need to be figured out:

• How would we make time for this in the schedule? We'd need a "portfolio week" where there were no classes, but only one-on-one meetings. At the end of each meeting, the grading for that student would be done, so we'd have a clean deadline for all teachers to be finished with grading (and it wouldn't bleed over into the break).

• Students would need to do some pre-work for that meeting to be efficient. They could select a sampling of their best work from that class, and do a self-reflection where they draft a narrative for themselves and suggest what grade they believe their work has earned. The portfolio of work could be documented and shared with teachers (similar to MTC) so that they could access the work directly when trying to understand the assessment.

• This is all great for narratives that come at the end of a term, but what about training students to engage in a continuous process of self-reflection? The Cornerstone program requires students to keep up a Google Site - that seemed to work well for those students. What if we required students to post their reflection and 3 best pieces of work on a Google Site that's made available to their teachers (and larger community)? It would be sort of a digital exhibition night! 

Gold Standard PBL: A Resource

I recently ran a professional development workshop for teachers at Khan Lab School about Project-Based Learning. I used, as a framework, the Gold Standard PBL guidelines from PBLWorks. The following represent some of the notes from that workshop, as we evaluated what the project design elements outlined by PBLWorks might mean for our practice in our context.

1) Driving Question: Teachers were encouraged to use a driving question that hints at the role students will play in the project, the product that will be created, and the audience who will view the work. It is sometimes helpful to use the framework: "How can we, as __[role]____ do/make/create ___[product]____ for __[audience]____? Teachers were pushed to go beyond thinking of the students as "students". For instance, we rewrote any questions that began "how can we as students...."

2) Sustained Inquiry: This is about the research phase of the project. Teachers were asked to anticipate the important questions students might ask, and how they would be held accountable for their pursuit and documentation of answers to those questions. Will they do online research and write a paper? Keep up an annotated bibliography? Maintain an engineering notebook? Interview stakeholders? How can they seek answers besides asking their teacher? How can the teacher support their growing skills as researchers?

3) Authenticity: Is the work we are asking students to do "authentic"? That is, are there people in the real world who are doing these kinds of tasks? Will an actual audience see (and care about) the work the students are producing? If the work doesn't have any relevance beyond practicing a specific skill taught in the class, it is not "authentic" in the sense of PBL. Students (and teachers) should be able to answer the "who cares?" question about their projects. 

4) Student Voice and Choice: A common mistake of early implementers of PBL is that they take voice and choice too far. They give students complete control over every aspect of the project. While this can be powerful for older students with excellent executive function and project management skills, it is not appropriate for every class. Teachers have to make intentional decisions about which choices they allow students, and which are made for them. Constraints can breed creativity, and not every choice you give a student is meaningful. When considering whether to open up a choice in your project, ask yourself - is this an important choice for the student to be able to make? Does it take anything away from the learning if I make this choice for them? Does giving the student a choice in this matter create unnecessary extra work for both them and me? Do the students have the skills and context necessary to make this choice?

5) Reflection: While engaging educational experiences are crucial for learning, the "stickiness" of the learning comes from reflection. Often, teachers think of reflection as an exercise that comes after the project is over; it's a chance for teachers to catch up on grading while students write paragraphs about their projects. This placement at the end of the project makes the reflection process seem like an after thought (literally), and means that it often gets cut for time if the project runs long (or it becomes optional for students). For more a meaningful reflection, consider integrating constant short-form reflection into the entire project, from the beginning. Reflection can take the form of short journal entries, 2 minute Flipgrid vlogs, self-assessment based on the project rubric, etc. 

6) Critique and Revision: As important as a student's self-reflection is the feedback they get from their peers. It is a skill we should build in our students to ask for feedback and to give useful feedback. "Useful feedback" is kind, constructive, and specific. Use protocols to teach students how to give useful feedback on a regular basis, and get them in the habit of using each other as resources. This has the added benefit of putting less pressure on the teacher's input, so students aren't stuck waiting for feedback from one person before moving on. Consider asking students to actually incorporate that feedback into their work by requiring revisions. Here's a useful video to help underscore the power of feedback and revision: https://www.youtube.com/watch?v=E_6PskE3zfQ 

7) Public Product: This is the hardest element to include in a project if you are in "build-as-you-go" mode. It requires teachers to plan far ahead, and arrange for an audience for their students' work. This sometimes involves coordinating with third parties outside the school, inviting experts to partner with the class, or arranging for a field trip. For low-stakes public presentation, just challenge yourself to get the students' work displayed outside the classroom in some way. Ask yourself - who in the larger community cares about this work? Why are we doing this project? What was the answer to the "who cares" question? The answers should push you towards a public product of some kind.

Making Learning Visible - A Story of Learning from my PD Class

For the past few months, I’ve been participating in a professional development class called “Project Zero Online: Making Learning Visible.” The goal of the course is to help teachers find ways to both document what’s going on in their classroom, and to create opportunities for students to use documentation of all kinds (recordings, posters, boardwork, etc) to make their understanding of the material visible. This visibility helps them to think about their learning from a metacognitive perspective, and helps their classmates (and teachers) to engage with their thinking.

One of the assignments for the class is to write a “learning story” - something that captures the questions you have as a teacher and the ways you’ve noticed things changing in your classroom as a result of the reflection you’ve been doing as a part of this class. This is my learning story.

---

Stepping Back from the Spotlight

Most of the time I’ve been a teacher, I’ve felt that it was my job to entertain the students. I am constantly worried that they are not having fun - like someone who brings a friend to their favorite restaurant and worries that they’re not liking the food. I know what I find interesting and exciting about physics, and I want to show my students those things so that they can share in that enthusiasm. As a part of this course, I’ve challenged myself to step back from the spotlight and trust my students to follow the breadcrumbs I’ve left, and to tell me what they find interesting and exciting about physics.

---

Phase 1: I’m doing hands-on labs…but whose hands are actually on the materials?

Students were asked to build a circuit with at least 3 components, and draw the circuit diagram for that circuit. A student had questions about why her circuit wasn’t responding the way she expected. First I walked her through her circuit diagram, helping her trace the path of the current…

…then I took over, fixing the problem in the circuit for her so that she could experience the payoff of it working.

---

Phase 2: Focusing on documentation, but is there any actual learning evident?

Students were asked to create a 3D model that helps them to “visualize” the electric field around a distribution of charges. Each pair of students were assigned a different distribution, and they were given an example by the teacher first.

These infographics were the product required during a project about household wiring. The students were given a template, so many of the posters turned out looking the same. It was hard to see evidence of individual student learning, and there were very few questions asked by my students during this project.

---

Phase 3: Push the documentation earlier on in the learning process

In this exercise, meant to give students an opportunity to run through practice problems ahead of a test, there were several stations set up throughout the room. One station had them working through a worksheet in pairs, one had them building physical circuits, and one had them working out problems on the white board with help from their group mates. The whiteboard station, in particular, required one student to make their thinking visible by writing on the board so that their group mates could critique their work, catch mistakes, or learn from them.

In this assignment, students were asked to create a poster that we could hang in the hallway that would explain a lab we just completed to 9th graders. The purpose of the assignment was twofold - one, to make their learning visible to the community so that they might be asked about it, and two, to help them figure out what was important (they can only fit so much on the paper) so that they could better organize their lab reports. This was the second lab they did for my class, and the lab reports were a huge improvement over the first round.

Phase 4: Use documentation as a way for students to reflect on their own learning process

For this activity, students were asked to write out solutions to problems we had already solved together in class. The solutions (without the questions) were hung around the room, and students had to figure out which solutions belonged to the same originating question. Then they had to use the solution to try to write the original question - what values were givens, and which were the students asked to solve in the solution? The last step (not pictured) was for them to critique the solution and grade - how could the student have made their reasoning more clear? What made it hard to guess at the original question for this solution. The students enjoyed this activity, especially those whose originating questions were guessed correctly.

Here students were asked to reflect on a prompt silently (a “chalk talk”). The prompt in this case was about the nature of group work, and what was going well (or poorly) in class. My voice is still prominent in this discussion though, as I felt the need to participate in the chalk-talk in order to get students to answer the subquestions I felt were important (e.g. “why do you think one person sometimes ends up doing all the work”).

Phase 5: Use documentation to generate new knowledge without my input

Using the Question Formulation Protocol, I had students generate questions based on playing with the simulator. I told them not to stop to judge the questions, and for the note-taker to write them exactly as they were phrased. They then chose 3 of those questions to prioritize and answer using nothing but the simulator.

This is a screenshot from a PhET simulation. Students were able to play with the simulator, change things, and see what the results were. I did not tell students how to use the simulator, and gave them ~5 minutes to silently observe what happened as they changed the settings.

Once they chose 3 priority questions, they had to write an answer to that question in the form of a declarative sentence. I wrote their answers exactly as they said them on the board. The result was a set of notes in their words that taught them exactly what I would have taught during a lecture. The entire process took a whole 50 minute class period.

Teaching High School Physics

It’s been a while since I updated this blog, but I often find it helpful to process my thoughts about a subject through writing. So I’ve returned several years later to write about what I’m doing now - teaching high school Physics.

I’m teaching two different classes right now - one is a 9th grade integrated science course, where students take two trimesters of physics content and one trimester of chemistry. The other is AP Physics (I’m teaching both AP Physics 1 and AP Physics C: Mechanics, but now that they took E&M and Waves off the AP Physics 1 exam, it’s basically the same curriculum). I’m trying to focus on being project-based, incorporating lots of hands-on labs (which students have been craving in quarantine), and teaching students professional skills like reading research papers, scientific writing, and thorough documentation of their work.

Right now I’m struggling with an open-ended circuitry project for my 9th graders. These are the instructions I gave them. I just want them to start playing around with circuits, but in an intentional way. I want them to design a circuit to accomplish a particular goal, then build towards that goal. It prevents them from taking the easy way out, building overly simple circuits, and not really learning much along the way. I also wanted to make sure that they weren’t building huge, bulky projects out of cardboard that would take up a ton of room in my classroom. So I had them focus on input and output. A user does something (turns a dial, flicks a switch), and something else happens (an actuator pokes something, a fan turns on, etc).

The interesting thing here is that the potentiometers and clickers I provided have three terminals - so students could build parallel circuits. And some of them did. So far in class, we’ve only talked about loop circuits, Ohm’s Law, and Kirchhoff’s Loop Law. This project will hopefully open the door to talking about parallel circuits in the coming weeks.

One major challenge I have is getting students to use the multimeters. I made a little cheat sheet and asked them to tape it into their notebooks, which tells them how to use the multimeter to measure voltage, current, and resistance, and provide’s a reminder about Ohm’s Law. But the amount of troubleshooting needed to get the multimeters working properly has been getting in the way of the learning. Burned out light bulbs, poorly attached motor terminals, half-dead batteries that aren’t providing enough voltage - these have been plaguing us. I don’t feel as though I have the time to teach them the sort of instincts needed to troubleshoot these sorts of issues, nor do I think it’s necessary for a lab where I just want them to learn about series and parallel circuits. But I also want them to face challenges that feel surmountable. I’m not sure I’ve reached the right balance yet, but I’m working on it.

Welcome!

Inspired at least in part by conversations at ComSciCon 2015, I've decided to start practicing science writing via this blog that I'm relatively certain few people will ever see. I plan to write about any topics that I find interesting, which will mainly include answers to questions 3rd graders ask me during outreach events, or topics that come up over holiday conversations with distant relatives. I hope it's useful to someone! I'll be sure to include lots of visual aids and links to articles for further reading! I'll tweet about it whenever I post something new.