As a part of the professional development program at Khan Lab School, we spent a term in 2023-24 reading research papers related to our teaching and learning practice. The rule was that the papers needed to have been published within the last five years, and we needed to identify at least one actionable takeaway from each.

Articles:

The key characteristics of project-based learning: how teachers implement projects in K-12 science education (Anette Markula & Maija Aksela 2022)

Making Sense of K-12 Competency-Based Education: A Systematic Literature Review of Implementation and Outcome Research from 2000-20219 ( Evans et al. 2020)

The need to disentangle assessment and feedback in higher education (Naomi E. Winstone and David Boud 2021)

The key characteristics of project-based learning: how teachers implement projects in K-12 science education (Anette Markula & Maija Aksela 2022)

Major Takeaway: To create hands-on, authentic projects, a unifying theme can be helpful.

Outstanding Questions:

How do our students interact with “projects” in classes, applied learning projects, etc?
How do teachers interact with a central theme?
How does the central theme become visible or relevant for students?

Actionable Learning: We should create a theme for next year (Our Local Community) and develop interdisciplinary projects in small professional learning community groups.

Notes: Things that often make PBL fail include poor articulation of the public product (in this paper, it’s called the “artifact”) and the driving question

Things that teachers struggle with: managing long-term projects and assessing group work
Learning from robotics:
- Students are naturally motivated by the end goal of a competition
- Students can identify the academic content ingrained in the process
Driving question are difficult to develop because there’s a sense that it needs to cover a priori designed content.
- What makes a good driving question?

What is the purpose of PBL?

To make learning fun? That should be a side effect!
To make learning relevant - that's the important takeaway

Making Sense of K-12 Competency-Based Education: A Systematic Literature Review of Implementation and Outcome Research from 2000-20219 ( Evans et al. 2020)

Major Takeaway: The concept of “flexible pacing” that aligns with CBE implies a minimum pace (with reasonable deadlines).

Outstanding Questions: What makes some deadlines flexible and some rigid? What are the benefits (and drawbacks) of more rigid deadlines?

Actionable Learning: Develop a school-wide policy around late work that is manageable for teachers and mission-aligned.

Notes:

What surprised you about the paper? What stood out?

3 R’s (relevance to the organization and its people, readiness to undertake the innovation, and resources to implement) are necessary for success (pg. 17)
Seems strange that CBE and mastery learning is both not that new, and not that well-studied. The paper makes it seem like implementation has been spotty and inconsistent, and therefore hard to reliably measure outcomes. This also makes it hard to evaluate the effectiveness of CBE, generally.
What does proper implementation look like? What training do teachers need?
It seems key to align assessment strategies to the teaching practices - but since the teaching practices focus on personalized learning, the assessment needed is by definition, not standardized. This makes evaluation difficult if not impossible!
Let’s not throw the baby out with the bathwater - what is “traditional education” as defined by the paper, and what useful practices can be extracted from it?
How many studies focus on STEM rather than humanities? What research is available that focuses on humanities courses and the implementation of CBE?

Where did you see connections to our context at KLS?

The paper uses the phrase “flexible-pacing” instead of self-pacing. Self-pacing requires students to be the arbiters of the appropriate pace, which requires them to be able to self-diagnose. “Flexible pacing” implies a minimum pacing target that stretches to encompass most learners. Questions are left for the teachers about how stretchy that pace should be.
Stable definitions of terms are one of the keys to success - where and how should we document our definitions?
As the school grows, we should remember that the flexibility we have to provide students with personalized learning plans requires small class sizes - so the faculty body has to grow alongside the student body. (question: (how) has CBE been implemented at larger public schools?)

What roads forward did the paper seem to suggest?

They suggest developing a “continuum” that spans from “traditional teaching” to the most radical framework you could imagine. There are many useful frameworks on that continuum that we could pull from if we could define them and what teaching practices and administrative policies go with them. This sounds like a large research undertaking
Definitions of terms and curriculum should not be tied to a teacher, but should live within the school and be easy to access.
We should lean on our relationship with B21 to help develop a stable framework for assessment
We should commit to an innovation for more than 1 year before measuring outcomes
We should develop a mechanism for faculty to share what’s working and what’s not in their classrooms.

The need to disentangle assessment and feedback in higher education (Naomi E. Winstone and David Boud 2021)

Major Takeaway: Students don’t read feedback when it is delivered alongside assessment.

Outstanding Questions: How can we provide holistic, thorough, timely feedback without our teachers feeling underwater? How can peer learning be leveraged to provide quality feedback? How can self-reflection be integrated into the feedback process? What are the best practices for drafting and revision?

Actionable Learning: Create Learning Journals and build in time to the schedule for teachers to meet with students and talk through their work in one-on-one meetings. Consider and research portfolio grading systems.

Notes:

This paper focused on the ways in which feedback (directed dialogue with students in which they gain insights into their own work and a pathway to develop their skills further) is often conflated with assessment (a grade given at the end of a term or course). It made the argument that when assessments (grades) are given alongside feedback (e.g. narratives), students often ignore the feedback and focus on the grades. The delivery of this feedback at the end of a term also prevents students from using it in an actionable way. Teachers writing narratives often direct their writing towards the parents reading the reports, trying to avoid conflict by explaining the grade, rather than entering into a conversation with the student.

This article resonated with many of us, and we discussed potential solutions to the problem:

1) Soliciting Feedback: Could feedback be more meaningfully integrated into the coursework itself? The paper suggested having students write cover letters to major assignments in which they ask for feedback on a specific learning outcome. The focused feedback they receive from the teacher is then an answer to a question, rather than unsolicited and unwanted. This would also mimic the peer review process that research journals use - they require resubmissions to come with a cover letter explaining how feedback was incorporated into the revision.

2) Focusing Feedback: Feedback should be parsable for students - rather than redlining an entire paper or providing complex feedback on a wide variety of learning outcomes, feedback should focus on one thing at a time, ideally in response to student-defined goals.

3) Feedback as a Dialogue: We discussed the possibilty of co-writing narratives alongside students, so that they are the first people parents ask when they want to understand a grade. Following a mastery approach, students could present a portfolio of work to their teacher, and in a one-on-one conversation, they could decide what grade the work has earned, and craft the narrative together before the report card goes out.

We then talked a bit about what solution #3 would look like, and what logistics would need to be figured out:

How would we make time for this in the schedule? We'd need a "portfolio week" where there were no classes, but only one-on-one meetings. At the end of each meeting, the grading for that student would be done, so we'd have a clean deadline for all teachers to be finished with grading (and it wouldn't bleed over into the break).
Students would need to do some pre-work for that meeting to be efficient. They could select a sampling of their best work from that class, and do a self-reflection where they draft a narrative for themselves and suggest what grade they believe their work has earned. The portfolio of work could be documented and shared with teachers (similar to MTC) so that they could access the work directly when trying to understand the assessment.
This is all great for narratives that come at the end of a term, but what about training students to engage in a continuous process of self-reflection? The Cornerstone program requires students to keep up a Google Site - that seemed to work well for those students. What if we required students to post their reflection and 3 best pieces of work on a Google Site that's made available to their teachers (and larger community)? It would be sort of a digital exhibition night!

I recently ran a professional development workshop for teachers at Khan Lab School about Project-Based Learning. I used, as a framework, the Gold Standard PBL guidelines from PBLWorks. The following represent some of the notes from that workshop, as we evaluated what the project design elements outlined by PBLWorks might mean for our practice in our context.

1) Driving Question: Teachers were encouraged to use a driving question that hints at the role students will play in the project, the product that will be created, and the audience who will view the work. It is sometimes helpful to use the framework: "How can we, as __[role]____ do/make/create ___[product]____ for __[audience]____? Teachers were pushed to go beyond thinking of the students as "students". For instance, we rewrote any questions that began "how can we as students...."

2) Sustained Inquiry: This is about the research phase of the project. Teachers were asked to anticipate the important questions students might ask, and how they would be held accountable for their pursuit and documentation of answers to those questions. Will they do online research and write a paper? Keep up an annotated bibliography? Maintain an engineering notebook? Interview stakeholders? How can they seek answers besides asking their teacher? How can the teacher support their growing skills as researchers?

3) Authenticity: Is the work we are asking students to do "authentic"? That is, are there people in the real world who are doing these kinds of tasks? Will an actual audience see (and care about) the work the students are producing? If the work doesn't have any relevance beyond practicing a specific skill taught in the class, it is not "authentic" in the sense of PBL. Students (and teachers) should be able to answer the "who cares?" question about their projects.

4) Student Voice and Choice: A common mistake of early implementers of PBL is that they take voice and choice too far. They give students complete control over every aspect of the project. While this can be powerful for older students with excellent executive function and project management skills, it is not appropriate for every class. Teachers have to make intentional decisions about which choices they allow students, and which are made for them. Constraints can breed creativity, and not every choice you give a student is meaningful. When considering whether to open up a choice in your project, ask yourself - is this an important choice for the student to be able to make? Does it take anything away from the learning if I make this choice for them? Does giving the student a choice in this matter create unnecessary extra work for both them and me? Do the students have the skills and context necessary to make this choice?

5) Reflection: While engaging educational experiences are crucial for learning, the "stickiness" of the learning comes from reflection. Often, teachers think of reflection as an exercise that comes after the project is over; it's a chance for teachers to catch up on grading while students write paragraphs about their projects. This placement at the end of the project makes the reflection process seem like an after thought (literally), and means that it often gets cut for time if the project runs long (or it becomes optional for students). For more a meaningful reflection, consider integrating constant short-form reflection into the entire project, from the beginning. Reflection can take the form of short journal entries, 2 minute Flipgrid vlogs, self-assessment based on the project rubric, etc.

6) Critique and Revision: As important as a student's self-reflection is the feedback they get from their peers. It is a skill we should build in our students to ask for feedback and to give useful feedback. "Useful feedback" is kind, constructive, and specific. Use protocols to teach students how to give useful feedback on a regular basis, and get them in the habit of using each other as resources. This has the added benefit of putting less pressure on the teacher's input, so students aren't stuck waiting for feedback from one person before moving on. Consider asking students to actually incorporate that feedback into their work by requiring revisions. Here's a useful video to help underscore the power of feedback and revision: https://www.youtube.com/watch?v=E_6PskE3zfQ

7) Public Product: This is the hardest element to include in a project if you are in "build-as-you-go" mode. It requires teachers to plan far ahead, and arrange for an audience for their students' work. This sometimes involves coordinating with third parties outside the school, inviting experts to partner with the class, or arranging for a field trip. For low-stakes public presentation, just challenge yourself to get the students' work displayed outside the classroom in some way. Ask yourself - who in the larger community cares about this work? Why are we doing this project? What was the answer to the "who cares" question? The answers should push you towards a public product of some kind.