Using the Design Thinking Paradigm to Evaluate Professional Development

I have become more and more convinced that the design thinking framework is more like a mindset that can be applied to any activity which requires planning and execution, including professional development or professional learning experiences. By considering design thinking as a mindset instead of an optional approach to creative design activity, all our thinking is augmented, challenged, elevated.

The question that I am investigating for this week has to do with evaluating professional development or professional learning. The focus of ISTE Coaching Standard 5C is evaluation. The standard requires coaches to evaluate the impact of professional learning and continually make improvements to meet the schoolwide vision for using technology for high-impact teaching and learning. How can we tell if the time and effort teachers are expending on professional learning is actually having an impact on their teaching and more importantly on their students’ engagement and achievement? I’ve reframed the question in the light of my understanding of design thinking.

How can using a design thinking framework to create a professional learning program lead to more focused and ongoing evaluations which result in steady improvements which enhance teaching and learning?

To answer this question I will focus on the prototype ~ test phases of the design thinking paradigm because this is where most evaluation occurs.

In the context of professional development, I’m suggesting that prototype refers to the practical outcome of the goal of PD. It could be creating a workable solution for classroom management, building a lesson plan or unit study which integrates SEL, designing effective assessments, or developing best practices for implementing new technology tools. The prototype is the product a teacher will use in the classroom to enhance student learning and engagement. For this investigation, I will substitute the word prototype with the words new learning to refer to the new learning teachers receive from professional development.

The test phase involves monitoring the impact of professional learning on teacher practice, student outcomes as well as the impact on school goals and standards. In a design thinking paradigm, the test phase is not a once-and-done affair. Testing is part of a feedback loop: test ~ feedback ~ refine. This implies that the new learning is malleable, pliant, dynamic. For the sake of this investigation, I’m going to substitute the word test with the word evaluate. For evaluations to be effective they need to be frequent, transparent, and reflective. Using the insights gained from evaluations teachers can make necessary tweaks or changes to improve their own practice and student outcomes.

Creating, conducting, and analyzing frequent evaluations seems like a logistical nightmare, which may be why administrators resort to the typical quick end-of-PD survey indicating whether or not a certain PD experience has been useful or relevant, or fun. However, as teachers, we know all the reasons why effective assessment benefits our students (we even attend PD sessions on effective assessment techniques,) so why shouldn’t our own learning benefit from effective evaluations as well?

It seems more practical to me, as we engage in more personalized professional learning, that evaluation is also personalized.  How has teacher Jan reached her personal learning goal? How have her students been impacted? Has there been a significant demonstration of increased achievement?

Just as teachers plan carefully and make ongoing assessments of student learning as an integral part of the instructional process, they need to make evaluation and integral part of their own professional development process.

Guskey, 2000

Using the Feedback Loop for Sustainable, Frequent, Ongoing, Effective Evaluations

Guskey has shown that changing a teacher’s attitude, beliefs, and classroom practice occurs when a teacher can see evidence of the new program or innovation working well in their classroom and benefiting their students. One of the principles he suggests which stems from his Model of Teacher Change is that teachers should receive regular feedback on student progress.

So, what type of evaluations are sustainable, ongoing, and effective? I think the answer is formative evaluations carried out in the feedback loop and done by coaches or mentors as well as the teacher and students. More frequent formative evaluations would provide a more complete picture of how effective the new learning has been on teacher practice and student achievement. The iterative process of tweaking and refining in the feedback loop provides effective and ongoing evaluation which in turn leads to the successful implementation of innovative teaching practices and student success.

One of the guiding assumptions about evaluating teacher professional development according to the creators of Teacher Professional Development Evaluation Guide is that teachers have a key role to play when evaluating professional development. Teachers should be involved in:

  • planning evaluation
  • sharpening evaluation questions
  • collecting and analyzing data
  • reporting on evaluation results
  • making recommendations based on results

By so doing, teachers complement and extend their professional learning, have more opportunities for reflection, professional discourse, and collaboration with colleagues, and increase their professional responsibility and leadership. (Haslam, 2010)

The Poorvu Center for Teaching and Learning suggests that reflection and evaluation can range from low-key to formal and personal to inter-collegial and can come from a variety of sources including student perceptions, colleague perceptions, personal experience, and theory and research. (Brookfield. 2017). Information from the website is targeted toward tertiary educators, however, most of the ideas can be used in a school setting. I especially appreciate their drawing attention to using protocols for discussions and observations, very much like the protocols used in PBL. The site provides a list of teaching inventories and class observation protocols that can be used for low-key reflective assessment. A novel approach to evaluation which I discovered on this site is Teaching Squares (a structured discussion where four instructors observe one another’s teaching and meet to discuss and provide feedback.)

Collecting data need not be onerous or cursory. The following is a list of methods from the Australian Institute for Teaching and School Leadership which administrators, coaches, teachers, and students can use to provide feedback regarding new learning.

Surveys Informal Formal Collaborative
Student perception survey Self-reflection Journal Student learning gains Inquiry Research
Student well-being survey Teacher portfolio Classroom observations Video analysis of lesson
Parent Survey Team teaching Reviews of unit plans, assessments & teaching materials Focus groups
Teacher-devised survey Social media comments and discussion Analysis of teacher appraisal records Interviews with a range of stakeholders
The list provided by AITSL (2013)

I recently watched a webinar hosted by ISTE and presented by Dr. Nathan Lang-Raad titled Creating Authentic Learning Experiences in a Distance/Blended Learning Environment. Lang-Raad encourages coaches to create warm,non-judgmental spaces for teachers to receive and be encouraged to implement new learning. He points out that for successful buy-in from teachers, they need to know they are supported, understand the purpose, and see the results of their new learning in the classroom. He suggests that “it’s really important for [coaches] to ask questions throughout the training and then right after about how [teachers] are feeling about [implementing the new learning], what is [their] interpretation of the results?” One method coaches can use to collect this quick feedback is a ‘Weekly Pulse Checks’ using Google docs. Lang-Raad calls these ‘mini data points’ which if used consistently can help inform both the teacher and the coach. Watch the whole webinar, if you can, it’s really worthwhile:)

My biggest take-away from this investigation is that educators are on a lifelong journey of personal learning which blooms and matures because of continual evaluation, feedback, tweaking, and implementation, very much like the design thinking paradigm. Educators on this journey who have a growth mindset, humble attitude, and love of learning will always be challenged to improve their practice resulting in better student engagement and achievement.

References

Brookfield, S. (2017). Becoming a Critically Reflective Teacher, 2nd ed. San Francisco: Jossey Bass.

Feedback on teaching: Poorvu center for teaching and learning. (n.d.). Retrieved March 07, 2021, from https://poorvucenter.yale.edu/FacultyResources/Feedback-on-Teaching

Guskey, T. R. (2002). Professional Development and Teacher Change. Retrieved March 07, 2021, from http://keystoliteracy.com/wp-content/pdfs/orc-implement-science/PD%20and%20Teacher%20Change.pdf

Haslam, M. B. (2010, January). Teacher Professional Development Guide. Retrieved March 07, 2021, from https://learningforward.org/wp-content/uploads/2017/08/teacher-professional-development-evaluation-guide.pdf

Iste standards for coaches. (n.d.). Retrieved March 07, 2021, from https://www.iste.org/standards/for-coaches

Lang-Raad, N., Dr. (2021, February 02). Creating Authentic Learning Experiences in a Distance/Blended Learning Environment. Retrieved March 09, 2021, from https://iste.zoom.us/rec/play/XkHdcTnQEzLWz-fpE6wGdOk42ko7ys9VIkRqDBgY_FOeomH_5WHbQ7BdG-iPRbRFqmoAEYOFuaEZfaA-.GncHrCs_8Sn45iU9?startTime=1612288859000&_x_zm_rtaid=LLU0BbjwSDeavwVG6mo4ow.1615303037596.c72ddb390296781677b8277a26756b39&_x_zm_rhtaid=851

The Essential Guide to PROFESSIONAL Learning: Evaluation. (n.d.). Retrieved March 07, 2021, from https://www.aitsl.edu.au/tools-resources/resource/the-essential-guide-to-professional-learning-evaluation

Using teaching inventories and classroom observation protocols: Poorvu center for teaching and learning. (n.d.). Retrieved March 07, 2021, from https://poorvucenter.yale.edu/Observation-Protocols-Teaching-Inventories

css.php