Foreword
Mary Huber is a senior scholar at the Carnegie Foundation for the Advancement of Teaching. She is a cultural anthropologist by background, and she used her ethnographic field skills to interview a broad sample of faculty members about their working lives. What she found were the core practices and customs of the academic clan, and she has suggested in a number of places that there are several attributes common to all scholarly activity. These include being well informed by prior work and using methods of inquiry agreed upon by peers. Once evidence is gathered appropriate to the inquiry at hand, a member of the community of scholars will reflect on that evidence and make it accessible for peers and colleagues to learn from and review. In that vein, we are very pleased to make public the reflections of some of our colleagues at KU. They have been working on teaching projects, asking good questions about how to be more effective as teachers, and we invited them to offer us some of their lessons learned.
Chris Haufler offers a glimpse into how one academic unit is making an effort to improve the conversations faculty members have about teaching. Since he and his colleagues were not prepared for talking about teaching in their own graduate education, they are borrowing some ideas to try out in their setting. Beth Manolescu describes how her department has come to value setting aside time for conversations about teaching and learning. Their focused and collaborative time reflecting on teaching and learning has had tangible results, and it is clear why the CTE Advisory Board recognized the Department of Communication Studies for its collaborative work on learning.
Lisa Friis writes about the gradual development of her reflection about her teaching, beginning with a cut and paste approach to producing an organized documentation of one of her classes. Over a longer period of time she found that structured occasions that called for reflection on the quality of student work resulted in improved learning for her students. And she has an online course portfolio that represents well her energy in developing that course. Todd Little also tries to make quality of ideas accessible to his students of advanced statistical modeling techniques, and he has found that metaphors and analogies help students make sense of very complex mathematics. He notes that is at least as important that students understand the function and meaning of a statistical procedure as it is for them to appreciate the precision of the mathematics that drives it.
For Mike Vitevitch, participation in the KU Thematic Learning Communities provided an occasion for reflection on how well his teaching was supporting interdisciplinary study by students. The opportunity to create a really interdisciplinary bridging course gave rise to a number of insights into his teaching. Ann Cudd found that meeting with KU colleagues gave rise to useful reflection on implementing community engagement into her teaching. Her experience connects with Lisa Friis’ in that participation in a faculty program provided an occasion for generative reflection, and she too has developed her work into an electronic course portfolio that makes her work visible to colleagues. Ann’s upbeat conclusion from her engaged learning class gives a great example of the kind of change in both teachers and learners that can come from reflection on teaching with colleagues.
Overall we are happy to share these experiences related to teaching from KU faculty members. Together they demonstrate that intentional teaching is alive and well at the University, and the continuing focus on successful learning by our students is embraced by many faculty members.
Dan Bernstein Director, Center for Teaching Excellence
Peer Review of
Teaching and
Learning:
Can We Talk?
Christopher Haufler Heard on the streets of Mt. Oread: Ecology and Evolutionary “Hey, Dan, would you mind
Biology reading this manuscript I’ve written? I’ve put together some new ideas and I’m looking for a fresh perspective on how well I’ve argued my points.”
“Sure, Chris, no problem! I’ll let you know what I think.”
Not heard on the streets of Mt. Oread:
“Hey, Dan, would you mind sitting in on the lecture I’m giving tomorrow? I’ve developed a new way of helping students learn, and I’d like to know whether or not you think I’ve succeeded.”
“Sure, Chris, no problem! I’ll
see you in class and let you know
what I think.”
Asking a colleague to read and
comment on a paper or to cri
tique a grant proposal is some
thing that most of us consider
routine. It is an accepted part of
our academic culture, and we
genuinely benefit from the input
of our peers—it energizes our
scholarship and helps us hone
our ideas and present them effec
tively. And everyone gains.
Writers polish their products.
Readers gain perspectives on
their colleague’s research and
may pick up ideas relevant to
their own work.
But how many of us (except those in the promotion pipeline) invite our colleagues to sit in on a lecture, review a syllabus, or comment on an exam? Talking about teaching is not something we have been guided to do; it is not something we even think about regularly. Whereas we seek our peers’ opinions about our research and creative scholarship, we keep our teaching to ourselves and our students.
Student evaluations are necessary but not sufficient
Like most of my colleagues in the Division of Biological Sciences, and those across KU, I appreciate the value of student evaluations of teaching. Students can provide perspectives on the instructor’s lecture style, accessibility, and course organization. These perceptions are vital to our efforts to improve the quality of our teaching and develop our curricula. Student perceptions certainly should be taken into account as we pursue the complex endeavor of communicating knowledge effectively and educating students. However, students cannot supply the range of feedback we need as instructors. Obviously, they are not equipped to evaluate their instructor’s command of the subject, nor can they comment constructively on the range and quality of their instructor’s
Reflections 3
teaching methods. Relying primarily on students for feedback on our teaching is a little like expecting our research subjects to tell us how well we have conducted our experiments.
Are there options for gaining deeper knowledge of teaching dynamics? Research has demonstrated that peer review can provide feedback and promote dialog about teaching, and such review is a required part of the process of applying for promotion and tenure. However, in spite of the positive value of peer review of teaching, it has not become a regular feature of our academic pursuits.
Impediments to peer review of teaching
One roadblock to peer review is that the process is time consuming, and finding such time can be daunting. But it is not always time pressures that stand in the way of peer review. Often, my colleagues and I wrestle with the basic questions, “What are the components of peer review?” and “Is this something we should be expected to know?”
There is a very quietly stated assumption that the act of obtaining a PhD imbues the recipient with the necessary skills to teach and the insight to know how to help students learn. By extension we are also supposed to be able to recognize effective teaching and provide appropriate feedback to peers.
Good teachers can become good reviewers
Most KU faculty members are good teachers—and many are extraordinary—and care deeply about helping students learn. However, in the same way that gifted athletes can benefit from excellent coaching, even the best teachers can learn from their peers. But as any athlete will be quick to tell you, good coaching is seldom intuitive, and good coaches must know what to look for in building great teams. In the academic realm, peer review might be seen as the half-time pep-talk that invigorates and helps orient teaching efforts.
In the Division of Biological Sciences, many faculty members actively reach out for opportunities to enhance learning. They
- When visiting a class, the peer reviewer may not know what to expect, what to observe, what the teacher sees as the goals of that session, etc.
- Faculty peers may not know how to translate their observations into constructive recommendations for improvement. They may perceive that something is not right, but they may not be able to transform that perception into a set of positive critiques that will help the peer educator improve.
Relying primarily on students for feedback on our teaching is a little like expecting our research subjects to tell us how well we have conducted our experiments.
attend Center for Teaching Excellence events, discuss their teaching successes and frustrations, and see value in sharing ideas about pedagogy. For most, however, peer review remains a bit murky and misunderstood.
In biology, peer review typically follows a lecture-attendance model consisting of visiting a lecture or class session; developing reactions, recommendations, perceptions, etc.; and preparing a summary of observations and suggestions for improvement. This can be, of course, a valuable component of peer review, but it should not be the only element that is included because:
• A course is much more than one lecture. The peer reviewer who attends one class session sees just one component of the course; at best, a snapshot of the semester-long activities; at worst, a bad day in the classroom.
Expanding peer review
To build on the lecture-attendance model and extend my colleagues’ skills as peer reviewers, I worked with Dan Bernstein at CTE to develop ways to improve the peer review process. Specifically, we sought to:
- Help peer reviewers know what to observe and review
- Suggest what to look for and what questions to ask
- Provide insights on how to gain perspectives on the whole course
- Reveal ways to develop constructive feedback. Beginning with materials
designed for use by education scholars, we extracted critical elements and rewrote the text to focus specifically on university courses. We wanted to develop a document that would prompt reviewers to think about the possible components of the review process (including such items as a consideration of the faculty member’s goals, the course syllabus, and evaluation procedures and instruments) and the reporting of review outcomes.
We did not want to mandate any particular procedure, but we did want to lower the frustration levels of colleagues who were seeking better ways to do a good job as a peer reviewer. We decided that we could develop a “guide” to peer review that would indicate the elements usually included in a thorough review of a class.
Launching a prototype
Finding the best way to present the guide to faculty colleagues was the next challenge. By fortunate circumstance, my department was conducting its periodic consideration of the written procedures describing the annual evaluation of faculty. I noticed that one of the appendices to that document was a copy of the biological sciences student evaluation of teaching form. It seemed that an ideal complement to this appendix would be one that described the peer review process. I shared the guide draft with my colleagues and discovered that it was not only accepted as an appendix but was actually welcomed as an informational explanation for conducting peer reviews of teaching. As an optional resource, this guide was not intrusive or controlling, but it was available to those who wanted to take advantage of it.
From my own perspective, I see an additional result from the availability of this resource; by helping my colleagues understand more about the process of peer review, we might have more opportunities to contemplate teaching methods and share experiences that have improved student learning. I see value for both instructors and students in moving university teaching from its long tradition as a self-guided, introspective activity to something that we discuss and debate. Although this guide is only beginning to be included as part of the mentoring of junior faculty, as it percolates through the culture of my department and perhaps as it is adopted by other units, we can use it to initiate constructive dialogs about helping our students learn. So, can we talk about teaching?!
Below is the text of the appendix that is now part of the document describing EEB departmental annual evaluation procedures.
Guiding the Peer Review Process
The questions provided below can be adapted as prompts for requesting feedback from a colleague on different portions of your teaching. They are excerpted from sets of questions developed as “exercises” for peer review/collaboration by Pat Hutchings for the first AAHE peer review project. The complete set can be found in Hutchings’ (1995) From Idea to Prototype: The Peer Review of Teaching (Washington DC: American Association for Higher Education).
Syllabus Review
According to Hutchings, “Every course we craft is a lens into our fields and our personal conceptions of those disciplines or interdisciplines.” She suggests that faculty members think carefully about the goals and content of courses.
Ask colleagues to offer comments like these from a review of your syllabus:
- How does the course begin? What do the instructor and the students do as the course progresses? What are the key assignments? How does the course end?
- In what ways does the course teach students how scholars work in this field; the methods, procedures and values that demonstrate how knowledge advances within the field? How does it open doors to the critical dialogues and key arguments scholars are engaged in on the cutting edge of this field?
- How does the course integrate with other courses in the instructor’s own or other fields? To what extent does the course lay a foundation for others that follow it? Or build on what students have learned in other courses?
- What does the peer reviewer expect students to find particularly fascinating about the course? Where will they encounter greatest difficulties of either understanding or motivation? How does content of the course connect to matters students already understand or have experienced? Where will it seem most alien? How does the instructor address common student responses in the course?
Student Work
Choose an assignment— instructions for a student project, paper, problem set, classroom assessment, computer simulation,
Reflections 5
etc.—that is designed to elicit an important aspect of the learning you intend for students in one of your courses. Attach to the assignment several samples of student work, illustrating a range of responses, preferably with your feedback included.
Ask your colleague to address issues such as these:
- How does the assignment’s particular question, problem, or application reveal differences in student understandings or interpretations of a critical concept? What patterns emerge from students’ work?
- What evidence does the assignment provide that demonstrates student learning?
- What does the assignment and students’ responses to it tell a peer reviewer about how students are constructing the ideas that are central to the course and to the teaching goals? What misconceptions do they have about these ideas? How does the instructor identify and address student errors and misinterpretations?
- On what standards does the instructor judge student work on this assignment? How do these standards compare with those in a more introductory/advanced class? How are the standards related to the standards the peer reviewer would use to evaluate a piece of scholarship that a colleague has asked a peer to critique?
- What thoughts does the peer reviewer have about improving the assignment, the course, or the instructor’s teaching as a consequence of completing this exercise?
Instructional Implementation
Follow these guiding principles for classroom observations:
- Conduct visits as part of a consultation process that includes a pre-visit meeting to discuss goals for each class and purpose for the visit, and a post-visit debriefing to discuss what happened and how the class went.
- Let students know what is happening and why.
- Think of classroom observation as an occasion for discussion of departmental expectations. Discuss these questions with
- the faculty member who is observing you:
- Why did you choose to have this particular classroom episode observed? Is it a unique interpretation you bring to the topic or is it simply a “typical” day in your class? And, if so, why did you choose that basis for the observation?
- What context is needed to understand the class session? Where are we in the schedule of the semester? How does this topic fit within the context of other topics? How does it relate to what was discussed the day or week before and what is planned to follow?
- What were your goals for this day? Did the class session go as planned or deviate from your expectations? How so? Why? Did you change direction to take advantage of some new opportunity, get around an obstacle, deal with a new circumstance?
• What does this session say about your teaching? Does it show a characteristic style? A distinctive approach to material? Would others in your field be likely to teach this topic/ concept/whatever differently? Are you trying something new? Something you will continue to work at and improve?
Do you like what you see? u
Reflections on Retreats: Time for Considering the Intellectual Work of Teaching
Beth Innocenti Manolescu Communication Studies
Although in the communication studies department we discuss teaching in meetings at the level of area of specialization (organizational communication, public communication, and so on) and in bimonthly departmental faculty meetings, several years ago we noticed that our many responsibilities prevented us from reflecting upon the intellectual work of teaching for a significant period of time with undivided attention on a departmental level. We decided that we could do it only if we made time for it and set it at a time when our other burdens and passions would not be so pressing. Our solution: an annual faculty retreat. Each fall, about a week before the new semester’s class meetings begin or shortly after, we meet for a day or two— sometimes a weekday, sometimes a weekend—to discuss specific issues related to teaching and learning. In what follows I will discuss how our three department teaching retreats came about and what we did, as well as what they have meant to us.
As I mentioned, much of our reflection upon teaching and learning takes place among faculty members working in one of the areas of the field including organizational, interpersonal, intercultural, new technology, and public. The retreats have provided an opportunity to systematically reflect upon connections among areas and the challenges and successes we have had teaching undergraduate and graduate students and assessing their learning. Each retreat has had a different focus. The first addressed issues related to teaching and learning for undergraduate students. The second focused on the same issues as related to graduate students. Our most recent focused on assessment.
Undergraduate retreat
We tried to systematically analyze and resolve issues about undergraduate teaching and learning that surfaced during faculty meetings by holding a retreat. At the retreat, faculty members received a stack of index cards, and on each card they wrote one issue regarding undergraduate education in our department that they felt needed to be addressed. We collected the cards/issues into a single pile and organized them into themes. We then decided which were most important and began developing potential solutions for them. Among the key themes to emerge were that students in higher level courses were not
Reflections 7
always well prepared by lower level courses, that students did not have a clear sense of how the major fit together, and that too many students decided they wanted to be majors without knowing what we teach. We decided that one solution was to create COMS 104, Introduction to Communication Studies, a collaboratively prepared survey of the field that would introduce students to the major, help them see connections among the department’s courses, and be better prepared for higher level courses. Since this idea had been discussed before but never implemented, the remainder of the retreat focused on how the course would be developed. Following one semester, the course was taught and has been taught every semester since then.
The course has been truly collaborative. Faculty members have contributed lesson plans and visited the class to discuss topics in their areas of expertise. In each semester in which it has been offered, a different faculty member has served as instructor. Due to the number of students enrolling in it—rising from 30 in its first semester to 122 this semester and still growing—we now employ a graduate teaching assistant. Faculty members who teach and have taught the course meet to discuss assignments and continue to work with a collaboratively-developed syllabus and course materials. The course has received positive student evaluations. Students report that they appreciate the course and that it is meeting our goals of introducing the major, integrating the different areas of the department, and preparing them for higher level courses.
Graduate retreat
The impetus for the graduate retreat was the same song, second verse: in faculty meetings we regularly shared some of our problems and successes with graduate teaching and learning and thought that we could repeat the positive outcomes of the undergraduate retreat with one focusing on the graduate program. This retreat was a two-day affair. On the first day, faculty members generated a list of issues regarding the graduate program that they believed needed to be addressed. On the second day, Carol Nalbandian, a professional facilitator, led the discussion on what we liked about the program, what needed to be improved, and what programs and initiatives could be developed to address problems. In the area of graduate student teaching and learning, we concurred that our discipline expects doctoral students to graduate with publications (difficult due in part to the lack of large, funded team projects), graduate students needed assistance making research presentations, and their proficiency in research methods needed to be strengthened.
One initiative that we developed to address these issues was a department-wide research colloquium. Three faculty members agreed to lead it. Because we wanted to continue to foster two-way communication with graduate students, two different graduate students each year also participated in organizing and running the colloquia. All faculty members regularly attended colloquia and sometimes presented their own work. At each monthly meeting we typically heard two graduate student presentations: one using social scientific methods and one using rhetorical methods. To date at least two of the graduate student presentations have been published.
We refined this initiative as we realized that the challenges facing students working from a social science perspective were different from those working from a humanistic perspective. We replaced the department colloquia with more specialized, intense efforts to assist graduate students in preparing their work for submission to academic journals. We initiated two new programs—the Social Science Journal Club and Rhetoric Writing Workshop—designed to help graduate students develop research projects begun in courses into journal submissions.
The Social Science Journal Club met every two weeks, and all graduate students were invited to participate. Participants first completed an evaluation of the manuscript they would work through during the meetings so Joann Keyton, the faculty facilitator, would have information about their needs and be able to use her experiences as a researcher and editor of a national journal to help students meet them. She provided examples of published research and created short writing guides. Students were responsible for establishing a plan of writing for their manuscript, demonstrating progress at each meeting, and giving feedback to one or more students at each meeting. Because social science often requires intercoder reliability, participants also used the time to code data for those who needed it.
The Rhetoric Writing Workshop meets about every month during the academic year to workshop one or two graduate student papers. Since research in rhetoric tends to be single-authored, faculty and students agreed that a workshop format would be appropriate. Authors post papers on BlackBoard at least a week in advance, and participants write comments on the manuscripts. At meetings, the author briefly introduces his or her paper, reads one paragraph, and then is a “fly on the wall” as participants discuss the draft.
and what we could do. Although bringing in an assessment expert was too expensive, Dan offered guidelines about what we could do and how. I then met with several colleagues to try to tailor these guidelines to our needs and particularly to the fact that the entire faculty would be meeting as a group. We wanted to provide people with some ideas about what they could do in their individual courses, but we also wanted to focus on accomplishing something that we could
Although each of our retreats has produced tangible results, the less tangible results also help us to sustain a department culture that values teaching excellence.
The author may then question participants. We expect this initiative to enhance graduate students’ research presentations and result in publications.
Assessment retreat
Our most recent retreat focused on assessing undergraduate teaching and learning. We decided that this was an appropriate topic because on an individual level, we want to know if students are learning what we’re trying to teach. On a departmental level, we want to be able to show external audiences what our courses equip students to do.
Since I was directly involved in planning this, I can say a few words about that process. It took time and effort but not an inordinate amount of either. Our department’s administrative staff took care of reserving a room, ordering food, and gathering supplies. I spoke with Dan Bernstein about who we could invite to facilitate the discussion do only as a department. We thought we could first generate department teaching and learning goals, break into smaller groups based on our areas to discuss ways of assessing whether the goals were being achieved, then return to the large group to share what we had discussed.
This was an ambitious plan. We covered some of the basics of assessment: setting learning goals and identifying student performances where we can see if goals are being achieved. But we spent the rest of the time reflecting on course, area, and departmental goals. First each of us thought about the one or two undergraduate classes that we frequently teach and took a few minutes to individually generate goals for the courses. Faculty members in each area met in small groups to generate a set of area goals which we wrote on flip charts. Each group then walked around the room and read the other groups’ goals. We taped each area’s goals to a wall at the front of the room and from this generated a list of department goals.
We recognized that we would need to continue to refine this list, so following the retreat we circulated a document that summarized each area’s goals and department goals. We also reflected on potential uses we could make of our reflections. Not only would they help us to think about assessment in our individual classes but would also help with departmental assessment and attract students with appropriate interests to the communication studies major.
Although each of our retreats has produced tangible results, the less tangible results also help us to sustain a department culture that values teaching excellence. At these annual retreats we see our colleagues from all areas of the department and from all career stages voluntarily committing a full day or two, not to teaching activities as such but to reflection upon the intellectual work involved in teaching. We enact core departmental values as we collaboratively reflect upon this work. We see that despite and perhaps because of our methodological and other differences, we can come together to think and share ideas about how to achieve common pedagogical goals. The retreats have provided us with a time and space to reflect upon our teaching practices—something that it is difficult to do individually but, without the retreats, would be impossible to do as a department community with shared values and a shared commitment to
teaching excellence. u
Reflections 9
Convergence
| Elizabeth Friis | Quite frankly, I was a skeptic. | was minimal. I made the portfo- |
| Mechanical Engineering | An online course portfolio for reflection about teaching? In my | lio like I make research posters to present at scientific confer- |
| mind, I had already “reflected” | ences. I didn’t reflect much; I |
| on my course: what I taught and | wrote an introduction, methods, |
| how the science content might | results, and discussion. My |
| be updated was very important | “assignment” was finished—my |
| to me. I started an online course | mission accomplished. The first |
| portfolio for my Biomaterials | draft of the course portfolio was |
| class as a required task for the | done, without much extra effort |
| CTE Faculty Seminar in which I | (or true reflection) from me. |
| was participating. In many ways, | To my surprise, several months |
| I approached this assignment like | later Ann Volin of the CTE staff |
| some of my engineering under- | contacted me about improving |
| graduate students do the home- | and adding to that initial portfo |
| work I assign, just jumping over | lio. This time, I was given an |
| a hurdle that has to be overcome | outline of categories that Ann |
| to get to the seminar series of | had developed to organize what |
| interesting discussions with my | I wrote previously and to guide |
| colleagues. As I was “jumping,” | me along the way. Since I was |
| little did I realize that the work I | teaching the course again the |
| was doing would be so helpful in | coming semester, I thought it |
| the coming years. | would be good to organize my |
| In the Faculty Seminar, our | thoughts. |
| first step in approaching portfo- | As I started to write sections, I |
| lio development was to explore | began to think more about my |
| examples from other faculty at | class and how it was structured. |
| different institutions in similar | I had expended a great deal of |
| areas. Even after looking at other | effort to set up the course origi |
| online portfolios, it was still | nally, and I had spent a lot of |
| unclear how this exercise would | time revising and improving it |
| be of benefit to me. In forming | each of the three times I taught |
| the first draft of my portfolio, I | it. I had even made an education |
| just cut and pasted from other | presentation about this different |
| documents, such as my course | style of course at a national |
| syllabus, an abstract I had writ- | research conference. But as I |
| ten for a conference, and my | began to write more about it in |
| project assignment descriptions. | the CTE format, I realized that I |
| I did some new writing, but it | hadn’t really looked back very |
often to think about my students’ performance or progress throughout the semester. I was so focused on what I taught that I often didn’t look enough at what and how my students actually learned through their assignments. Were assignments helpful to students? Did the quality of their work actually improve during the semester? Did I know if students realized what defined quality in their work?
Writing sections in response to Ann’s prompts probably took me about one full day to complete. At first, I had difficulty seeing the three-dimensional nature of the portfolio; I couldn’t quite see the end result and how it these quality differences. I know quality when I see it, but did my students know, as well? I give students mostly open-ended questions where there is no one right answer; as engineering students, most were at first taken aback to find that these questions could not be answered by one correct number with a box around it at the end of the solution. Had I ever given students examples of different levels of quality work or taken the time to write out descriptions of what makes a quality homework assignment or project research proposal? Perhaps the lower performing students were not even aware of the standards levels.
I was so focused on what I taught that I often didn’t look enough at what and how my students actually learned through their assignments.
would all fit together on the web site. Ann guided me through this stage with ease. She took what I wrote, edited it, and put it together in an appropriate format. As she questioned me, I began to look more at student outcomes in my course and started to reflect about how I structured the course and how students progressed through the semester in response to this organization. She helped me look at my course in a different way.
Ann asked me for examples of previous student work in the course, examples of high, average, and low quality. Yes, I had given these assignments grades that should reflect quality, but as I picked out and examined the works a year after they were graded, I realized I had never demonstrated to the students Perhaps just like me when first starting to write the online course portfolio, they couldn’t see how to approach open-ended questions and produce quality results.
CTE staff members asked for short responses to their prompts, they then did most of the work in formatting and editing. Their questions were structured to help lead me through a thought process of reflection of quality work and demonstration of that quality through examples and written description. By the end of this process, the CTE staff put these snippets into a form that actually made sense! Only one prompt required significant effort from me; Ann requested that I write a rubric demonstrating levels of quality work for the major project of writing an NIH-style research proposal in place of a traditional term paper. Since I was teaching the Biomaterials course again that semester, I decided to take the time to write the rubric.
As I formulated the rubric, I began to realize just how much of my personal professional experience went into determining the quality of my students’ work and hence their grades on the project proposals. I had reviewed various styles of grant proposals for national organizations for over seven years—how could students understand the quality criteria by which I review without me giving them a full multi-page NIH-style review? Unfortunately, this type of feedback is prohibitively time consuming for a class of 22 students. Even then, as most of us know from our own research proposal reviews, comments from long critiques of our work are difficult to fully comprehend when they criticize the proposal in which one has invested a lot of time and energy.
Instead of giving intense formal feedback to each student team, I decided to invest more time into developing the rubric. Since I was teaching the Biomaterials course that semester, I also gave this rubric out to my students and reviewed it with them before they started work on writing the grant proposals. The rubric was intended to guide the students through development of the research proposal, to give examples of how quality was influenced by specific choices and forms of information presentation. Perhaps it was related, perhaps not, but the quality of proposal projects appeared to improve significantly that semester.
Reflections 11
Every time I have worked with the CTE staff on developing aspects of the online course portfolio, I have learned and have been able to incorporate that awareness into my courses. I’m using the rubric again now in another graduate course; it will save me time, as well as again give students a sense of quality in developing their class research proposals. Though I have a lifetime to go in learning how to become a better instructor, I now think more about how students might perceive quality results in their assignments and how students at all levels can better learn how to improve the professional quality of their work.
My Biomaterials course portfolio is far from perfect. I could spend many hours making it continuously better—but I don’t have that time. Few, if any, faculty members do. However, in just the limited time I have spent thinking about this course and helping CTE put together the portfolio for me, I have learned a lot about myself and my students through this process of true reflection. I can only imagine the multitude of downstream effects on my teaching in this class as well as in all others after working through the process of making my thoughts explicit.
Has working on the portfolio made me a better instructor? Hopefully. Has it affected the way in which I think about students’ learning and so helped me provide a better learning environment for my students? Definitely. I now have a tool that can be easily updated and used from year-to-year to give me an institutional memory of student outcomes for this course. I can share my experiences and ideas with colleagues from around the world. Later this spring I am scheduled to give a presentation at a national meeting about the unique way I structure this course. I will share this portfolio with pride about how KU helps their faculty members strive for excellence in our fundamental directive of educating students. I look forward to the time when more people in my area develop their own online portfolios and I can learn from their examples to continuously work on and
improve my own courses. u
Thought Candy: Metaphor in Education
Musings of Todd D. Little, Psychology Enhanced by some current students
| "But the greatest thing by far is to have a command of metaphor. This alone cannot be imparted by another; it is the mark of genius, for to make good metaphors implies an eye for resemblances." —Aristotle, Poetics | I am the kind of person who prefers to kill two birds with one stone. When Dan Bernstein asked me to write on my use of metaphor and analogy in my advanced statistics courses, I thought I’d take advantage of the opportunity to actually ask my students whether using them aided in their understanding. Metaphors can be quite complex, ranging from computer metaphors for understanding human cognition to simple clichés such as “a bird in the |
|---|
| hand is twice as good as two in the bush.” They can come in mixed forms as well, such as “a |
|---|
| stone in the hand will trump two birds formerly of the bush.” And regardless of whether the attempt at metaphor, analogy, or simile is technically a metaphor, analogy, or simile is less important than the basic discovery that I have made in my 20 years of teaching advanced, mathematically based concepts; namely, a good metaphor is worth a thousand numbers, equations, and formulae. |
|---|
| I use terms like metaphor and analogy loosely. I am generally trying to capture the idea of making a comparison, such as when a well-understood word, |
|---|
| Reflections 13 |
|---|
phrase, concept, or image that ordinarily designates one thing is used in reference to another. Such comparisons are drawn in order to show a similarity in some respect, usually under the assumption that if things agree in some respects, they probably agree in others.
Perhaps the most demanding course that I teach is Structural Equation Modeling (SEM; PSYC 991, n = 21 this semester). SEM is a second generation family of statistical procedures that involves a lot of mathematically based concepts and ideas.1 A handful of students will instantly resonate with the material presented in footnote 1. Most students (and I expect many of the readers of this essay), on the other hand, would find this material bewildering at best and more likely beyond comprehension. My struggle as a teacher of such material has always been to figure out ways to make the concepts and issues that surround them accessible to the math phobic.
As a graduate student, I took a course on the general use of metaphor in science. It struck me that drawing such parallels among things that exist in nature, or “kinds,” the precise terms created to capture the essence of the “kinds,” and the imprecise (but rich-by-analogy) metaphors used to represent the relations between them, might be beneficial in my teaching. One clearly runs the risk of imprecision and, but with rare exception, all good metaphors eventually fall apart. However, a good metaphor or analogy taps existing knowledge structures about how a set of things relate, thereby aiding in learning the precise “term” for the “kind” and the relational nature among the “terms” for the “kinds.”
Back to bird hunting with stones, I was particularly pleased to get the following responses to my query. This first response captures the scaffolding idea when introducing imprecise terms as proxies of the precise (which in turn reflect the mathematical underpinnings):
“One of the first things I noticed about your lectures from the onset was your use of concrete examples and analogies. This was especially useful in the initial lectures in which you described what SEM was all about. Also I noticed in your earlier lectures you used very little SEM terminology and more analogies which I think helped to lay a foundation for the uses and purposes of SEM. Later, I noticed you started pairing the analogies along with the terminology used in SEM. When used in tandem, it provides students with multiple avenues for understanding. For example, if one student does not understand the technical terms/description, at least she can still grasp the basics of the lecture. Not all students come to the class with the same level of understanding and background so analogies help to put all students on the same playing field, so to speak.”
There are very few times when PowerPoint and its animation capabilities actually enhance a lecture. However, when carefully considered and applied, visual metaphors (animated or not) can be tremendously helpful. Student B captures this idea. This student also captures well my reasons for using metaphor and analogy in conveying math intensive concepts to the “less-mathematically inclined”:
“I’ve never had a stats teacher use metaphors to explain key concepts. Instead, they have always thrown a formula at me. I am a little math-phobic and when I see a really complicated formula, I usually drift off. I much prefer to think of things at the conceptual level and metaphors can be very helpful in understand
1 For example, multiple group mean and covariance structures models would begin with the matrix algebra notations for the general factor model, which, for multiple populations g=1,2 .. G, is represented by:
xg= τg + Λg ξg + δg (1)
E(xg) = µxg = τg + Λg κg (2)
Σg = Λg Φg Λ ’ g + Θg (3) where x is a vector of observed or manifest indicators, ξ is a vector of latent constructs, τ is a vector of intercepts of the manifest indicators, Λ is the factor pattern or loading matrix of the indicators, κ represents the means of the latent constructs, Φ is the variance-covariance matrix of the latent constructs, and Θ is a symmetric matrix with the variances of the error terms along the diagonal and possible covariances among the residuals in the off diagonal. All of the parameter matrices are subscripted with a g to indicate that the parameters may take different values in each population. For the common factor model, we assume that the indicators (i.e., items, parcels, scales, responses, etc.) are continuous variables that are multivariate normal in the population and the elements of Θ have a mean of zero and are independent of the estimated elements in the other parameter matrices (see Little
ing concepts, even if I don’t completely understand the math behind it all. Given the conceptual nature of SEM, your metaphors have helped me to grasp its basic principles. Perhaps the most helpful metaphor has been the example of the three-legged stool. If you had given me just the math of why three indicators of a construct are more stable than four indicators, I doubt I would have gotten it. I probably still don’t completely get the math, but I understand the principle at a conceptual level. The use of visual
the student something richer to
hang the ideas on: “Your teaching style makes things very visual and puts complicated concepts into the common English language. It’s as if the metaphors and analogies point to a familiar cognitive framework that can be used to make sense of far less familiar, complicated statistical concepts.”
and more than one way to
understand the concepts: “I like analogies in general, and in multivariate statistics classes specifically… I think presenting things in more than
One of the key reasons why I think metaphors work so well in my courses … [is that] they give the student something richer to hang the ideas on … and more than one way to understand the concepts.
metaphors has also been extremely helpful. The metaphor of the construct as a circle with a centroid has helped me visualize the concepts of measurement error and triangulating around the construct centroid. These demonstrations have also been helpful in literally seeing the relationship between reliability and validity in a new way. Seeing the concepts allowed me to ease my way into the class and tackle the math later with more of a conceptual background. Honestly, if the first day was all formulas and no metaphors, I am not sure I would have stuck with the class.”
Student C and D capture per
haps one of the key reasons why
I think metaphors work so well
in my courses; namely, they give one way helps students who learn differently better understand the concepts.” Some metaphors work better
than others and some work better for some people in different ways, but I was also very pleased to see that some of these metaphors might actually proliferate and help others learn some of the key concepts of the area:
“The water in the ice cube tray metaphor worked very well for me today. I also actually used your pinning down a cloud analogy to a friend of mine who is also taking SEM (not here). I did not do it justice, but it seemed to have sunk in with her.”
In SEM, we attempt to represent latent constructs from numerous measured indicators. The latent construct is like a floating cloud. To measure it and quantify it, one simply defines a point somewhere in the cloud to pin it down. Once pinned down, one can measure from that point to the edges of the cloud and quantify it. The point where the measurements are taken is arbitrary. You simply need to pick a point and pin it down. This analog roughly communicates the often difficult concept of setting the scale of measurement for a latent construct.
Turning to the final student comment, this student emphasizes the point that one cannot survive on metaphor alone (and the point that even the math able like them):
“I don’t think analogies can
substitute completely for
math. I am more of a math-
oriented learner than verbal.
So while the analogies help,
the math helps me understand
concepts just as much. I
wouldn’t want to see math
replaced completely with
analogies and metaphors.”
As for how I find metaphors, I don’t know if it’s just a gift (which I doubt) or if it’s just a straight forward process of when you want one, it will come to you. I suspect that many readers of this essay will find that metaphors are rather easy to come by if they decide to try using them (visual or verbal) as part of their curriculum content. I also suspect that you will find your students to be favorably inclined. Think of them as
thought candy. u
Reference:
Little, T. D., and Slegers, D. W. (2005).
Factor analysis: Multiple groups. In B.
Everitt & D. Howell (Eds.), & D.
Rindskopf (Section Ed.), Encyclopedia of
Statistics in Behavioral Science (Vol. 2,
pp 617-623). Chichester, UK: Wiley.
n of Today nts of Tomorrow?
business majors might enroll in courses paired together in the TLC entitled Brazil: Business and Society to learn about the language and culture of what is predicted to be one of the largest markets of the future. Pre-law students in the Law and Society TLC learn how various court rulings have affected society and how society has influenced the legal system. Engineering students in The Leading Edge TLC might consider moral and ethical implications of something they might build rather than just how to build it. It was obvious that the TLC experience helped levels of a problem, who would take theo
retical inspiration from many fields, and who would work with researchers from other domains to solve a problem rather than become a psychologist, computer scientist, physicist, or some other “-ist” who would examine only one piece of a significantly larger puzzle. Could the seeds of cross-disciplinary research be planted in the fresh, fertile minds of first-year students, take hold in a few students here and there, and blossom into a new breed of researcher in the future?
During the bridging seminar, we visited different laboratories across campus, including the Developmental Neurolinguistics Lab directed by Mabel Rice in SPLH where we learned how an electroencephalograph—a device that measures tiny electrical potentials on the scalp produced by the “firing” of cells in the brain called neurons—could be used to obtain precise measures of how and when spoken language is comprehended. In another seminar I brought in an Artificial Neural Network (ANN), a computer program that mimics how the human brain processes information. The brain consists of many simple, highly interconnected cells (neurons) that produce different patterns of electrical activity (which can be measured by an electroencephalograph) to give us our conscious experiences. Like the brain, an ANN is a computer program comprised of many simple yet highly interconnected units that process information. These programs have been used to model and better understand various cognitive processes.
In other seminars I showed videotapes of individuals with various neurological disorders to give TLC students a better idea of what it might be like to have sustained damage to part of their
Reflections 17
Could the seeds of cross-disciplinary research be planted in the fresh, fertile minds of first-year students, take hold in a few students here and there, and blossom into a new breed of researcher?
freshmen see their first semester as something more than the semester they were enrolled in course X, or fulfilled requirement
Z. It helped orient students to the concept of “general education” by exposing them to important issues with implications that reached beyond the one course in which they learned about that issue.
I wondered, however, if a TLC could also inspire first-year students to become scientific leaders in the future who would use a variety of methodological approaches to understand several
To increase undergraduate students’ exposure to cognitive science, I worked with Linda Dixon in the Thematic Learning Community Office to develop the Mind, Brain, and Behavior TLC. In this TLC, first-year students enroll in PSYC 104, General Psychology (with me as the instructor), a one-hour bridging seminar, and one other introductory level course from the other disciplines that traditionally comprise the cognitive sciences: anthropology, computer science, linguistics, philosophy, or speech-language-hearing.
brain involved in short-term memory or in recognizing faces of people they know. Guest speakers also visited the seminar to talk about their work examining how children learn language or of studies employing functional magnetic resonance imaging to identify which part of the brain is involved in performing a given cognitive process. In Fall 2004, students also went to the Lied Center for the Humanities Lecture Series presentation by Steven Pinker, a cognitive psychologist at Harvard University.
Through these experiences— some of which could not have been done in a traditional classroom—TLC students were able to see that something they learned in one class actually can apply to another. More importantly, they were exposed to core disciplines of cognitive science and developed an appreciation of how big the questions are that scientists study; only with cooperation among scientists from many disciplines could these questions be answered.
In my most recent TLC class, I wanted to more explicitly show relationships among different disciplines of cognitive science and, more broadly, relationships among other KU courses, between the classroom and the real world, and between theory and practice. To help illustrate how everything is connected, in the seminar we spent part of the semester reading the book Linked: The new science of networks, by Albert-László Barabási. In Linked, Barabási describes how networks consisting of nodes and links can be used to help us understand the underlying structure that exists among numerous, interacting entities.
Nodes in the networks represent individual entities, perhaps people, cities, or countries, and links connect two nodes if they are related in some fashion, perhaps people who know each other, cities that have airline flights between them, or countries that engage in economic trade. Different types of structures that emerge from interactions among entities have significant implications for the efficiency and stability of the underlying system.
During one of the seminars I illustrated how this so-called new science of networks—populated primarily by physicists, computer scientists, mathematicians, and engineers—influenced my own psychological research on language processing and resulted in several new discoveries in my lab. Words stored in memory are linked together in an elaborate and stable network, much like networks that characterize airline travel and other complex systems in the real world. By using these mathematical techniques, we can better describe how the words we know are organized in that part of memory we call the mental lexicon. Seeing that mathematics had something important to say about psychology also helped TLC students discover that there could (and should) be connections among the other classes they will enroll in while at KU, not just among courses that constitute the cognitive sciences. By making connections explicit among concepts, theories, or, more broadly speaking, other courses in the university, these students could now use that framework as a model to help them discover other relationships and to build their own efficient and stable knowledge structures.
Encouraging signs
Although I’ve planted the seeds for cognitive science over the past few years in several cohorts through the Mind, Brain, and Behavior TLC, it is still too early to point to a mature, new breed of researcher. However, I am encouraged knowing that these ideas have taken hold in a few students. Two previous MBB students are currently participating in the KU BioSciences Initiative, a federally-funded program that promotes diversity in biomedical research. One student is mentored by Holly Storkel in SPLH, the other by Monica Biernat in psychology, and both are working on honors theses for their programs. Another former MBB student has worked in my lab the past three years (most students don’t become involved in research experiences until their senior year) and is now completing an honors thesis in psychology. I have also run into other TLC students at faculty colloquy and the Undergraduate Research Symposium. Granted, it is difficult to know if the experiences these students had in the TLC caused them to pursue these additional experiences in the first place, or to pursue them earlier than they might have otherwise, but the nurturing environment provided by the TLC program certainly couldn’t have hurt. I hope the Mind, Brain, and Behavior and other TLCs, like Women in Science, can continue to nourish the minds of our first-year students and help many
more blossom and mature. u
For more on Thematic Learning Communities at KU, see http://www.tlc.ku.edu.
Ann Cudd Philosophy/Women’s Studies
The Body in the Mind: Energizing Philosophy Through Service
I envy athletic coaches and theatre directors for the enormous efforts they can coax from our students. Students seem to be willing sacrifice their leisure time, their health, and their other coursework (including my courses) for those activities without a second thought. Rarely does a student say to me, “I told my coach that I couldn’t go to practice until I finished that paper on Kant’s argument for the categorical imperative. I just had to get it right.” But coaches and directors regularly get every last ounce of physical energy and emotional enthusiasm out of their students. They can expect that level of effort. Coaches can tell their player to get on the treadmill at top speed for 30 seconds, or directors can rage about how actors are performing a scene, demanding attention when it wanders even slightly from the goal. Imagine the headlines if a philosophy professor told her students to “give me 50” pushups or shouted at them that they were not passionate enough every time they revealed that they had not really tried to understand the assigned readings!
Why this difference in effort, enthusiasm, and commitment? I can think of at least three reasons that are not directly related to my inadequacies as a teacher: first, students can very quickly see how their efforts make a difference in their performance; second, they create their product with other people; third, they get to use their whole person in those activities, that is, their bodies as well as their minds. In short, students want to do as well as think; they want to see concrete results beyond a pile of papers they have written; they want to talk, laugh, cry, and sweat over their work. This sort of reflection led me to wonder whether service learning might help me to enthuse my students. If scholarly work could involve students’ physical selves to a greater degree, perhaps they would generate the same level of enthusiasm for my courses as for athletics or theatre.
For several years I thought about adding service learning to the curricula of programs that I was a part of, but as one voice in those units, the idea never went anywhere. Then it struck me that I could just do it myself in my own course. After all, I am a full professor. About the same time the opportunity to learn something more about service learning arose through a “Service Learning Institute” offered by CTE. So I applied and started to think about what to do.
Reflections 19
I chose to add a service component to a course I have taught five times in my years at KU with five completely different sets of readings. It is not that I have never been happy with it, but rather that there are too many possible topics and I have wanted to explore different ones at different times. Feminism and Philosophy is now a cross-listed course for each of my two academic homes: philosophy and women’s studies, and it enrolls approximately equal numbers of students from each unit. This course was the right course to try a service learning component in for several reasons: the material is more applied than the usual philosophy course (consider: logic or metaphysics—what service would be appropriate to those topics?!), many of the students (i.e., the women’s studies students) are likely to be going into service- or policy- oriented professions, and it is a course that I continually try to improve with new and different teaching methods. It is a course I love, it enrolls well, the students come in enthusiastic, but somehow I always feel that they go out a little less enthusiastic than they came in. The Service Learning Institute allowed me to think through the details of setting up a service experience for the students and brainstorm with other faculty about what kinds of service would be appropriate, how to implement it, and how to evaluate it.
As a philosopher, I am drawn first to respond to skeptics. Skeptics about service learning question the relevance of volunteer work to abstract concepts that students should be studying in good, rigorous classes. They wonder whether there is any valid monitoring of students’ work and reflection in service projects. They challenge instructors to demonstrate the learning in service learning through actual evidence. I set out to meet these challenges with the service assignments for the course.
Obvious places to volunteer in a Feminism and Philosophy class are the local rape crisis center and the domestic violence shelter. These places require a very serious commitment of time by volunteers, because they must go through a long training period and commit to serving for a number of hours that justifies that investment in training. These would be great places for feminist philosophy students to serve, but they require a kind of commitment that I didn’t feel I could ask of everyone. Since the main topic of feminist philosophy is the meaning, construction, and effects of gender for individuals and society, it was natural to send students to do service where gender is developed and honed. Thus I contacted agencies that serve children in daycares and schools, asking if they could use some help. Of course they could, but the kind of work to be done would be tucking children in at nap time, mopping up spilled juice, hanging out with girls and boys in the school lunchroom, or watching children play and study in after-school programs. So some service opportunities would be intense but obviously connected, others would be more laid back and require more reflection to connect. Still, all were relevant to understanding how the concept of gender plays out in life. With this set of agencies for volunteering, I thought that I had successfully met the skeptic’s first challenge of relevance.
The second challenge, monitoring effort and reflection, was more difficult. To address it, I assigned the students to not only perform the service required by the agency, but also to observe the construction of gender (and race and class) at the volunteer site, write about what they observed in journals they kept for each volunteer session, and then design an activity to engage their agency’s clients in ways that would address gender and/or racial issues in a creative, progressive, and provocative manner. I sent them to the Southern Poverty Law Center’s Teaching Tolerance web site1 for ideas on how to do this. The journals allowed some monitoring by me and facilitated students’ reflection. I followed up with supervisors on attendance, to check against their journal entries. Finally, I asked students to make explicit connections to the readings in their final writeups of their projects. While there were varying levels of quality of reflection in their journals and final writeups, these assignments met the challenge.
The final challenge, to gather evidence and document the learning in service learning, is one I have yet to fully meet. I designed the course as an experiment to test the hypothesis that service learning increases the understanding of the course reading material as measured through the grade on the final exam. The course did not require all students to participate in service learning. They could replace the service learning assignments with additional philosophical essay assignments. Roughly half the students chose to do the service, and half to do the essays; it broke down almost exactly along the lines of the enrollment: those who enrolled in the women’s studies number chose service, those in the philosophy number chose essays. However, the results on the final exam were inconclusive. I graded the exams and the essays throughout the course blindly by having the students provide only student numbers as identifiers. I found that there was no statistically significant difference in either the grades on the final exam or the final grades in the course between students who chose to do service and those who chose to do essays. Of terms of time and efforts. They made phone calls and appointments, introduced themselves to supervisors, took charge of groups of children, took directions from teachers and counselors, played with children, talked with children about bullying, race, poverty, class, sex, stereotypes, and more. Students who volunteered at the women’s shelter did more than three times the amount of service I required during the term and committed to at least 100 more hours after the term ended. Some students did follow up independent service learning courses. Some students took jobs at the places where they had volunteered because they enjoyed their work so much.
The students better understood how academic coursework is connected to life. And the youthful enthusiasm that I observed among these students was as real as any I have seen on the playing field or the stage.
course, the results imply that students who chose service did no worse on the written final exam than students who did the traditional academic work, and that in itself says something about the quality of learning in service.
In addition to responding to the skeptic’s challenges, there was another outcome that I was searching for—an increased enthusiasm and commitment to the course that I hoped service learning would generate in the students. This involves a very qualitative judgment with anecdotal evidence, yet I think the results are clear. The students who participated in service went far beyond my requirements in
Then there were the final projects and writeups. The students designed unbelievably creative projects that took many hours to prepare. One student had to design something related to gender for toddlers who are just learning language—not an easy assignment. So she designed and created large cut-outs of women and men dressed for different occupations that ran counter to gender stereotypes and props that would be appropriate for those occupations, such as fire fighters and their hose, hats, and hydrant; or a father and a baby and baby bottle; or a female chef, a chef’s hat and a frying pan. Then she played with the children and asked them to match the props and headgear with the people. The children displayed a shocking amount of genderization already at their tender ages. She caught the interaction on film and made an electronic poster about her experiences as a final writeup. Her project poster, along with several other projects, are posted on the women’s studies web site.2
The volunteers at the women’s shelter did the most creative PowerPoint slide show I have ever seen. It was animated and accompanied by well chosen feminist music. The images and explanations documented the cycle of domestic violence, relayed statistics and theories, and provided lists of resources and a bibliography. It was so powerful that the class was speechless for a few minutes after the presentation. I imagine that the slide show must have taken at least 40 hours to construct. It cannot be displayed on our web site because it is so large a file, but we played it at the women’s studies banquet at the end of the year, and it received a thunderous ovation.
The students who participated in service learning became passionate advocates for the people with whom they worked. They internalized the problems of sexism, bullying, racism, and poverty as I think students rarely do when they simply read about these issues. The students better understood how academic coursework is connected to life. And, finally, the youthful enthusiasm that I observed among these students was as real as any I have seen on the playing field or the stage.
I have also been energized by
Reflections 21
this experience. I am more committed to adding service learning to appropriate courses in the future, and certainly to every incarnation of Feminism and Philosophy. Service learning is one way to ask as much from our students as coaches ask from them, and to actually see students throw themselves into their coursework with the same level of enthusiasm that they express
on the field. u
References 1http://www.tolerance.org
2http://www.womensstudies.ku.edu/ service_learning
