Learning is Not a Mechanism: Assessment, Student Agency, and Digital Spaces

The Presentation inside:

Slide 0

Learning is not a Mechanism: Assessment, Student Agency, and Digital Spaces Jesse Stommel (@Jessifer) Photo by flickr user Kristina Alexanderson

Slide 1

Grading has become the elephant in almost every room where discussions of education are underway. Photo by flickr user mirando

Slide 2

A brief aside on “grade-grubbing.” Photo by flickr user mirando

Slide 3

Slide 4

We have built a system that puts far too much emphasis on grades, and we shouldn't blame students for the failures of that system. Photo by flickr user Peter Lee

Slide 5

Grades also motivate, in at least some small way, every tool developed by edtech software and hardware engineers. The grade has been coded into all our institutional and technological systems. Photo by flickr user Viewminder

Slide 6

Digital pedagogy is not equivalent to teachers using digital tools. Rather, digital pedagogy demands we think critically about our tools, demands we reflect actively upon our own practice. This means knowing when and how to put tools down, as much as knowing when and how to take them up. Photo by flickr user Pedro Figueiredo

Slide 7

• The large-format blackboard was first used in the U.S. in 1801. • The vacuum tube-based computer was introduced in 1946. • In the 1960s, Seymour Papert began teaching the Logo programming language to children. • The first Learning Management System, PLATO (Program Logic for Automatic Teaching Operations), was developed in 1960. At the invent of each, there was fear, resistance, and thoughtless enthusiasm. At the introduction of the Radio Lecture in the 1930s, Lloyd Allen Cook warned, “This mechanizes education and leaves the local teacher only the tasks of preparing for the broadcast and keeping order in the classroom.” This sentence is not all that different from ones we’ve read about the MOOC over the last 3 years, or about online learning over the last 25. Photo by flickr user Paul

Slide 8

Prior to the late 1700s, performance and feedback systems in U.S. Education were incredibly idiosyncratic. Throughout the 19th Century, they became increasingly comparative, numerical, and standardized. Photo by flickr user in pastel

Slide 9

An “objective” system for grading was created so systematized schooling could scale. And we’ve designed technological tools in the 20th and 21st Centuries that have allowed us to scale further. Toward standardization and away from subjectivity, human relationships, and care. Photo by flickr user Matt Barnett

Slide 10

Ranking. Metrics. Norming. Objectivity. Uniformity. Measurement. Rubrics. Outcomes. Quality. Data. Performance. Averages. Excellence. Curves. Inflation. Mastery. Standardization. Photo by flickr user www.GlynLowe.com

Slide 11

“Unless the mass of workers are to be blind cogs and pinions in the apparatus they employ, they must have some understanding of the physical and social facts behind and ahead of the material and appliances with which they are dealing.” John Dewey, Schools of To-Morrow Photo by flickr user Thomas Hawk

Slide 12

When I first taught online, I encountered the horror that is the grade book inside most learning management systems, which reduces students (often color coding them) into mere rows in a spreadsheet. I’ve watched this tool proliferate into all the institutions where I’ve worked. Even teachers that don’t use the LMS for its decidedly more pleasurable uses have made its grade book more and more central to the learning experience for students. Photo by flickr user Shelly

Slide 13

On its surface, the LMS grade book does not seem all that functionally different from an analog grade book, which also reduces students to mere rows in a spreadsheet. But most learning management systems now offer (or threaten) to automate a process which is, in fact, deeply idiosyncratic. They make grading more efficient, as though efficiency is something we ought to celebrate in teaching and learning.

Slide 14

And I find myself genuinely confused when anyone suggests there is a way for us to do this work objectively. For me, teaching and learning have always been (and will always be) deeply subjective. Photo by flickr user Mirai Takahashi

Slide 15

Learning management system grade books, often mapped to rubrics and outcomes, assume students (and their experiences) are interchangeable. Neatly comparable. Photo by flickr user Victoria Pickering

Slide 16

“ANGEL helps busy educators manage grades with flexible features that are easy to use. Automated Agents Save Time.”

Slide 17

Canvas: “Grades can serve as a communication tool between students and instructors and allow instructors to track the progress of students.”

Slide 18

“Blackboard is a course management platform that allows instructors to interact with students … from putting up copies of handouts and presentations to quizzing students on what they’ve learned to calculating student grades and putting them online.”

Slide 19

The problem is not just the fact of grades but the fetishization of them. Photo by flickr user Kristina Alexanderson

Slide 20

Slide 21

Slide 22

Slide 23

Slide 24

“Assessment tends so much to drive and control teaching. Much of what we do in the classroom is determined by the assessment structures we work under.” Peter Elbow, “Ranking, Evaluating, and Liking: Sorting Out Three Forms of Judgment” Photo by flickr user Marcus Trimble

Slide 25

We’ve built an impenetrable phalanx of clarity, certainty, and defensibility. There is no space for student agency in a system of incessant grading, ranking, and scoring. Photo by flickr user A L

Slide 26

The grade takes the complexity of human interaction within a learning environment and makes it machine-readable: [A/A-] [A-/B+] [F+] [97%] [59%] [18/20] [10/20] Photo by flickr user ninniane

Slide 27

Why do we attempt so often to resolve this... Photo by flickr user Victoria Pickering

Slide 28

Into this? Photo by flickr user Victoria Pickering

Slide 29

If there’s a better sort of mechanism we need for teaching in digital environments, it’s a machine, an algorithm, a platform tuned not for delivering and assessing content, not for ranking and sorting, but for helping all of us listen better to students. Photo by flickr user Chris Campbell

Slide 30

And, by “listen,” I decidedly do not mean “surveil.” The former implies an invitation to open dialogue, whereas the latter implies a hierarchical relationship through which learners are made into mere data points. My call, then, is for more emphasis on the tools that help us fully and genuinely inhabit digital environments, tools like ears, eyes, or fingers. Photo by flickr user Lorrie McClanahan

Slide 31

Can we imagine assessment mechanisms that encourage discovery, ones not designed for assessing learning but designed for learning through assessment? Photo by flickr user Steve Johnson

Slide 32

“The course as composition is not fundamentally instrumental, producing an article or living up to an outcome; but rather the course as composition is an action which has intrinsic value.” ~ Sean Michael Morris Photo by flickr user Giovanni Arteaga

Slide 33

Can we work to design a new approach to assessment within digital systems that focuses on formative rather than summative assessment, intrinsic rather than extrinsic motivation, relationships rather than ranking, emergence rather than predetermined outcomes? Photo by flickr user Taro Taylor

Slide 34

In “Bartleby, the Scrivener,” Herman Melville writes, “Nothing so aggravates an earnest person as a passive resistance.” Photo by flickr user Ken Douglas

Slide 35

With its incessant refrain, “I would prefer not to,” Melville’s story critiques the changing nature of labor in the 19th Century. In “Bartleby,” Melville wonders at what becomes of humans and human bodies in the wake of rapid changes in how we work and how that work gets recognized. And Bartleby’s response is a kind of civil disobedience. Photo by flickr user Sage Ross

Slide 36

When do we decide that a tool isn’t working, and how can we work together to set it down en masse? Photo by flickr user Victoria Pickering Photo by flickr user Pedro Figueiredo

Slide 37

I’ve foregone grades on individual assignments for over 13 years, relying on qualitative feedback, peer review, and self-assessment. My goal in eschewing grades has been to more honestly engage student work rather than simply evaluate it. Over many years, this has meant carefully navigating, and even breaking, the sometimes draconian rules of a half-dozen institutions. Photo by flickr user Luke Hayfield

Slide 38

Some Alternate Approaches to Assessment: * Peter Elbow’s Minimal grading A scale with only one, two, or three levels: turned in, pass/fail, strong/ satisfactory/weak. A “zero scale.” Assignments that are not collected at all. * Peer-evaluation * Self-evaluation * Portfolios or process-based grading * Contract grading * Authentic Assessment * Feedback, or even better, dialogue

Slide 39

“Nonconformity on our part was viewed with suspicion, as empty gestures of defiance aimed at masking inferiority or substandard work” (5). bell hooks, Teaching to Transgress Photo by flickr user Fio

Slide 40

Schools, and the systems we’ve invented to support them, condition us to believe there are always others (experts or even algorithms) who can know better than us the value of our own work. Photo by flickr user Jasn

Slide 41

Slide 42

When we ask students if and how they're learning, we have to just believe their answer. This data is more valuable to me than any “objective” study. Photo by flickr user Donovan Shortey

Slide 43

Can educators find ways to stop having conversations about teaching and learning technologies without students present? Photo by flickr user Fio

Slide 44

“We often ignore the best resource for informed change, one that is right in front of our noses every day—our students, for whom the most is at stake.” ~ Martin Bickman, “Returning to Community and Praxis” Photo by flickr user tai chang hsien

Slide 45