WEBVTT

00:00:01.700 --> 00:00:03.120 align:center
ERIC TREKELL: All
right, everyone.

00:00:03.120 --> 00:00:03.630 align:center
Welcome.

00:00:03.630 --> 00:00:05.540 align:center
Thank you for
joining us to discuss

00:00:05.540 --> 00:00:07.910 align:center
the topic of developing
alternative grading

00:00:07.910 --> 00:00:10.110 align:center
assessments for students.

00:00:10.110 --> 00:00:12.830 align:center
My name is Eric Trekell with the
DO-IT Center at the University

00:00:12.830 --> 00:00:13.830 align:center
of Washington.

00:00:13.830 --> 00:00:15.530 align:center
And I'm joined by
my DO-IT colleagues

00:00:15.530 --> 00:00:19.230 align:center
Kayla Brown, Wendy Huang,
Andrea Mano, and Brianna Blaser.

00:00:19.230 --> 00:00:21.500 align:center
Our speaker today is Kevin Lin.

00:00:21.500 --> 00:00:23.520 align:center
This webinar is
sponsored by AiiCE,

00:00:23.520 --> 00:00:26.210 align:center
the Alliance for
Interdisciplinary Innovations

00:00:26.210 --> 00:00:29.510 align:center
in Computing Education, and is
funded by a National Science

00:00:29.510 --> 00:00:33.920 align:center
Foundation grant
number 211-84-543.

00:00:33.920 --> 00:00:37.100 align:center
As we begin today, I'd like to
share some information regarding

00:00:37.100 --> 00:00:39.570 align:center
accessibility features
for today's meeting.

00:00:39.570 --> 00:00:42.590 align:center
This information will also
be posted in the chat.

00:00:42.590 --> 00:00:45.020 align:center
We are recording this meeting.

00:00:45.020 --> 00:00:47.000 align:center
Professional captions
are being provided

00:00:47.000 --> 00:00:49.340 align:center
and can be accessed
in an external browser

00:00:49.340 --> 00:00:55.200 align:center
window via the SteamText
link posted in the chat.

00:00:55.200 --> 00:00:58.460 align:center
Alternately, you can
utilize auto captions

00:00:58.460 --> 00:01:03.540 align:center
by selecting the CC icon
in the Zoom menu bar.

00:01:03.540 --> 00:01:05.830 align:center
We do have ASL
interpreters today.

00:01:05.830 --> 00:01:08.400 align:center
They can be multipinned
in your Zoom window

00:01:08.400 --> 00:01:11.200 align:center
if you would like to
utilize ASL interpreting.

00:01:11.200 --> 00:01:15.120 align:center
If you need multipin feature,
please request multipin

00:01:15.120 --> 00:01:18.450 align:center
by messaging one of my
colleagues, Wendy Huang or Kayla

00:01:18.450 --> 00:01:21.930 align:center
Brown, and requesting that.

00:01:21.930 --> 00:01:25.090 align:center
And now I'd like to introduce
and welcome our presenter.

00:01:25.090 --> 00:01:27.390 align:center
Kevin Lin is an assistant
teaching professor

00:01:27.390 --> 00:01:30.030 align:center
in the Paul G. Allen
School of Computing Science

00:01:30.030 --> 00:01:32.530 align:center
and Engineering at the
University of Washington.

00:01:32.530 --> 00:01:36.240 align:center
He's also codirector of the
Center for Learning, Computing

00:01:36.240 --> 00:01:37.690 align:center
and Imagination.

00:01:37.690 --> 00:01:40.200 align:center
He provides leadership on
instructional innovation

00:01:40.200 --> 00:01:42.540 align:center
and data programming
and data structures.

00:01:42.540 --> 00:01:45.300 align:center
Kevin received his MS in
computer science from UC

00:01:45.300 --> 00:01:47.820 align:center
Berkeley, where he coordinated
the teaching and delivery

00:01:47.820 --> 00:01:51.600 align:center
of very large-scale CS
undergraduate courses to over

00:01:51.600 --> 00:01:54.150 align:center
1,000 students every semester.

00:01:54.150 --> 00:01:57.840 align:center
Kevin, it's a pleasure
to have you with us.

00:01:57.840 --> 00:01:59.948 align:center
KEVIN LIN: Thank you for
the introduction, Eric.

00:01:59.948 --> 00:02:00.740 align:center
ERIC TREKELL: Yeah.

00:02:00.740 --> 00:02:02.300 align:center
Take it away.

00:02:02.300 --> 00:02:03.300 align:center
KEVIN LIN: Hi, everyone.

00:02:03.300 --> 00:02:07.500 align:center
My name is Kevin Lin, and I'm
giving the talk for today.

00:02:07.500 --> 00:02:09.410 align:center
And in this today's
talk, I'm going

00:02:09.410 --> 00:02:11.870 align:center
to reflect on my experiences
with alternative grading

00:02:11.870 --> 00:02:15.050 align:center
practices that I think
hope to better represent

00:02:15.050 --> 00:02:17.330 align:center
the learning that
students achieve over time

00:02:17.330 --> 00:02:19.170 align:center
by producing more
equitable outcomes,

00:02:19.170 --> 00:02:21.720 align:center
changing the way that we
determine final grades.

00:02:21.720 --> 00:02:23.630 align:center
And moreover, I see
alternative grading

00:02:23.630 --> 00:02:25.820 align:center
as really having the
potential to empower students

00:02:25.820 --> 00:02:28.190 align:center
by making space for
really creative student

00:02:28.190 --> 00:02:31.305 align:center
work that we might not otherwise
believe we could incorporate

00:02:31.305 --> 00:02:32.930 align:center
in our classes because
oftentimes we're

00:02:32.930 --> 00:02:34.190 align:center
thinking about
things in terms of,

00:02:34.190 --> 00:02:36.210 align:center
how many percentage points
should this be worth?

00:02:36.210 --> 00:02:38.227 align:center
Or how many points do I
need to grade this on?

00:02:38.227 --> 00:02:39.810 align:center
And that can really
restrict, I think,

00:02:39.810 --> 00:02:41.727 align:center
some of our own creativity
as faculty in terms

00:02:41.727 --> 00:02:43.550 align:center
of the assignments we give.

00:02:43.550 --> 00:02:45.050 align:center
But I'll also point
out that I think

00:02:45.050 --> 00:02:48.980 align:center
that grading policies on
their own aren't often enough.

00:02:48.980 --> 00:02:51.507 align:center
For example, here I work at
the University of Washington.

00:02:51.507 --> 00:02:53.340 align:center
There's a lot of
grade-focused culture here.

00:02:53.340 --> 00:02:54.720 align:center
I think students are
very high achieving

00:02:54.720 --> 00:02:56.390 align:center
but that also means
high achieving

00:02:56.390 --> 00:02:58.380 align:center
toward that end of
getting a certain grade

00:02:58.380 --> 00:02:59.850 align:center
in addition to learning
the material that they

00:02:59.850 --> 00:03:00.433 align:center
want to learn.

00:03:00.433 --> 00:03:03.780 align:center
So I'll also discuss some of
the challenges that I faced

00:03:03.780 --> 00:03:05.850 align:center
and how I work toward
really building

00:03:05.850 --> 00:03:10.110 align:center
better relationships between
students, educators, and grades.

00:03:10.110 --> 00:03:13.770 align:center
I will also note that I have
all the materials for this slide

00:03:13.770 --> 00:03:16.120 align:center
that you're seeing
right now on my website,

00:03:16.120 --> 00:03:18.270 align:center
and I think Eric
will be sending this

00:03:18.270 --> 00:03:20.310 align:center
out shortly, either
over Zoom chat

00:03:20.310 --> 00:03:23.880 align:center
or maybe later over the email.

00:03:23.880 --> 00:03:25.870 align:center
But the outline of the
talk goes like this.

00:03:25.870 --> 00:03:28.213 align:center
We'll first talk about the
motivation for the problem.

00:03:28.213 --> 00:03:29.880 align:center
Then I'll discuss
some of the philosophy

00:03:29.880 --> 00:03:32.700 align:center
of alternative grading before
sharing some example policies

00:03:32.700 --> 00:03:35.610 align:center
that we've used in undergraduate
computing courses here at UW.

00:03:35.610 --> 00:03:38.500 align:center
And then finally reflect on some
of the details and subtleties--

00:03:38.500 --> 00:03:39.960 align:center
that's kind of
point four there--

00:03:39.960 --> 00:03:41.910 align:center
of alternative grading.

00:03:41.910 --> 00:03:44.070 align:center
I think there will be
ways for you to engage

00:03:44.070 --> 00:03:46.497 align:center
using, for example,
Q & A. Or if you

00:03:46.497 --> 00:03:48.330 align:center
have a question on your
mind, because you're

00:03:48.330 --> 00:03:50.610 align:center
in front of your computer,
you can type it down.

00:03:50.610 --> 00:03:52.270 align:center
I'm going to go over
all the material,

00:03:52.270 --> 00:03:53.880 align:center
and I think it's probably
best to take questions more

00:03:53.880 --> 00:03:56.365 align:center
toward the end if you don't
have a particular preference

00:03:56.365 --> 00:03:56.865 align:center
for that.

00:03:59.380 --> 00:04:01.150 align:center
Jumping in, I wanted
to first start

00:04:01.150 --> 00:04:04.120 align:center
with this opening question
of really understanding

00:04:04.120 --> 00:04:09.310 align:center
what relationships are
in this issue of teaching

00:04:09.310 --> 00:04:12.340 align:center
and relationships really
being at the heart of teaching

00:04:12.340 --> 00:04:15.740 align:center
between students, educators,
between students and peers,

00:04:15.740 --> 00:04:18.440 align:center
between students and their
texts or problems or worlds.

00:04:18.440 --> 00:04:21.910 align:center
I think all of this is
really framed by this answer

00:04:21.910 --> 00:04:25.370 align:center
to this question of, why are
we gathered here to learn this?

00:04:25.370 --> 00:04:27.020 align:center
And specifically,
like I mentioned,

00:04:27.020 --> 00:04:28.240 align:center
there's that students
and teaching team.

00:04:28.240 --> 00:04:29.050 align:center
That's the "we."

00:04:29.050 --> 00:04:30.250 align:center
There's a time and place.

00:04:30.250 --> 00:04:32.523 align:center
There's here and then
this, which is the subject.

00:04:32.523 --> 00:04:33.940 align:center
And I think all
three of these are

00:04:33.940 --> 00:04:37.000 align:center
things that we as educators are
thinking about or navigating

00:04:37.000 --> 00:04:38.980 align:center
when we're conducting teaching.

00:04:38.980 --> 00:04:41.830 align:center
But I think one thing, one
relationship that I have studied

00:04:41.830 --> 00:04:45.580 align:center
a lot in discussion of this
talk, is how grading, I think,

00:04:45.580 --> 00:04:47.590 align:center
is in the background
or possibly sometimes

00:04:47.590 --> 00:04:50.350 align:center
in the foreground of all this
work that we're doing, right?

00:04:50.350 --> 00:04:52.040 align:center
Grading affects all
these relationships.

00:04:52.040 --> 00:04:53.915 align:center
It moderates our
relationships with students,

00:04:53.915 --> 00:04:55.340 align:center
how students relate
to each other,

00:04:55.340 --> 00:04:57.460 align:center
for sure, how they move
through physical spaces

00:04:57.460 --> 00:04:59.628 align:center
of our classrooms,
the policies we allow,

00:04:59.628 --> 00:05:01.670 align:center
the policies we don't
allow, the things we grade,

00:05:01.670 --> 00:05:03.670 align:center
the things we don't
grade, and also

00:05:03.670 --> 00:05:05.710 align:center
their interest in
that subject matter.

00:05:05.710 --> 00:05:07.755 align:center
So in this talk, I'll
tend to focus more so

00:05:07.755 --> 00:05:09.130 align:center
on our relationship
with students

00:05:09.130 --> 00:05:12.700 align:center
as moderated by this
grading activity

00:05:12.700 --> 00:05:15.100 align:center
that we're all
conducting here when

00:05:15.100 --> 00:05:18.760 align:center
we teach at an institution
of higher education.

00:05:18.760 --> 00:05:20.860 align:center
I wanted to start
with a story that I've

00:05:20.860 --> 00:05:24.700 align:center
learned from Mina Zavary, who's
a PhD student in Human-Centered

00:05:24.700 --> 00:05:27.740 align:center
Design and Engineering here at
the University of Washington.

00:05:27.740 --> 00:05:30.610 align:center
And one of her studies that
she's investigated is really

00:05:30.610 --> 00:05:33.790 align:center
some of the work studying
UW archives and debates from

00:05:33.790 --> 00:05:37.240 align:center
the 1970s, from UW history,
talking about changes to UW

00:05:37.240 --> 00:05:39.380 align:center
grading policies,
which it turns out,

00:05:39.380 --> 00:05:42.850 align:center
UW has this numeric grading
scale that is somewhat unusual

00:05:42.850 --> 00:05:46.660 align:center
even in the US, where instead of
assigning A's, B's, or C's, UW

00:05:46.660 --> 00:05:49.310 align:center
has this policy that
assigns on 0.1 increments.

00:05:49.310 --> 00:05:52.510 align:center
So every student in every major
course that they're taking

00:05:52.510 --> 00:05:53.930 align:center
can get a 4.0.

00:05:53.930 --> 00:05:54.980 align:center
They can get a 3.9.

00:05:54.980 --> 00:05:55.820 align:center
They can get a 3.8.

00:05:55.820 --> 00:05:57.050 align:center
They can get 3.7.

00:05:57.050 --> 00:05:59.590 align:center
And so we're assigning
grades as instructors

00:05:59.590 --> 00:06:01.270 align:center
on these 0.1 increments.

00:06:01.270 --> 00:06:02.720 align:center
But that wasn't always the case.

00:06:02.720 --> 00:06:05.740 align:center
Mina, like I mentioned, she
actually noted that this was

00:06:05.740 --> 00:06:08.770 align:center
a discussion that happened in
the 1970s at the UW University

00:06:08.770 --> 00:06:11.470 align:center
level, and there were concerns
and debates raised about

00:06:11.470 --> 00:06:15.387 align:center
changing from ABC letter grades
to these 0.1 numeric grades.

00:06:15.387 --> 00:06:17.470 align:center
Would such a change to the
University-wide grading

00:06:17.470 --> 00:06:20.150 align:center
practices actually
produce better grades?

00:06:20.150 --> 00:06:22.150 align:center
Some proponents of the
change argued that grades

00:06:22.150 --> 00:06:23.480 align:center
would become more accurate.

00:06:23.480 --> 00:06:26.140 align:center
But opponents of the
proposal suggested

00:06:26.140 --> 00:06:28.690 align:center
that actually making these
grading increments smaller

00:06:28.690 --> 00:06:31.760 align:center
would increase precision but
not necessarily accuracy.

00:06:31.760 --> 00:06:34.355 align:center
So you have numbers that
have more significant digits,

00:06:34.355 --> 00:06:36.730 align:center
but does it actually give you
more information about what

00:06:36.730 --> 00:06:38.043 align:center
students can actually achieve?

00:06:38.043 --> 00:06:40.210 align:center
And can you really tell the
difference between a 3.3

00:06:40.210 --> 00:06:41.506 align:center
and a 3.4?

00:06:41.506 --> 00:06:44.900 align:center
How would this system improve
on what we've done in the past?

00:06:44.900 --> 00:06:49.330 align:center
And really, what does
grading actually serve?

00:06:49.330 --> 00:06:52.120 align:center
And so these questions of
grading, I think, in the UW,

00:06:52.120 --> 00:06:55.140 align:center
University
Washington-specific community

00:06:55.140 --> 00:06:56.640 align:center
are part of a
larger dialogue that

00:06:56.640 --> 00:06:59.760 align:center
have happened over the
past 100 or more years.

00:06:59.760 --> 00:07:02.160 align:center
So Mark Guzdial is a
really prolific researcher

00:07:02.160 --> 00:07:04.410 align:center
in computing education
research and a professor

00:07:04.410 --> 00:07:06.180 align:center
at the University of Michigan.

00:07:06.180 --> 00:07:09.060 align:center
He recounts how the US education
system was significantly

00:07:09.060 --> 00:07:10.470 align:center
shaped by four words--

00:07:10.470 --> 00:07:11.440 align:center
Thorndike won.

00:07:11.440 --> 00:07:12.660 align:center
Dewey lost.

00:07:12.660 --> 00:07:14.790 align:center
And I'm going to read
this quote by Mark.

00:07:14.790 --> 00:07:16.903 align:center
"Dewey believed in
educating the student,

00:07:16.903 --> 00:07:18.570 align:center
meeting them where
they were and helping

00:07:18.570 --> 00:07:19.987 align:center
them to develop
in their community

00:07:19.987 --> 00:07:22.500 align:center
through teacher-driven
innovations in the classroom.

00:07:22.500 --> 00:07:24.970 align:center
Thorndike was about
administrative systems, grades,

00:07:24.970 --> 00:07:27.640 align:center
teacher credentialing, teacher
requirements and credentialing,

00:07:27.640 --> 00:07:30.750 align:center
preparing students for
vocations, testing, and teachers

00:07:30.750 --> 00:07:32.620 align:center
implementing what
researchers invent.

00:07:32.620 --> 00:07:35.700 align:center
The US education system
favors the latter."

00:07:35.700 --> 00:07:38.440 align:center
Mark goes on to quote
David Labaree's paper,

00:07:38.440 --> 00:07:39.390 align:center
"How Dewey Lost--

00:07:39.390 --> 00:07:43.470 align:center
The Victory of David Sneddon and
Social Efficiency in the Reform

00:07:43.470 --> 00:07:45.300 align:center
of American Education."

00:07:45.300 --> 00:07:47.070 align:center
"The pedagogically
progressive vision

00:07:47.070 --> 00:07:50.310 align:center
of education, child-centered,
inquiry-based, personally

00:07:50.310 --> 00:07:52.960 align:center
engaging education
is a hothouse flower

00:07:52.960 --> 00:07:56.180 align:center
trying to survive in the stony
environment of public education.

00:07:56.180 --> 00:07:57.730 align:center
It won't thrive
unless conditions

00:07:57.730 --> 00:07:59.600 align:center
are ideal since,
among other things,

00:07:59.600 --> 00:08:02.770 align:center
it requires committed,
creative, energetic, and highly

00:08:02.770 --> 00:08:05.620 align:center
educated teachers who are
willing and able to construct

00:08:05.620 --> 00:08:09.710 align:center
education to order, customized
for students in the classroom

00:08:09.710 --> 00:08:12.940 align:center
and requires broad public and
fiscal support for education

00:08:12.940 --> 00:08:15.790 align:center
as an investment in students,
rather than an investment

00:08:15.790 --> 00:08:17.652 align:center
in economic productivity.

00:08:17.652 --> 00:08:19.360 align:center
But the administrative
progressive vision

00:08:19.360 --> 00:08:21.880 align:center
of education as a
prudent investment

00:08:21.880 --> 00:08:25.100 align:center
in a socially and economically
efficient future is a weed.

00:08:25.100 --> 00:08:27.340 align:center
It will grow almost anywhere."

00:08:27.340 --> 00:08:30.760 align:center
So I think what Thorndike and
Dewey, this dialogue that Mark

00:08:30.760 --> 00:08:33.080 align:center
is foregrounding,
making apparent to us,

00:08:33.080 --> 00:08:35.320 align:center
reminding us about,
teaching us about, really,

00:08:35.320 --> 00:08:37.179 align:center
and I think it's
really kind of talking

00:08:37.179 --> 00:08:40.669 align:center
about this interesting dialogue
that happens constantly,

00:08:40.669 --> 00:08:42.669 align:center
even now today, even
more so today, you

00:08:42.669 --> 00:08:45.358 align:center
could argue, between, what
is the purpose of education?

00:08:45.358 --> 00:08:46.400 align:center
Who should be leading it?

00:08:46.400 --> 00:08:47.480 align:center
Why should we fund it?

00:08:47.480 --> 00:08:50.440 align:center
And our grading systems today
are really ultimately shaped

00:08:50.440 --> 00:08:52.880 align:center
by these administrative
progressive vision of education

00:08:52.880 --> 00:08:54.620 align:center
that ultimately won.

00:08:54.620 --> 00:08:55.440 align:center
Thorndike won.

00:08:55.440 --> 00:08:57.180 align:center
Dewey lost.

00:08:57.180 --> 00:08:59.070 align:center
So it's worth keeping
all of that in mind.

00:08:59.070 --> 00:09:02.040 align:center
And yet, as much as I've painted
a really depressing picture,

00:09:02.040 --> 00:09:03.960 align:center
I think that there
are alternatives.

00:09:03.960 --> 00:09:07.230 align:center
I think that these alternatives
actually exist in real places.

00:09:07.230 --> 00:09:10.280 align:center
So I'll give you an example
here in Washington State.

00:09:10.280 --> 00:09:13.770 align:center
Less than maybe like
two hours drive away,

00:09:13.770 --> 00:09:16.920 align:center
one and a half hour's drive
away is Olympia, Washington.

00:09:16.920 --> 00:09:20.220 align:center
We have our Evergreen State
College in Washington State.

00:09:20.220 --> 00:09:22.020 align:center
Our colleagues in
Evergreen State College,

00:09:22.020 --> 00:09:24.230 align:center
they actually don't
assign grade points

00:09:24.230 --> 00:09:28.290 align:center
like A, B, C. They don't
assign 0.1 increment grades.

00:09:28.290 --> 00:09:30.350 align:center
Instead, they write these
narrative evaluations

00:09:30.350 --> 00:09:31.017 align:center
of student work.

00:09:31.017 --> 00:09:32.933 align:center
So each faculty, when
they're assigning grades

00:09:32.933 --> 00:09:34.520 align:center
at the end of the
quarter, they write

00:09:34.520 --> 00:09:36.740 align:center
narratives that
explain, that evaluate

00:09:36.740 --> 00:09:38.790 align:center
student's academic
achievement in writing,

00:09:38.790 --> 00:09:40.550 align:center
and discuss them with
students one on one

00:09:40.550 --> 00:09:41.820 align:center
during evaluations week.

00:09:41.820 --> 00:09:44.900 align:center
So they have a separate week
for that in their courses.

00:09:44.900 --> 00:09:47.030 align:center
Students then have a
chance to write and turn

00:09:47.030 --> 00:09:49.872 align:center
in self evaluations of
how they did themselves,

00:09:49.872 --> 00:09:52.080 align:center
in addition to course
evaluations of how they thought

00:09:52.080 --> 00:09:53.110 align:center
the instructor did.

00:09:53.110 --> 00:09:55.930 align:center
So it really becomes not just
the students are evaluating you,

00:09:55.930 --> 00:09:57.270 align:center
you're evaluating the
students, but there's

00:09:57.270 --> 00:09:58.980 align:center
a lot of dialogue
and self-reflection

00:09:58.980 --> 00:10:00.188 align:center
that happens in this process.

00:10:00.188 --> 00:10:02.605 align:center
And it's something that I find
really amazing that they've

00:10:02.605 --> 00:10:05.070 align:center
actually made this work across
the entire Evergreen State

00:10:05.070 --> 00:10:06.265 align:center
College.

00:10:06.265 --> 00:10:08.890 align:center
Now, you might wonder, OK, maybe
that's just a smaller college,

00:10:08.890 --> 00:10:12.210 align:center
but even at scale at MIT,
the registrar's office has

00:10:12.210 --> 00:10:15.090 align:center
this page that talks about their
first-year grading policy that's

00:10:15.090 --> 00:10:18.180 align:center
designed to give students
time to adjust by gradually

00:10:18.180 --> 00:10:19.380 align:center
introducing grades.

00:10:19.380 --> 00:10:21.210 align:center
In the fall term,
first-year students

00:10:21.210 --> 00:10:23.730 align:center
receive a grade of
either pass, or it's

00:10:23.730 --> 00:10:24.998 align:center
dropped from their transcript.

00:10:24.998 --> 00:10:26.790 align:center
And in the spring term,
first-year students

00:10:26.790 --> 00:10:29.828 align:center
receive either A, B, C, or it's
drop from their transcript.

00:10:29.828 --> 00:10:31.620 align:center
And so I think there's
a lot of cool things

00:10:31.620 --> 00:10:34.290 align:center
that we can actually learn,
ideas that we can actually

00:10:34.290 --> 00:10:36.300 align:center
borrow from other
places that have still

00:10:36.300 --> 00:10:38.730 align:center
found a balance between
we live in this reality,

00:10:38.730 --> 00:10:40.480 align:center
we live in this current
reality right now,

00:10:40.480 --> 00:10:41.938 align:center
and how can we
actually make things

00:10:41.938 --> 00:10:45.585 align:center
better, even at this entire
university or college level?

00:10:45.585 --> 00:10:47.460 align:center
And the Evergreen State
College, because they

00:10:47.460 --> 00:10:49.880 align:center
have to advertise to students,
prospective students,

00:10:49.880 --> 00:10:52.660 align:center
they have an example of a
quote from a student here.

00:10:52.660 --> 00:10:55.540 align:center
"It's rare to have a detailed
essay from your professor

00:10:55.540 --> 00:10:57.860 align:center
that not only reflects
your skills and learning

00:10:57.860 --> 00:10:59.570 align:center
but who you are
as an individual.

00:10:59.570 --> 00:11:02.060 align:center
My transcripts all sound like
letters of recommendation.

00:11:02.060 --> 00:11:04.210 align:center
They reflect everything
I learned and everything

00:11:04.210 --> 00:11:05.507 align:center
my teachers saw within me."

00:11:05.507 --> 00:11:07.090 align:center
And I think that's
just an amazing way

00:11:07.090 --> 00:11:09.060 align:center
to imagine what
grades could be, what

00:11:09.060 --> 00:11:11.560 align:center
they could be in a world that
wasn't necessarily constrained

00:11:11.560 --> 00:11:14.290 align:center
by this idea that they
have to fit A, B C, buckets

00:11:14.290 --> 00:11:15.800 align:center
or 0.1 increments.

00:11:15.800 --> 00:11:19.090 align:center
And I want to take this as an
opportunity to urge all of us

00:11:19.090 --> 00:11:20.740 align:center
to think about, What
could those grades

00:11:20.740 --> 00:11:22.390 align:center
look like in our
classroom? because I

00:11:22.390 --> 00:11:24.730 align:center
think that's the subject
that I'm really close to dear

00:11:24.730 --> 00:11:26.925 align:center
to my heart in this talk today.

00:11:30.091 --> 00:11:31.570 align:center
So I think we
should question how

00:11:31.570 --> 00:11:33.040 align:center
we grade, because
grades moderate

00:11:33.040 --> 00:11:34.190 align:center
our relationship with students.

00:11:34.190 --> 00:11:35.290 align:center
And I think
regardless of how you

00:11:35.290 --> 00:11:37.010 align:center
feel about these different
policy proposals,

00:11:37.010 --> 00:11:39.020 align:center
I think there are strengths
and weaknesses to both.

00:11:39.020 --> 00:11:40.600 align:center
I think you can
consider scholarships

00:11:40.600 --> 00:11:43.360 align:center
and the requirements required by
scholarships, that students have

00:11:43.360 --> 00:11:45.820 align:center
to keep up with as
being one possibly

00:11:45.820 --> 00:11:47.440 align:center
mediating factor for
thinking through,

00:11:47.440 --> 00:11:50.390 align:center
how do we design
grading systems?

00:11:50.390 --> 00:11:52.180 align:center
But I think one of
the questions that

00:11:52.180 --> 00:11:55.370 align:center
was really interesting
that I did not hear so much

00:11:55.370 --> 00:11:58.450 align:center
was this question that Mina
brought up to me, that she

00:11:58.450 --> 00:12:01.180 align:center
noticed that what was notably
absent in all of this discussion

00:12:01.180 --> 00:12:02.572 align:center
was what students thought.

00:12:02.572 --> 00:12:05.030 align:center
What did students think about
changing this grading system?

00:12:05.030 --> 00:12:07.072 align:center
What did students think
about this particular way

00:12:07.072 --> 00:12:13.578 align:center
of constructing grades
around their experiences?

00:12:13.578 --> 00:12:15.370 align:center
And so I want to bring
this question up of,

00:12:15.370 --> 00:12:18.005 align:center
what are the impacts
of grading policies?

00:12:18.005 --> 00:12:20.380 align:center
So next I'll talk a little
bit about some research that's

00:12:20.380 --> 00:12:22.030 align:center
being done in this space.

00:12:22.030 --> 00:12:25.450 align:center
And then I'll be more concrete
about what we've done here in CS

00:12:25.450 --> 00:12:28.210 align:center
in particular at UW.

00:12:28.210 --> 00:12:30.370 align:center
So first, I want to claim
that I think policies

00:12:30.370 --> 00:12:31.670 align:center
have really big impacts.

00:12:31.670 --> 00:12:34.180 align:center
So Eman Sherif, one
of our PhD students

00:12:34.180 --> 00:12:37.240 align:center
in computer science engineering,
she does interesting research

00:12:37.240 --> 00:12:39.070 align:center
at the intersection
of assessment

00:12:39.070 --> 00:12:41.270 align:center
and marginalized
student experiences.

00:12:41.270 --> 00:12:44.260 align:center
She has a paper at ICER
2024 here called "Exploring

00:12:44.260 --> 00:12:47.110 align:center
the Impact of Assessment
Policies on Marginalized

00:12:47.110 --> 00:12:49.270 align:center
Students' Experiences in
Post-Secondary Programming

00:12:49.270 --> 00:12:50.778 align:center
Courses."

00:12:50.778 --> 00:12:52.570 align:center
And through these
semistructured interviews

00:12:52.570 --> 00:12:54.820 align:center
of students with
underrepresented social

00:12:54.820 --> 00:12:57.550 align:center
identities in computing,
Eman identified 10 ways

00:12:57.550 --> 00:12:59.410 align:center
that policies interacted
with student lives

00:12:59.410 --> 00:13:01.420 align:center
to create or
heighten inequities.

00:13:01.420 --> 00:13:04.060 align:center
For instance, one student
she interviewed really

00:13:04.060 --> 00:13:07.180 align:center
didn't want to miss
opportunities for extra credit

00:13:07.180 --> 00:13:10.630 align:center
because previously the professor
didn't explain expectations

00:13:10.630 --> 00:13:13.180 align:center
for assignments, causing
her to perform poorly

00:13:13.180 --> 00:13:14.840 align:center
on the majority of assignments.

00:13:14.840 --> 00:13:16.600 align:center
So one of the quotes
here is like, "I even

00:13:16.600 --> 00:13:18.350 align:center
remember my family was
like, what are you doing?

00:13:18.350 --> 00:13:19.400 align:center
You're sick with COVID."

00:13:19.400 --> 00:13:21.817 align:center
And then she felt like, "Yeah,
I got to go to quiz section

00:13:21.817 --> 00:13:23.272 align:center
in case he gives extra credit."

00:13:23.272 --> 00:13:24.730 align:center
Our assessment and
grading policies

00:13:24.730 --> 00:13:26.570 align:center
have real impacts
on student lives.

00:13:26.570 --> 00:13:29.230 align:center
When students are forced to
decide between prioritizing

00:13:29.230 --> 00:13:31.600 align:center
their course policy
and themselves,

00:13:31.600 --> 00:13:33.880 align:center
their personal
responsibilities, they often

00:13:33.880 --> 00:13:36.035 align:center
choose to follow course policy.

00:13:36.035 --> 00:13:38.410 align:center
So what does this really
reflect about our relationships?

00:13:38.410 --> 00:13:39.840 align:center
is the question I want to ask.

00:13:39.840 --> 00:13:42.340 align:center
If students feel like they have
to choose between doing what

00:13:42.340 --> 00:13:44.810 align:center
is best for themselves versus
what's best for their grade,

00:13:44.810 --> 00:13:46.310 align:center
what does this mean
for students who

00:13:46.310 --> 00:13:49.490 align:center
need to miss a few sections
due to inflexible part-time job

00:13:49.490 --> 00:13:52.138 align:center
shifts, caretaking
responsibilities, students who

00:13:52.138 --> 00:13:54.680 align:center
are behind on course materials
and wouldn't find quiz section

00:13:54.680 --> 00:13:56.100 align:center
beneficial anyways, right?

00:13:56.100 --> 00:13:57.770 align:center
There can be so many
reasons why they

00:13:57.770 --> 00:14:01.370 align:center
may want to opt out of an
activity on a certain day.

00:14:01.370 --> 00:14:04.607 align:center
And these participation policies
can really affect final grades.

00:14:04.607 --> 00:14:06.440 align:center
At the same time, we
think that-- at least I

00:14:06.440 --> 00:14:08.280 align:center
think that participation
is important.

00:14:08.280 --> 00:14:10.543 align:center
And without the extrinsic
motivation of grades,

00:14:10.543 --> 00:14:11.960 align:center
I find that many
students are less

00:14:11.960 --> 00:14:14.390 align:center
likely to prioritize it
compared to other work that's

00:14:14.390 --> 00:14:15.327 align:center
less flexible.

00:14:15.327 --> 00:14:17.160 align:center
So because students are
taking many courses,

00:14:17.160 --> 00:14:18.952 align:center
they may see that this
assignment is graded

00:14:18.952 --> 00:14:21.230 align:center
and it's due, and
then therefore it's

00:14:21.230 --> 00:14:23.713 align:center
important versus, say,
participation in a class.

00:14:23.713 --> 00:14:24.630 align:center
Maybe it's not graded.

00:14:24.630 --> 00:14:26.672 align:center
Maybe it's not as important.

00:14:26.672 --> 00:14:28.380 align:center
But just as grades
can motivate students,

00:14:28.380 --> 00:14:30.680 align:center
I think they can also
demotivate students, too.

00:14:30.680 --> 00:14:33.530 align:center
Robert Talbert, a
professor of mathematics

00:14:33.530 --> 00:14:36.500 align:center
at Grand Valley State
University, talks

00:14:36.500 --> 00:14:39.020 align:center
about this idea about--

00:14:39.020 --> 00:14:41.030 align:center
it's really a story
that he had that got

00:14:41.030 --> 00:14:42.530 align:center
him really interested
in alternative

00:14:42.530 --> 00:14:44.525 align:center
grading in the first place.

00:14:44.525 --> 00:14:46.400 align:center
So he's talking about
a student in his class.

00:14:46.400 --> 00:14:48.417 align:center
She crashed and burned
on the first exam

00:14:48.417 --> 00:14:50.500 align:center
and was eliminated from
getting an A in the class.

00:14:50.500 --> 00:14:54.470 align:center
In one shot, no A. Second
exam, same thing-- in one shot,

00:14:54.470 --> 00:14:56.080 align:center
she can't get a B in the class.

00:14:56.080 --> 00:14:59.110 align:center
And I sat there and
just watched her sense

00:14:59.110 --> 00:15:01.090 align:center
of self-worth and her
excitement in the class

00:15:01.090 --> 00:15:03.640 align:center
just decay away
right before my eyes.

00:15:03.640 --> 00:15:06.212 align:center
And I think that's
something that maybe you

00:15:06.212 --> 00:15:07.670 align:center
may have experienced
this yourself,

00:15:07.670 --> 00:15:09.587 align:center
or maybe you've experienced
from someone else.

00:15:09.587 --> 00:15:12.443 align:center
But it's something that I think
really rings deeply to my heart,

00:15:12.443 --> 00:15:14.110 align:center
too, because I've
personally experienced

00:15:14.110 --> 00:15:17.400 align:center
that in my own course and the
way that I grade it as well.

00:15:20.350 --> 00:15:23.140 align:center
So all this that I've said
in the first 15 minutes or so

00:15:23.140 --> 00:15:26.200 align:center
is that I think it's really
interesting that grading

00:15:26.200 --> 00:15:29.350 align:center
is an activity that moderates
our relationship with students

00:15:29.350 --> 00:15:30.290 align:center
and teachers.

00:15:30.290 --> 00:15:31.795 align:center
It's not really
only that students

00:15:31.795 --> 00:15:33.920 align:center
have to choose between
themselves and their grades.

00:15:33.920 --> 00:15:36.650 align:center
When one student asks for
extra time, as an instructor,

00:15:36.650 --> 00:15:39.700 align:center
I'm also thinking to myself,
can I offer that extra time

00:15:39.700 --> 00:15:40.910 align:center
to everyone in the class?

00:15:40.910 --> 00:15:43.300 align:center
How do I design my
policy so that, if I'm

00:15:43.300 --> 00:15:46.470 align:center
offering this person extra time
on the assessment, is it fair?

00:15:46.470 --> 00:15:49.910 align:center
Can I apply that policy the
same to everyone in my class?

00:15:49.910 --> 00:15:51.810 align:center
Or if a student struggled
with an assignment,

00:15:51.810 --> 00:15:53.250 align:center
can I give them an extra makeup?

00:15:53.250 --> 00:15:54.990 align:center
And in the large
classes that I teach,

00:15:54.990 --> 00:15:57.440 align:center
I really feel like
I really often

00:15:57.440 --> 00:15:59.060 align:center
worry about all the
students who don't

00:15:59.060 --> 00:16:01.760 align:center
feel like it's their prerogative
to ask me these questions

00:16:01.760 --> 00:16:03.900 align:center
or request exceptions
to our policies.

00:16:03.900 --> 00:16:05.930 align:center
And the end effect is
that really my own course

00:16:05.930 --> 00:16:07.045 align:center
policies bind me.

00:16:07.045 --> 00:16:08.670 align:center
They really restrict
my own creativity.

00:16:08.670 --> 00:16:11.850 align:center
They restrict what I feel like
I can do to support students

00:16:11.850 --> 00:16:14.697 align:center
because I'm just constantly
feeling like I have to be fair.

00:16:14.697 --> 00:16:17.030 align:center
But at the same time, I really
want to support students.

00:16:17.030 --> 00:16:21.020 align:center
And that puts me in a
difficult situation.

00:16:21.020 --> 00:16:23.300 align:center
And so what personally got
me started on this path

00:16:23.300 --> 00:16:25.200 align:center
was a desire to
right these wrongs.

00:16:25.200 --> 00:16:28.040 align:center
And just like the story Robert
shared about these grading

00:16:28.040 --> 00:16:31.280 align:center
policies trapping him in a box,
in my first quarter teaching

00:16:31.280 --> 00:16:34.430 align:center
at UW here, I also accused an
embarrassingly large number

00:16:34.430 --> 00:16:36.207 align:center
of my students for
academic misconduct.

00:16:36.207 --> 00:16:37.790 align:center
And I think there's
a lot of questions

00:16:37.790 --> 00:16:39.600 align:center
that I have about
my own actions,

00:16:39.600 --> 00:16:41.795 align:center
about the things that
I've done to students

00:16:41.795 --> 00:16:44.170 align:center
and the relationships that
I've had that I'm honestly not

00:16:44.170 --> 00:16:44.960 align:center
very proud of.

00:16:44.960 --> 00:16:47.530 align:center
I'm very ashamed of some of
these things that I've done.

00:16:47.530 --> 00:16:49.510 align:center
But all of this got
me thinking that,

00:16:49.510 --> 00:16:50.960 align:center
could we change the structures?

00:16:50.960 --> 00:16:52.760 align:center
Can we change the
rules of the game

00:16:52.760 --> 00:16:55.640 align:center
so that we can actually avoid
things like academic misconduct,

00:16:55.640 --> 00:16:58.810 align:center
that we can avoid issues
like, hey, Dwight, can

00:16:58.810 --> 00:17:01.570 align:center
I offer this revision
opportunity or this extension

00:17:01.570 --> 00:17:04.270 align:center
to someone if that
means offering

00:17:04.270 --> 00:17:05.630 align:center
that opportunity to everyone?

00:17:05.630 --> 00:17:10.368 align:center
And that's what got me started
on alternative grading here.

00:17:10.368 --> 00:17:12.160 align:center
So alternative grading,
I think, can really

00:17:12.160 --> 00:17:15.130 align:center
help us design systems that
align grading practices

00:17:15.130 --> 00:17:16.930 align:center
with our own learning values.

00:17:16.930 --> 00:17:19.119 align:center
And broadly to define
that term, it's

00:17:19.119 --> 00:17:21.520 align:center
a broad umbrella term
referring to grading practices

00:17:21.520 --> 00:17:23.260 align:center
that recognize and
value the learning

00:17:23.260 --> 00:17:25.300 align:center
that students achieve over time.

00:17:25.300 --> 00:17:27.640 align:center
I'll quote Drew Lewis here,
who has these three bullet

00:17:27.640 --> 00:17:30.263 align:center
points for a shared
philosophy among

00:17:30.263 --> 00:17:32.680 align:center
the many different alternative
grading schemes people have

00:17:32.680 --> 00:17:35.080 align:center
invented.

00:17:35.080 --> 00:17:36.910 align:center
Students should have
multiple opportunities

00:17:36.910 --> 00:17:38.560 align:center
to demonstrate proficiency.

00:17:38.560 --> 00:17:40.330 align:center
Grades should reflect
the proficiency

00:17:40.330 --> 00:17:42.500 align:center
that students
demonstrate by the end

00:17:42.500 --> 00:17:44.870 align:center
and that grades and points
are de-emphasized in favor

00:17:44.870 --> 00:17:46.930 align:center
of rich, deep feedback.

00:17:46.930 --> 00:17:49.430 align:center
And so there are many different
types of alternative grading

00:17:49.430 --> 00:17:50.870 align:center
systems you might have heard of.

00:17:50.870 --> 00:17:52.860 align:center
Maybe you've heard of a
specifications grading,

00:17:52.860 --> 00:17:54.530 align:center
standards grading, ungrading.

00:17:54.530 --> 00:17:56.930 align:center
These are all different
ideas that share

00:17:56.930 --> 00:18:00.110 align:center
many of these elements
that Drew has pointed out.

00:18:00.110 --> 00:18:03.510 align:center
But rather than try to explain
each of these one by one,

00:18:03.510 --> 00:18:06.200 align:center
I wanted to start by
how changes to policies

00:18:06.200 --> 00:18:08.210 align:center
help me produce more
equitable grades without,

00:18:08.210 --> 00:18:11.210 align:center
say, throwing out my
entire grading system.

00:18:11.210 --> 00:18:13.410 align:center
I think there are even
small things we can do.

00:18:13.410 --> 00:18:16.310 align:center
So one small thing that
I feel like, if any of us

00:18:16.310 --> 00:18:18.740 align:center
are using final exams,
an easy one to do

00:18:18.740 --> 00:18:21.800 align:center
is this idea of final
exam clobbering.

00:18:21.800 --> 00:18:24.360 align:center
In STEM courses, like the
ones that I tend to teach,

00:18:24.360 --> 00:18:25.820 align:center
midterms and finals
are oftentimes

00:18:25.820 --> 00:18:28.040 align:center
used as this form of summative
assessment of student

00:18:28.040 --> 00:18:29.630 align:center
achievement and proficiency.

00:18:29.630 --> 00:18:31.790 align:center
And in the past, it was
always difficult for me

00:18:31.790 --> 00:18:34.325 align:center
to explain to students who
did poorly on a midterm exam

00:18:34.325 --> 00:18:36.200 align:center
that they could demonstrate
their improvement

00:18:36.200 --> 00:18:38.360 align:center
on the final exam
because the typical way

00:18:38.360 --> 00:18:40.830 align:center
that, if you have a
points-based system,

00:18:40.830 --> 00:18:42.840 align:center
you've already established
some amount of points

00:18:42.840 --> 00:18:43.810 align:center
for the midterm.

00:18:43.810 --> 00:18:45.900 align:center
And that whatever you've
lost in that midterm

00:18:45.900 --> 00:18:47.740 align:center
is lost forever on your record.

00:18:47.740 --> 00:18:49.740 align:center
There's no way to
make it up, right?

00:18:49.740 --> 00:18:52.590 align:center
But rather it becomes
really this grading deficit

00:18:52.590 --> 00:18:54.210 align:center
for students.

00:18:54.210 --> 00:18:57.855 align:center
So in my mind, I think,
to align these grading

00:18:57.855 --> 00:19:00.480 align:center
policies, to change my grading
policy so that it could actually

00:19:00.480 --> 00:19:02.350 align:center
capture students
learning over time,

00:19:02.350 --> 00:19:05.430 align:center
maybe you could actually
replace that midterm score

00:19:05.430 --> 00:19:08.820 align:center
with some corresponding
portion of the final exam.

00:19:08.820 --> 00:19:10.495 align:center
So we call that final
exam clobbering.

00:19:10.495 --> 00:19:12.870 align:center
So for those of you who missed
an exam, have a bad night,

00:19:12.870 --> 00:19:15.130 align:center
or make major improvements
over the semester,

00:19:15.130 --> 00:19:16.830 align:center
the exam clobbering
policy gives you

00:19:16.830 --> 00:19:18.450 align:center
a chance to use your
final exam score

00:19:18.450 --> 00:19:20.658 align:center
to make up some of the points
on your midterm scores.

00:19:20.658 --> 00:19:22.617 align:center
You don't have to allow
them to make it all up,

00:19:22.617 --> 00:19:24.220 align:center
but maybe you can
make up some amount.

00:19:24.220 --> 00:19:27.550 align:center
The clobber policy will only be
applied if it helps your score.

00:19:27.550 --> 00:19:29.872 align:center
For example, if you score
the median on both midterms

00:19:29.872 --> 00:19:32.080 align:center
but then have a bad day and
do terribly in the final,

00:19:32.080 --> 00:19:34.325 align:center
we will not change
your midterm scores.

00:19:34.325 --> 00:19:35.700 align:center
So that way it's
still benefiting

00:19:35.700 --> 00:19:38.325 align:center
students who are really learning
and demonstrating what they're

00:19:38.325 --> 00:19:40.720 align:center
learning during the term.

00:19:40.720 --> 00:19:42.940 align:center
The clobber policy can
only be used to make up

00:19:42.940 --> 00:19:44.000 align:center
points on the midterms.

00:19:44.000 --> 00:19:45.650 align:center
For example, if you score
well in the midterms,

00:19:45.650 --> 00:19:47.900 align:center
that cannot be used to make
up points on the final exam.

00:19:47.900 --> 00:19:49.300 align:center
So we are still
expecting that they

00:19:49.300 --> 00:19:51.217 align:center
are able to demonstrate
all the skills that we

00:19:51.217 --> 00:19:54.310 align:center
expect by the end of the quarter
in that typical final exam.

00:19:54.310 --> 00:19:56.320 align:center
And you can do all of
this without even changing

00:19:56.320 --> 00:19:58.510 align:center
how you're doing traditional
points-based grading

00:19:58.510 --> 00:20:00.350 align:center
or whatever grading
system you're using.

00:20:00.350 --> 00:20:03.820 align:center
There are ways that you can
use future or later assessments

00:20:03.820 --> 00:20:06.140 align:center
of student proficiency
and think about,

00:20:06.140 --> 00:20:09.933 align:center
can I use that and apply it
back to earlier assessments that

00:20:09.933 --> 00:20:12.100 align:center
may not be accurate
representations of what students

00:20:12.100 --> 00:20:13.570 align:center
know by the end of the term?

00:20:13.570 --> 00:20:15.850 align:center
Now, that's still really a
conversation and reflection

00:20:15.850 --> 00:20:18.310 align:center
for yourself about like, is
this an appropriate assessment

00:20:18.310 --> 00:20:19.430 align:center
that I can map back?

00:20:19.430 --> 00:20:21.440 align:center
Are the concepts mapped
back effectively?

00:20:21.440 --> 00:20:23.240 align:center
Am I asking the same
type of questions?

00:20:23.240 --> 00:20:24.920 align:center
There's still a lot
of nuances in that,

00:20:24.920 --> 00:20:27.097 align:center
but I wanted to raise this
question that I think,

00:20:27.097 --> 00:20:29.680 align:center
even though maybe there are all
these kind of really ambitious

00:20:29.680 --> 00:20:32.170 align:center
ways you can change your grading
policy, like the examples

00:20:32.170 --> 00:20:34.190 align:center
we saw with narrative
evaluations, even,

00:20:34.190 --> 00:20:37.238 align:center
but I think there are even these
small ways where we can look at,

00:20:37.238 --> 00:20:39.780 align:center
hey, I actually want my grades
to represent what students can

00:20:39.780 --> 00:20:42.300 align:center
do at the end of the term,
that surprisingly don't

00:20:42.300 --> 00:20:43.960 align:center
take too much more work to do.

00:20:43.960 --> 00:20:48.185 align:center
This is just changing your
grading spreadsheet, basically.

00:20:48.185 --> 00:20:50.310 align:center
Now I want to turn my
attention to something that's

00:20:50.310 --> 00:20:51.810 align:center
more specific in
some of the courses

00:20:51.810 --> 00:20:54.280 align:center
that I've taught and
supported in development.

00:20:54.280 --> 00:20:57.330 align:center
So in the introductory
courses I teach, we oftentimes

00:20:57.330 --> 00:21:00.240 align:center
have these frequent weekly
or multiweek programming

00:21:00.240 --> 00:21:04.320 align:center
assignments that apply concepts
in more authentic settings.

00:21:04.320 --> 00:21:05.715 align:center
So historically,
these were like,

00:21:05.715 --> 00:21:10.305 align:center
we have a TA spend a lot of time
reviewing these assignments.

00:21:10.305 --> 00:21:12.180 align:center
And the students would
oftentimes really want

00:21:12.180 --> 00:21:13.990 align:center
to know why they lost points.

00:21:13.990 --> 00:21:16.930 align:center
And just like as an exam, if you
lose points on an assignment,

00:21:16.930 --> 00:21:19.320 align:center
it's not so easy to
recover those points when

00:21:19.320 --> 00:21:21.758 align:center
it comes to final grade, right?

00:21:21.758 --> 00:21:23.550 align:center
And it came to a point
that really students

00:21:23.550 --> 00:21:27.845 align:center
who entered this course with
credit would oftentimes--

00:21:27.845 --> 00:21:29.970 align:center
with enough credit to skip
our introductory courses

00:21:29.970 --> 00:21:32.478 align:center
would still go back and retake
it, retake those courses

00:21:32.478 --> 00:21:34.020 align:center
that they technically
had credit for,

00:21:34.020 --> 00:21:36.720 align:center
just so they could get practice
with understanding our grading

00:21:36.720 --> 00:21:41.630 align:center
systems and understanding, how
did we want to evaluate them?

00:21:41.630 --> 00:21:43.880 align:center
And that just felt like a
really difficult environment

00:21:43.880 --> 00:21:45.220 align:center
because some of our
introductory courses

00:21:45.220 --> 00:21:46.595 align:center
were really intended
for students

00:21:46.595 --> 00:21:49.300 align:center
with no prior
programming experience.

00:21:49.300 --> 00:21:51.250 align:center
So to address this,
we tried to make it

00:21:51.250 --> 00:21:54.490 align:center
so that the following
courses, the courses that most

00:21:54.490 --> 00:21:56.780 align:center
students are taking in
are programming sequence,

00:21:56.780 --> 00:22:00.223 align:center
we give them opportunities
for revision and resubmission.

00:22:00.223 --> 00:22:02.140 align:center
Students are encouraged
to revise and resubmit

00:22:02.140 --> 00:22:03.440 align:center
to address feedback.

00:22:03.440 --> 00:22:06.070 align:center
And so this not
only helps students

00:22:06.070 --> 00:22:08.660 align:center
provide a grade-based incentive
to improve their work.

00:22:08.660 --> 00:22:11.530 align:center
But it also turned
this the manual labor

00:22:11.530 --> 00:22:13.960 align:center
of grading into feedback that
students were incentivized

00:22:13.960 --> 00:22:16.120 align:center
to learn from, rather than
just see the point value

00:22:16.120 --> 00:22:18.110 align:center
and be like, well,
that's my points.

00:22:18.110 --> 00:22:19.720 align:center
I can't change that, right?

00:22:19.720 --> 00:22:21.958 align:center
So the specific
policy is listed here.

00:22:21.958 --> 00:22:23.500 align:center
And we talk about
both the motivation

00:22:23.500 --> 00:22:25.780 align:center
for the policy to
students in the syllabus

00:22:25.780 --> 00:22:27.700 align:center
by first saying like,
learning from mistakes

00:22:27.700 --> 00:22:29.750 align:center
is an important part
of mastering any skill,

00:22:29.750 --> 00:22:31.220 align:center
especially for novices.

00:22:31.220 --> 00:22:33.580 align:center
To enable this, you are
allowed to revise and resubmit

00:22:33.580 --> 00:22:35.080 align:center
your work on
programming assignments

00:22:35.080 --> 00:22:37.410 align:center
and creative projects to
demonstrate improved mastery

00:22:37.410 --> 00:22:38.678 align:center
after your initial submission.

00:22:38.678 --> 00:22:40.720 align:center
Resubmissions are subject
to the following rules,

00:22:40.720 --> 00:22:43.000 align:center
and then there are some
practical rules around that.

00:22:43.000 --> 00:22:45.622 align:center
So for example, there will be
eight opportunities for you

00:22:45.622 --> 00:22:47.580 align:center
to make a resubmission
after receiving feedback

00:22:47.580 --> 00:22:51.010 align:center
on your work, which is basically
corresponding to one every week.

00:22:51.010 --> 00:22:53.467 align:center
So we have a general
cadence for how much

00:22:53.467 --> 00:22:55.800 align:center
labor that the TAs are willing
to put in because the TAs

00:22:55.800 --> 00:22:58.260 align:center
are paid, and they're limited
in the number of hours

00:22:58.260 --> 00:23:00.443 align:center
they can offer to students.

00:23:00.443 --> 00:23:02.110 align:center
And as well as some
other general rules,

00:23:02.110 --> 00:23:03.985 align:center
like you have to first
get initial submission

00:23:03.985 --> 00:23:07.830 align:center
feedback before you can
get your resubmission.

00:23:07.830 --> 00:23:09.330 align:center
And there are also
time limits, too,

00:23:09.330 --> 00:23:12.360 align:center
that an assignment may only
be resubmitted in the three

00:23:12.360 --> 00:23:14.250 align:center
resubmission cycles
or three weeks

00:23:14.250 --> 00:23:17.080 align:center
after feedback for the
assignment has been released.

00:23:17.080 --> 00:23:19.590 align:center
So that way the TAs don't have
to keep in mind that there

00:23:19.590 --> 00:23:21.120 align:center
are all these assignments
that are building up

00:23:21.120 --> 00:23:22.830 align:center
and they have to keep like
seven or eight assignments

00:23:22.830 --> 00:23:23.500 align:center
in their head.

00:23:23.500 --> 00:23:25.042 align:center
They only have to
keep the three that

00:23:25.042 --> 00:23:27.720 align:center
are most recent in their head.

00:23:27.720 --> 00:23:30.420 align:center
So all of that, you can
see these restrictions are

00:23:30.420 --> 00:23:32.580 align:center
things that I think
we are motivating

00:23:32.580 --> 00:23:33.933 align:center
as practical considerations.

00:23:33.933 --> 00:23:35.850 align:center
But you can also possibly
see that maybe there

00:23:35.850 --> 00:23:38.300 align:center
are some learning-based
motivations for this,

00:23:38.300 --> 00:23:41.480 align:center
too, that maybe having
spaced out opportunities

00:23:41.480 --> 00:23:43.940 align:center
can be useful, even
though maybe these might

00:23:43.940 --> 00:23:47.840 align:center
run into the wall of
restrictions in the end as well.

00:23:47.840 --> 00:23:51.020 align:center
Now, in practice, though, since
there are very limited number

00:23:51.020 --> 00:23:53.030 align:center
of weeks in the quarter,
it is very important

00:23:53.030 --> 00:23:55.070 align:center
that you as a student
stay on top of your work

00:23:55.070 --> 00:23:56.973 align:center
as much as possible.

00:23:56.973 --> 00:23:58.640 align:center
Our resubmission
policy is designed such

00:23:58.640 --> 00:24:00.560 align:center
that you should only be
using a single resubmission

00:24:00.560 --> 00:24:02.670 align:center
on any particular assignment
throughout the quarter.

00:24:02.670 --> 00:24:04.170 align:center
So there is a little
bit of pressure

00:24:04.170 --> 00:24:06.140 align:center
for students to be
turning in their best work

00:24:06.140 --> 00:24:08.400 align:center
as much as they can on
the time that it's due

00:24:08.400 --> 00:24:10.460 align:center
because the TAs only have
a fixed amount of time

00:24:10.460 --> 00:24:11.960 align:center
per week to grade.

00:24:11.960 --> 00:24:13.810 align:center
And then using that
feedback, we're

00:24:13.810 --> 00:24:15.560 align:center
giving students the
opportunity to improve

00:24:15.560 --> 00:24:17.445 align:center
what they've submitted.

00:24:17.445 --> 00:24:19.820 align:center
So all of this comes together
to make the policy actually

00:24:19.820 --> 00:24:22.350 align:center
reasonable in these large
courses that we're working with.

00:24:22.350 --> 00:24:24.240 align:center
But I think even if you're
teaching a smaller course,

00:24:24.240 --> 00:24:25.823 align:center
you might wonder,
maybe there are ways

00:24:25.823 --> 00:24:27.450 align:center
that I could adapt the system.

00:24:27.450 --> 00:24:29.567 align:center
Maybe you could
say, hey, instead

00:24:29.567 --> 00:24:31.650 align:center
of having students resubmit
the entire assignment,

00:24:31.650 --> 00:24:34.220 align:center
maybe there are particular key
points I want them to address,

00:24:34.220 --> 00:24:36.603 align:center
maybe two or three questions
I want students to answer.

00:24:36.603 --> 00:24:38.520 align:center
And by answering those
two or three questions,

00:24:38.520 --> 00:24:40.410 align:center
maybe students can
demonstrate improved learning.

00:24:40.410 --> 00:24:41.952 align:center
And it doesn't take
too much time out

00:24:41.952 --> 00:24:43.355 align:center
of my sight to do that as well.

00:24:43.355 --> 00:24:45.980 align:center
So I think there are different
ways you can scale the system up

00:24:45.980 --> 00:24:48.360 align:center
in terms of work or scale
it down in terms of work,

00:24:48.360 --> 00:24:53.150 align:center
based on what you
have available to you.

00:24:53.150 --> 00:24:56.852 align:center
By focusing on this
more limited ESN

00:24:56.852 --> 00:24:59.060 align:center
scale, so the way we actually
grade students work is,

00:24:59.060 --> 00:25:01.760 align:center
instead of using points, we use
this table on the right that

00:25:01.760 --> 00:25:04.340 align:center
shows, for the four
dimensions of quality,

00:25:04.340 --> 00:25:07.800 align:center
we care about students code
behavior, code concepts,

00:25:07.800 --> 00:25:09.810 align:center
code quality, and code testing.

00:25:09.810 --> 00:25:11.660 align:center
For those four dimensions,
we'll rate them

00:25:11.660 --> 00:25:15.170 align:center
on a three-point scale,
excellent, satisfactory,

00:25:15.170 --> 00:25:17.630 align:center
or not yet, as an example.

00:25:17.630 --> 00:25:20.670 align:center
This helps us communicate what
area students should improve.

00:25:20.670 --> 00:25:24.660 align:center
So we're no longer tied to,
oh, it's 15 out of 16 points.

00:25:24.660 --> 00:25:26.793 align:center
And you have to, as a
student, figure out,

00:25:26.793 --> 00:25:28.460 align:center
what is that one point
that I'm missing?

00:25:28.460 --> 00:25:30.477 align:center
Here we're able
to say, hey, these

00:25:30.477 --> 00:25:32.060 align:center
are the particular
areas that we think

00:25:32.060 --> 00:25:33.373 align:center
you could improve the best on.

00:25:33.373 --> 00:25:35.040 align:center
And then the feedback
that we provide in

00:25:35.040 --> 00:25:36.692 align:center
that assignment
will help identify,

00:25:36.692 --> 00:25:37.900 align:center
hey, you should work on this.

00:25:37.900 --> 00:25:38.890 align:center
You should work on this.

00:25:38.890 --> 00:25:40.682 align:center
And once you've addressed
all those points,

00:25:40.682 --> 00:25:42.840 align:center
then I think you'll
be good to go.

00:25:42.840 --> 00:25:45.773 align:center
So I think as we're moving
more toward revision,

00:25:45.773 --> 00:25:47.940 align:center
then that also puts a little
bit more pressure on us

00:25:47.940 --> 00:25:49.740 align:center
as instructors to
provide feedback that's

00:25:49.740 --> 00:25:51.500 align:center
more directly
helpful for students

00:25:51.500 --> 00:25:53.000 align:center
to address those
points of revision.

00:25:57.750 --> 00:26:00.270 align:center
Finally, I'll talk a little bit
about specifications grading

00:26:00.270 --> 00:26:02.890 align:center
and then switch over
to a different topic.

00:26:02.890 --> 00:26:06.550 align:center
So if we have all these ESN
grades that I've suggested,

00:26:06.550 --> 00:26:09.390 align:center
so instead of having point
values for our assignments,

00:26:09.390 --> 00:26:10.830 align:center
because we're
focusing on feedback

00:26:10.830 --> 00:26:13.260 align:center
and we want the grading
for the assignment

00:26:13.260 --> 00:26:16.440 align:center
to reflect that feedback, we've
been using that ESN scale,

00:26:16.440 --> 00:26:18.760 align:center
the way that we assign
final grades in the course

00:26:18.760 --> 00:26:20.980 align:center
also relies on that ESN scale.

00:26:20.980 --> 00:26:24.540 align:center
So instead of saying, hey, a
3.5 is getting 90% of the course

00:26:24.540 --> 00:26:27.900 align:center
overall, we're instead looking
at how many S's and how many

00:26:27.900 --> 00:26:29.110 align:center
E's you've received.

00:26:29.110 --> 00:26:31.380 align:center
So how many
satisfactory evaluations

00:26:31.380 --> 00:26:36.680 align:center
and exemplary evaluations have
you received from an instructor?

00:26:36.680 --> 00:26:38.443 align:center
We call this
specifications grading,

00:26:38.443 --> 00:26:39.860 align:center
or bundle-based
grading, depending

00:26:39.860 --> 00:26:41.027 align:center
on what you want to call it.

00:26:41.027 --> 00:26:43.650 align:center
But basically, you can say that,
hey, in the grading system,

00:26:43.650 --> 00:26:46.160 align:center
we can have a minimum
grade of 3.5 correspond

00:26:46.160 --> 00:26:48.840 align:center
to a certain number
of S's or higher

00:26:48.840 --> 00:26:50.940 align:center
and a certain number
of E's or higher.

00:26:50.940 --> 00:26:52.670 align:center
So by the end of
the term, students

00:26:52.670 --> 00:26:56.090 align:center
should have demonstrated that
amount of S's or E's to receive

00:26:56.090 --> 00:26:57.740 align:center
a certain final grade.

00:26:57.740 --> 00:26:59.790 align:center
So we are still, as
instructors here at UW,

00:26:59.790 --> 00:27:02.430 align:center
constrained by the 0.1
increment grading system.

00:27:02.430 --> 00:27:04.700 align:center
But we can find ways
to provide students

00:27:04.700 --> 00:27:07.652 align:center
the flexibility and
benefits of revisions,

00:27:07.652 --> 00:27:09.110 align:center
resubmitting their
work, improving,

00:27:09.110 --> 00:27:12.110 align:center
and showing that demonstrated
improvement in their grades

00:27:12.110 --> 00:27:15.890 align:center
without necessarily throwing out
the entire reality of 0.1 grade

00:27:15.890 --> 00:27:19.040 align:center
increments being the
system that we work in.

00:27:19.040 --> 00:27:20.870 align:center
So to quote the
syllabus a little bit

00:27:20.870 --> 00:27:23.308 align:center
here, "Minimum requirements
for each grade.

00:27:23.308 --> 00:27:25.850 align:center
Note that all requirements for
a particular grade must be met

00:27:25.850 --> 00:27:28.760 align:center
to guarantee that minimum,
though failing to do so does not

00:27:28.760 --> 00:27:30.330 align:center
mean that grade
cannot be earned."

00:27:30.330 --> 00:27:32.690 align:center
This is always a very
confusing sentence to students,

00:27:32.690 --> 00:27:34.330 align:center
but basically it
means that while these

00:27:34.330 --> 00:27:36.460 align:center
are minimum
guarantees, if you have

00:27:36.460 --> 00:27:38.590 align:center
some more unusual
combination, you may still

00:27:38.590 --> 00:27:41.110 align:center
get more than a 3.5, even
if you don't technically

00:27:41.110 --> 00:27:43.185 align:center
meet the minimum
requirements for it per se.

00:27:46.900 --> 00:27:48.710 align:center
So finally, to put
all this together,

00:27:48.710 --> 00:27:50.600 align:center
I've talked about a couple
of example grading systems.

00:27:50.600 --> 00:27:52.933 align:center
I want to remind us, what's
the big picture of all this?

00:27:52.933 --> 00:27:56.500 align:center
So I want to point to Robert
Talbert and David Clark's

00:27:56.500 --> 00:27:58.900 align:center
summary of the philosophy
of alternative grading here

00:27:58.900 --> 00:28:00.650 align:center
with these four pillars.

00:28:00.650 --> 00:28:02.540 align:center
They really like
this symbol here,

00:28:02.540 --> 00:28:04.537 align:center
so I'll explain this
in a little bit.

00:28:04.537 --> 00:28:06.370 align:center
These pillars really
represent the questions

00:28:06.370 --> 00:28:08.787 align:center
you can ask yourself when you
are considering your grading

00:28:08.787 --> 00:28:09.470 align:center
systems.

00:28:09.470 --> 00:28:13.280 align:center
Does your grading system have
clearly defined standards?

00:28:13.280 --> 00:28:15.710 align:center
Does your grading system
have helpful feedback?

00:28:15.710 --> 00:28:17.230 align:center
Does your grading
system have marks

00:28:17.230 --> 00:28:21.220 align:center
that indicate progress and/or
re-attempt without penalty?

00:28:21.220 --> 00:28:22.760 align:center
If you're able to
hit all of those,

00:28:22.760 --> 00:28:24.385 align:center
I think you're probably
in a good place

00:28:24.385 --> 00:28:27.190 align:center
in supporting students to
demonstrate what they learned

00:28:27.190 --> 00:28:29.490 align:center
by the end of the term and
have their grades reflect

00:28:29.490 --> 00:28:30.615 align:center
that demonstrated learning.

00:28:33.260 --> 00:28:35.430 align:center
So with clearly defined
standards, for example,

00:28:35.430 --> 00:28:37.847 align:center
student work is evaluated
using content standards that

00:28:37.847 --> 00:28:39.680 align:center
are appropriate for the
context and indicate

00:28:39.680 --> 00:28:41.372 align:center
what is acceptable
evidence of learning.

00:28:41.372 --> 00:28:43.580 align:center
And I want to focus on that
clearly defined standards

00:28:43.580 --> 00:28:45.510 align:center
in particular because,
for the next part,

00:28:45.510 --> 00:28:47.677 align:center
I actually want to talk
about some of the subtleties

00:28:47.677 --> 00:28:50.000 align:center
and nuances of how you
can get this to work well

00:28:50.000 --> 00:28:53.780 align:center
with students because I think
one of the clear questions

00:28:53.780 --> 00:28:56.510 align:center
that I oftentimes get from
new alternative graders

00:28:56.510 --> 00:28:58.612 align:center
is whether students
will like this change,

00:28:58.612 --> 00:29:00.320 align:center
whether they'll feel
comfortable with it.

00:29:00.320 --> 00:29:02.362 align:center
And I think whether they
feel comfortable with it

00:29:02.362 --> 00:29:05.960 align:center
depends on how you actually
implement the grading system.

00:29:05.960 --> 00:29:08.580 align:center
So first I want to
focus on details matter.

00:29:08.580 --> 00:29:11.420 align:center
So if we try to apply those
four questions that I just

00:29:11.420 --> 00:29:15.320 align:center
suggested to this
grading system,

00:29:15.320 --> 00:29:16.650 align:center
what would that look like?

00:29:16.650 --> 00:29:18.667 align:center
Well, I think there's a
lot of information here.

00:29:18.667 --> 00:29:20.750 align:center
So you as a student, you
have to keep track of all

00:29:20.750 --> 00:29:23.480 align:center
your E's, S's, and N's.

00:29:23.480 --> 00:29:25.998 align:center
And if you're aiming for
a certain final grade

00:29:25.998 --> 00:29:28.290 align:center
in the course, you still have
a pretty high bar to hit.

00:29:28.290 --> 00:29:31.870 align:center
You still have to demonstrate
that you have done 27,

00:29:31.870 --> 00:29:34.450 align:center
30 S's and E's.

00:29:34.450 --> 00:29:36.670 align:center
And at UW, at my
institution right now,

00:29:36.670 --> 00:29:38.688 align:center
University of Washington,
we're oftentimes

00:29:38.688 --> 00:29:40.980 align:center
facing students who want to
know with a high relatively

00:29:40.980 --> 00:29:43.320 align:center
good degree of certainty
that their grade will be this

00:29:43.320 --> 00:29:45.507 align:center
or that this is how
they're doing so far.

00:29:45.507 --> 00:29:47.590 align:center
And it can be a little
disconcerting for students,

00:29:47.590 --> 00:29:49.170 align:center
whether they feel
like they can do it

00:29:49.170 --> 00:29:51.503 align:center
or whether they can feel like
they can achieve the grade

00:29:51.503 --> 00:29:53.250 align:center
that they're looking for.

00:29:53.250 --> 00:29:57.240 align:center
So I think that there are some
risks to our approaches here.

00:29:57.240 --> 00:30:00.240 align:center
So using these four pillars, can
we evaluate our specifications

00:30:00.240 --> 00:30:02.160 align:center
grading here?

00:30:02.160 --> 00:30:04.450 align:center
So although I feel like
these requirements--

00:30:04.450 --> 00:30:06.870 align:center
and if we actually
zoom in to this rubric,

00:30:06.870 --> 00:30:07.912 align:center
it's not too complicated.

00:30:07.912 --> 00:30:09.412 align:center
But I think there's
a lot of nuance.

00:30:09.412 --> 00:30:11.700 align:center
So if you actually read
into some of our rubrics

00:30:11.700 --> 00:30:14.230 align:center
and some of the details
that we provide to students,

00:30:14.230 --> 00:30:16.630 align:center
one of the dimensions-- so
if you actually see here,

00:30:16.630 --> 00:30:19.630 align:center
I'm just pointing to
the quality dimension.

00:30:19.630 --> 00:30:21.700 align:center
Just the quality
dimension alone,

00:30:21.700 --> 00:30:24.180 align:center
there's actually a lot of
subpoints, sublearning goals

00:30:24.180 --> 00:30:25.810 align:center
that we want
students to achieve.

00:30:25.810 --> 00:30:28.560 align:center
We want their header comments
to be present on all their code

00:30:28.560 --> 00:30:29.895 align:center
classes and methods.

00:30:29.895 --> 00:30:31.270 align:center
We want all their
header comments

00:30:31.270 --> 00:30:33.700 align:center
to be concise,
meaningful, and include

00:30:33.700 --> 00:30:35.102 align:center
all relevant information.

00:30:35.102 --> 00:30:37.060 align:center
We want their names,
comments, and line lengths

00:30:37.060 --> 00:30:39.015 align:center
to make code more
readable and maintainable.

00:30:39.015 --> 00:30:40.390 align:center
And so these are
all requirements

00:30:40.390 --> 00:30:43.690 align:center
that as experts in the field
would like our students

00:30:43.690 --> 00:30:46.690 align:center
to be able to demonstrate in
terms of just code quality.

00:30:46.690 --> 00:30:49.580 align:center
But I think when it gets
so complicated like this,

00:30:49.580 --> 00:30:52.000 align:center
it can really start to
make it hard for students

00:30:52.000 --> 00:30:54.783 align:center
to see that this is a
clearly defined standard.

00:30:54.783 --> 00:30:56.950 align:center
It may be clear to experts
but possibly not as clear

00:30:56.950 --> 00:30:57.573 align:center
to students.

00:30:57.573 --> 00:30:59.740 align:center
So I think this is one
potential point of contention

00:30:59.740 --> 00:31:01.630 align:center
that students might face.

00:31:01.630 --> 00:31:03.860 align:center
And in fact, if you
actually go talk to Eman,

00:31:03.860 --> 00:31:05.277 align:center
this is actually
one of the things

00:31:05.277 --> 00:31:08.140 align:center
that she surfaced in her
interviews with students.

00:31:08.140 --> 00:31:11.093 align:center
So quoting one of the students
that Eman interviewed, "Yeah,

00:31:11.093 --> 00:31:13.010 align:center
my favorite thing was
definitely resubmissions

00:31:13.010 --> 00:31:16.270 align:center
because I could actively see the
mistake that I made or error,

00:31:16.270 --> 00:31:20.638 align:center
like what I got my points were
just reduced on and then fix it.

00:31:20.638 --> 00:31:22.180 align:center
Seeing how that
translated definitely

00:31:22.180 --> 00:31:24.320 align:center
helped me feel more
reassured and less panicked.

00:31:24.320 --> 00:31:26.440 align:center
For the final and
midterm, though,

00:31:26.440 --> 00:31:29.000 align:center
literally my hand was shaking
so bad because I was telling

00:31:29.000 --> 00:31:31.292 align:center
myself, like, my grade
literally depends on these tests

00:31:31.292 --> 00:31:33.650 align:center
because of how much I
used the resubmissions

00:31:33.650 --> 00:31:35.168 align:center
on the assignments."

00:31:35.168 --> 00:31:36.710 align:center
So there are
complications between we

00:31:36.710 --> 00:31:39.150 align:center
offer resubmissions for
some things but not others

00:31:39.150 --> 00:31:41.000 align:center
and to the point I
was just making about,

00:31:41.000 --> 00:31:42.513 align:center
clearly defined standards.

00:31:42.513 --> 00:31:44.930 align:center
Another thing that really
annoyed people is that sometimes

00:31:44.930 --> 00:31:48.110 align:center
they would let you do a regrade,
or resubmission, so in that way,

00:31:48.110 --> 00:31:49.568 align:center
it made it a little
bit more fair.

00:31:49.568 --> 00:31:51.860 align:center
But then on the other hand,
something that would happen

00:31:51.860 --> 00:31:53.585 align:center
is they mark you
off for one thing

00:31:53.585 --> 00:31:54.710 align:center
on the original submission.

00:31:54.710 --> 00:31:55.970 align:center
And then on your
resubmission, they'll

00:31:55.970 --> 00:31:57.345 align:center
fix that one thing
in your grade,

00:31:57.345 --> 00:31:59.595 align:center
but then they have to ding
you for something else they

00:31:59.595 --> 00:32:00.810 align:center
didn't even mention before.

00:32:00.810 --> 00:32:02.930 align:center
So I think that also
frustrated some people

00:32:02.930 --> 00:32:05.660 align:center
because it felt like it was
thrown at them last minute.

00:32:05.660 --> 00:32:07.890 align:center
So I think getting the
details right here,

00:32:07.890 --> 00:32:10.350 align:center
knowing how you're communicating
that feedback to students,

00:32:10.350 --> 00:32:13.470 align:center
making sure that it feels like,
from the student's perspective--

00:32:13.470 --> 00:32:15.380 align:center
and again, these are
mostly UW students who

00:32:15.380 --> 00:32:16.440 align:center
are very grade conscious.

00:32:16.440 --> 00:32:18.170 align:center
They're like, I want
to get all the points--

00:32:18.170 --> 00:32:19.587 align:center
because that's
their mentality, we

00:32:19.587 --> 00:32:21.210 align:center
want to be able to
provide support,

00:32:21.210 --> 00:32:23.480 align:center
understanding that that's
their position, how they're

00:32:23.480 --> 00:32:25.730 align:center
approaching that feedback
and using these resubmission

00:32:25.730 --> 00:32:26.230 align:center
systems.

00:32:29.220 --> 00:32:31.900 align:center
So I want to point out, though,
that's just only one option.

00:32:31.900 --> 00:32:33.820 align:center
There are many other
ways you can do things.

00:32:33.820 --> 00:32:36.210 align:center
So if you're using Canvas or
another learning management

00:32:36.210 --> 00:32:39.100 align:center
system like it, there
are other systems,

00:32:39.100 --> 00:32:40.900 align:center
you can actually use
to integrate progress

00:32:40.900 --> 00:32:42.733 align:center
because the grading
system that I showed you

00:32:42.733 --> 00:32:46.080 align:center
before with the 3.5,
the 3.0 table, that's

00:32:46.080 --> 00:32:48.840 align:center
just giving you one example
way of doing things.

00:32:48.840 --> 00:32:51.390 align:center
Another way that I tend to
use more often in my courses

00:32:51.390 --> 00:32:53.410 align:center
is using Canvas modules.

00:32:53.410 --> 00:32:56.170 align:center
And in a module, you can
actually indicate, hey,

00:32:56.170 --> 00:32:59.250 align:center
for example, assignment
that I'll call mapping,

00:32:59.250 --> 00:33:03.090 align:center
for this mapping assignment,
if it's out of 12 points,

00:33:03.090 --> 00:33:04.620 align:center
in Canvas, I can
set the requirement

00:33:04.620 --> 00:33:07.800 align:center
to be score at least
7 out of 12 points.

00:33:07.800 --> 00:33:10.602 align:center
And so students can still be
using these point systems.

00:33:10.602 --> 00:33:12.810 align:center
So maybe if you have an
existing point-graded rubric,

00:33:12.810 --> 00:33:14.950 align:center
you can still use that
points-graded rubric,

00:33:14.950 --> 00:33:16.140 align:center
but maybe indicate
that there's just

00:33:16.140 --> 00:33:17.100 align:center
some of these
general requirements

00:33:17.100 --> 00:33:19.710 align:center
that you want students to be
able to demonstrate to satisfy

00:33:19.710 --> 00:33:21.543 align:center
a module requirement.

00:33:21.543 --> 00:33:23.710 align:center
So then when you're doing
final grades, you can say,

00:33:23.710 --> 00:33:24.780 align:center
have you completed this module?

00:33:24.780 --> 00:33:26.405 align:center
Or have you completed
this requirement?

00:33:26.405 --> 00:33:29.030 align:center
Just means for the student
going back and looking at Canvas

00:33:29.030 --> 00:33:32.030 align:center
and being like, did they get a
check mark, a green check mark,

00:33:32.030 --> 00:33:33.660 align:center
for scoring at
least seven points?

00:33:33.660 --> 00:33:35.430 align:center
And if you did,
they're in good shape.

00:33:35.430 --> 00:33:38.840 align:center
If they're not, they probably
need to go back and resubmit.

00:33:38.840 --> 00:33:41.000 align:center
But I think there's also
a tension here, too,

00:33:41.000 --> 00:33:44.300 align:center
because like I mentioned
with the points emphasis,

00:33:44.300 --> 00:33:46.835 align:center
this way still really
emphasizes points.

00:33:46.835 --> 00:33:48.710 align:center
We have to make choices
about how many points

00:33:48.710 --> 00:33:50.460 align:center
to assign to an assignment.

00:33:50.460 --> 00:33:51.920 align:center
And from the
student's perspective,

00:33:51.920 --> 00:33:53.713 align:center
when they receive this
Canvas notification,

00:33:53.713 --> 00:33:55.880 align:center
they just look at the points,
and they're like, hmm,

00:33:55.880 --> 00:33:58.648 align:center
does that feel like a good point
value or a bad point value?

00:33:58.648 --> 00:34:00.440 align:center
And that's really, I
think, a judgment call

00:34:00.440 --> 00:34:03.330 align:center
that oftentimes students
are using to understand,

00:34:03.330 --> 00:34:05.582 align:center
am I good at this subject?

00:34:05.582 --> 00:34:07.040 align:center
Even though I think
as teachers, we

00:34:07.040 --> 00:34:09.498 align:center
would encourage a lot of our
students, all of our students,

00:34:09.498 --> 00:34:11.480 align:center
I hope, really, to be
pursuing that subject

00:34:11.480 --> 00:34:13.320 align:center
and learning and
improving over time

00:34:13.320 --> 00:34:15.739 align:center
and that low points or
low scoring assignments

00:34:15.739 --> 00:34:19.500 align:center
aren't indicators that they
aren't cut out for this subject.

00:34:19.500 --> 00:34:22.070 align:center
But oftentimes they will
take these point values

00:34:22.070 --> 00:34:24.949 align:center
as indications of whether
they're adept for it

00:34:24.949 --> 00:34:26.949 align:center
or whether they are fit
for this subject or not.

00:34:26.949 --> 00:34:30.630 align:center
So I think that this is
a possible middle ground

00:34:30.630 --> 00:34:32.429 align:center
approach but one that
I think you still

00:34:32.429 --> 00:34:36.870 align:center
have to think through
the subtleties of it.

00:34:36.870 --> 00:34:39.210 align:center
Now, I also get a
common questions about,

00:34:39.210 --> 00:34:43.260 align:center
how do you actually implement
this in Canvas in other ways?

00:34:43.260 --> 00:34:45.642 align:center
In Canvas, if you would
like to de-emphasize points,

00:34:45.642 --> 00:34:47.100 align:center
there are some
strategies that I've

00:34:47.100 --> 00:34:50.070 align:center
learned from Abigail Noyce
and David Largent here.

00:34:50.070 --> 00:34:52.770 align:center
So what Abigail and Dave suggest
is that you could actually

00:34:52.770 --> 00:34:54.870 align:center
make most of your
assignments one

00:34:54.870 --> 00:34:57.090 align:center
point so that either
students complete it

00:34:57.090 --> 00:35:00.960 align:center
or didn't complete it, 1 point
for complete, 0 for incomplete.

00:35:00.960 --> 00:35:03.370 align:center
And then you can go through
your Canvas Settings,

00:35:03.370 --> 00:35:06.510 align:center
go into More Options, and you
can do Hide Totals and Student

00:35:06.510 --> 00:35:07.550 align:center
Grade Summary.

00:35:07.550 --> 00:35:09.300 align:center
And like I mentioned,
for each assignment,

00:35:09.300 --> 00:35:12.510 align:center
make it 1 point and just
satisfactory or revise.

00:35:12.510 --> 00:35:15.010 align:center
And that's another way that you
can communicate to students,

00:35:15.010 --> 00:35:16.712 align:center
hey, the point
values, I'm not really

00:35:16.712 --> 00:35:18.420 align:center
trying to use them in
a way that actually

00:35:18.420 --> 00:35:22.282 align:center
indicates how good you are from
a 0 to 12 or 0 to 100 scale.

00:35:22.282 --> 00:35:23.740 align:center
Instead, it's just
here to tell you

00:35:23.740 --> 00:35:26.690 align:center
whether it looks good, thumbs
up, or if you need to revise.

00:35:26.690 --> 00:35:28.690 align:center
And that, I think, is
really deliberate in terms

00:35:28.690 --> 00:35:31.630 align:center
of how they're thinking about
the way we communicate student

00:35:31.630 --> 00:35:34.430 align:center
progress to students because
here it's not, you failed.

00:35:34.430 --> 00:35:36.410 align:center
It's not, you're in trouble.

00:35:36.410 --> 00:35:39.560 align:center
It's just saying, hey, we're
expecting revision from you.

00:35:39.560 --> 00:35:41.477 align:center
We are expecting
better work from you.

00:35:41.477 --> 00:35:43.810 align:center
And we think that you can
demonstrate that to us, right?

00:35:43.810 --> 00:35:45.580 align:center
That it's an issue
of time rather than

00:35:45.580 --> 00:35:50.650 align:center
an issue of the student's
capability or their ability.

00:35:50.650 --> 00:35:53.032 align:center
So then once you have that,
the practical side is, well,

00:35:53.032 --> 00:35:54.740 align:center
how do you use it to
track your progress?

00:35:54.740 --> 00:35:57.470 align:center
Well, now, because they're
all out of 0 or 1 points,

00:35:57.470 --> 00:35:59.920 align:center
the summaries from
the instructor side

00:35:59.920 --> 00:36:01.930 align:center
represent how many
of those assignments

00:36:01.930 --> 00:36:02.990 align:center
students have completed.

00:36:02.990 --> 00:36:05.350 align:center
So the student has
done 4 out of 5, 4 out

00:36:05.350 --> 00:36:07.100 align:center
of 5 points on the quizzes.

00:36:07.100 --> 00:36:10.450 align:center
And so they've gotten 80% of the
quizzes done to the satisfaction

00:36:10.450 --> 00:36:11.682 align:center
of the graders.

00:36:11.682 --> 00:36:13.390 align:center
So you can still use
all these techniques

00:36:13.390 --> 00:36:17.110 align:center
to be able to use
existing learning mastery

00:36:17.110 --> 00:36:21.093 align:center
learning management systems to
make it so that you can track

00:36:21.093 --> 00:36:23.010 align:center
student progress, and
you can help communicate

00:36:23.010 --> 00:36:26.520 align:center
those expectations to students
so even they know what to do.

00:36:26.520 --> 00:36:30.233 align:center
And you can use things like
having course assignment groups.

00:36:30.233 --> 00:36:31.650 align:center
You can even say
like, hey, I want

00:36:31.650 --> 00:36:34.350 align:center
to say drop one assignment from
this group using the three dot

00:36:34.350 --> 00:36:34.990 align:center
menu.

00:36:34.990 --> 00:36:37.630 align:center
So lots of ways where you can
provide flexibility to students,

00:36:37.630 --> 00:36:40.710 align:center
even not necessarily
using full revisions

00:36:40.710 --> 00:36:44.490 align:center
or even using the ESN system but
even using points-based systems

00:36:44.490 --> 00:36:46.470 align:center
or even using your
learning management system,

00:36:46.470 --> 00:36:47.345 align:center
like I've shown here.

00:36:50.532 --> 00:36:51.990 align:center
So the final thing
I'll end us with

00:36:51.990 --> 00:36:55.050 align:center
is of bringing home
this idea of that it's

00:36:55.050 --> 00:36:58.020 align:center
important to talk about not
just how we grade but also

00:36:58.020 --> 00:36:59.170 align:center
why we grade.

00:36:59.170 --> 00:37:02.190 align:center
And I think that goes to this
last section I've talked about

00:37:02.190 --> 00:37:05.310 align:center
with subtleties here, that
it's important to communicate

00:37:05.310 --> 00:37:07.720 align:center
to students explicitly,
affirmatively,

00:37:07.720 --> 00:37:10.380 align:center
and repeatedly why we're making
these changes, that we aren't

00:37:10.380 --> 00:37:11.880 align:center
just changing these
grading policies

00:37:11.880 --> 00:37:14.255 align:center
because someone is telling
us or requiring us to do it.

00:37:14.255 --> 00:37:15.630 align:center
I don't think
anyone is requiring

00:37:15.630 --> 00:37:17.080 align:center
us to change our
grading policies,

00:37:17.080 --> 00:37:18.150 align:center
but I think you're
here because you're

00:37:18.150 --> 00:37:20.567 align:center
curious to learn what it could
mean to change your grading

00:37:20.567 --> 00:37:23.060 align:center
policies to better
relationships with students.

00:37:23.060 --> 00:37:25.390 align:center
And I think starting from
that perspective, whatever

00:37:25.390 --> 00:37:27.730 align:center
it means for you, whatever
drives you to improve

00:37:27.730 --> 00:37:30.520 align:center
your practice, and being honest
and clear and communicative

00:37:30.520 --> 00:37:33.190 align:center
about that with students is the
first step toward making that

00:37:33.190 --> 00:37:35.740 align:center
better, to, really, I think
having more honest conversations

00:37:35.740 --> 00:37:37.428 align:center
with students in
general, that we're

00:37:37.428 --> 00:37:39.220 align:center
changing our grading
policies because we're

00:37:39.220 --> 00:37:42.020 align:center
working toward a better student
experience in the future.

00:37:42.020 --> 00:37:43.437 align:center
And I think you
can even be honest

00:37:43.437 --> 00:37:45.853 align:center
that it's an experiment, that
you're trying something out,

00:37:45.853 --> 00:37:48.250 align:center
and that we can have a dialogue
about how it goes halfway

00:37:48.250 --> 00:37:50.360 align:center
through the quarter and
see if we're happy with it

00:37:50.360 --> 00:37:52.485 align:center
or if we're not happy with
it and what parts we can

00:37:52.485 --> 00:37:53.750 align:center
try to change to improve it.

00:37:53.750 --> 00:37:55.330 align:center
I think having that
openness, having

00:37:55.330 --> 00:37:57.540 align:center
that opportunity and
dialogue with students

00:37:57.540 --> 00:37:59.290 align:center
can really put you in
a place where you're

00:37:59.290 --> 00:38:01.665 align:center
able to listen to feedback,
where students are giving you

00:38:01.665 --> 00:38:04.430 align:center
more grace, and you're also
giving yourself some grace

00:38:04.430 --> 00:38:06.350 align:center
to be open to some
potential for failure

00:38:06.350 --> 00:38:08.080 align:center
because I think a lot
of this oftentimes

00:38:08.080 --> 00:38:10.330 align:center
requires learning more
about your students, too,

00:38:10.330 --> 00:38:12.070 align:center
learning more about
their interests,

00:38:12.070 --> 00:38:13.570 align:center
why they're taking
this course, what

00:38:13.570 --> 00:38:17.440 align:center
drives them to earn high
grades or what drives them

00:38:17.440 --> 00:38:19.540 align:center
toward the career path
they're looking for

00:38:19.540 --> 00:38:22.610 align:center
and what they need
to achieve that.

00:38:22.610 --> 00:38:25.310 align:center
So if we consider really this
kind of alternative grading

00:38:25.310 --> 00:38:28.460 align:center
to be the process of questioning
our grading practices,

00:38:28.460 --> 00:38:30.593 align:center
I think that something
I think about a lot

00:38:30.593 --> 00:38:32.510 align:center
is that I think alternative
grading has helped

00:38:32.510 --> 00:38:34.760 align:center
me to move to a place
where I can probably

00:38:34.760 --> 00:38:38.240 align:center
say to students that,
yes, you can work on this

00:38:38.240 --> 00:38:41.330 align:center
and demonstrate your learning
later and get full credit

00:38:41.330 --> 00:38:42.622 align:center
for that demonstrated learning.

00:38:42.622 --> 00:38:44.622 align:center
And that's something that
I felt like I wouldn't

00:38:44.622 --> 00:38:46.855 align:center
be able to say in my old
version of me before this,

00:38:46.855 --> 00:38:49.230 align:center
where I felt like, yeah, well,
this is the midterm score,

00:38:49.230 --> 00:38:50.563 align:center
or this is the assignment score.

00:38:50.563 --> 00:38:52.115 align:center
And I can't extend it for you.

00:38:52.115 --> 00:38:53.990 align:center
I can't give you a second
chance because that

00:38:53.990 --> 00:38:56.150 align:center
would require giving
everyone a second chance.

00:38:56.150 --> 00:38:58.220 align:center
But I think if we
think about, What

00:38:58.220 --> 00:39:00.350 align:center
are our constraints actually?

00:39:00.350 --> 00:39:02.670 align:center
And can we actually
change those constraints?

00:39:02.670 --> 00:39:04.160 align:center
Can we actually
change those limits

00:39:04.160 --> 00:39:05.813 align:center
that we've put on ourselves?

00:39:05.813 --> 00:39:07.730 align:center
I think that can really
give us a lot of space

00:39:07.730 --> 00:39:10.150 align:center
to give students and make
for better relationships

00:39:10.150 --> 00:39:11.275 align:center
with students through this.

00:39:14.750 --> 00:39:16.400 align:center
Yeah, and so all
this, I think, has

00:39:16.400 --> 00:39:19.120 align:center
helped me learn a lot
about student work, student

00:39:19.120 --> 00:39:22.660 align:center
motivations, students'
learning goals really,

00:39:22.660 --> 00:39:25.210 align:center
and help put me in, I
think, a better relationship

00:39:25.210 --> 00:39:26.710 align:center
with my students.

00:39:26.710 --> 00:39:29.870 align:center
So with all that
said, I'll end here.

00:39:29.870 --> 00:39:31.960 align:center
So my call to
action is, really, I

00:39:31.960 --> 00:39:33.790 align:center
think I want us to
redesign our grading

00:39:33.790 --> 00:39:36.230 align:center
to be to better our
relationship with students.

00:39:36.230 --> 00:39:39.130 align:center
And I'll leave us with
those three points

00:39:39.130 --> 00:39:43.390 align:center
here that the three main ideas
that students should have

00:39:43.390 --> 00:39:45.590 align:center
multiple opportunities to
demonstrate proficiency,

00:39:45.590 --> 00:39:47.882 align:center
that grade should reflect
the proficiency that students

00:39:47.882 --> 00:39:50.560 align:center
demonstrate by the end, and
that these grades and points are

00:39:50.560 --> 00:39:52.787 align:center
de-emphasized in favor
of rich, deep feedback

00:39:52.787 --> 00:39:54.370 align:center
so that we can really,
I think, better

00:39:54.370 --> 00:39:55.730 align:center
our relationship with students.

00:39:55.730 --> 00:39:56.930 align:center
So thank you all so much.

00:39:56.930 --> 00:40:00.910 align:center
And I'll also give some thanks
to the four other people that I

00:40:00.910 --> 00:40:03.400 align:center
helped consult, who are all
PhD students at the University

00:40:03.400 --> 00:40:07.010 align:center
of Washington, Eman ,
Mina, Carly, and Rachel,

00:40:07.010 --> 00:40:11.710 align:center
who helped in some ways
in developing this talk.

00:40:11.710 --> 00:40:13.760 align:center
And so it couldn't have
been done without them.

00:40:13.760 --> 00:40:17.140 align:center
So thank you all so much, and
I'll leave it open for questions

00:40:17.140 --> 00:40:18.040 align:center
now.

00:40:18.040 --> 00:40:20.040 align:center
And this is a very
long talk with me,

00:40:20.040 --> 00:40:23.708 align:center
so I appreciate your
attention with this.

00:40:23.708 --> 00:40:26.250 align:center
And I'll just remind us that we
do have the link in the chat.

00:40:26.250 --> 00:40:28.292 align:center
So you can go and review
my slides there as well.

00:40:33.410 --> 00:40:34.700 align:center
ERIC TREKELL: All right.

00:40:34.700 --> 00:40:36.560 align:center
Thank you, Kevin.

00:40:36.560 --> 00:40:39.080 align:center
The floor is open
for any questions

00:40:39.080 --> 00:40:41.420 align:center
you may have for Kevin.

00:40:41.420 --> 00:40:43.640 align:center
Feel free to either
raise your hand

00:40:43.640 --> 00:40:46.073 align:center
or post a message in the chat.

00:40:59.480 --> 00:41:01.520 align:center
I see one hand raised, Kevin.

00:41:01.520 --> 00:41:05.420 align:center
Victoria, would you like to
unmute and ask your question?

00:41:05.420 --> 00:41:06.560 align:center
VICTORIA: Sure.

00:41:06.560 --> 00:41:07.590 align:center
It wasn't a question.

00:41:07.590 --> 00:41:11.240 align:center
I was going to say, I
just started last semester

00:41:11.240 --> 00:41:14.670 align:center
or last two semesters
on the final exam.

00:41:14.670 --> 00:41:16.970 align:center
Like the students, if
they missed a question

00:41:16.970 --> 00:41:19.530 align:center
on the midterm, they're
able to get the points back

00:41:19.530 --> 00:41:23.790 align:center
on it if they got a similar
question correct or same concept

00:41:23.790 --> 00:41:25.270 align:center
correct on the final.

00:41:25.270 --> 00:41:27.570 align:center
And the students
had to tell me that

00:41:27.570 --> 00:41:31.260 align:center
what questions they wanted me to
look at to get the points back.

00:41:31.260 --> 00:41:34.470 align:center
It was very successful.

00:41:34.470 --> 00:41:37.140 align:center
And the students really
appreciated the fact

00:41:37.140 --> 00:41:42.630 align:center
that the midterm could
almost be completely erased

00:41:42.630 --> 00:41:45.480 align:center
if they really blew it
because, as long as they--

00:41:45.480 --> 00:41:47.670 align:center
I said, I want you to
learn by the end of the--

00:41:47.670 --> 00:41:50.400 align:center
by the end of the semester, you
got to prove you learned it.

00:41:50.400 --> 00:41:52.276 align:center
And it's been very successful.

00:41:55.013 --> 00:41:55.680 align:center
KEVIN LIN: Yeah.

00:41:55.680 --> 00:41:56.680 align:center
It's great to hear that.

00:41:56.680 --> 00:41:58.710 align:center
Thank you for sharing
the story, Victoria.

00:41:58.710 --> 00:41:59.320 align:center
And I agree.

00:41:59.320 --> 00:42:01.445 align:center
I think that there are all
these small things where

00:42:01.445 --> 00:42:03.700 align:center
it's like, huh, I hadn't
thought of that idea before.

00:42:03.700 --> 00:42:05.488 align:center
But it sounds like really good.

00:42:05.488 --> 00:42:07.780 align:center
And we could try it out, see
how it goes for this term.

00:42:07.780 --> 00:42:11.318 align:center
And I think being flexible and
open to changing in futures

00:42:11.318 --> 00:42:13.860 align:center
and knowing that, hey, I think
giving yourself a little grace

00:42:13.860 --> 00:42:15.670 align:center
for it too, especially when
you're trying new things,

00:42:15.670 --> 00:42:17.580 align:center
be like, yeah, I want
to give this a try.

00:42:17.580 --> 00:42:19.372 align:center
I want to actually put
some thought into it

00:42:19.372 --> 00:42:20.942 align:center
and think about
what fits me best.

00:42:20.942 --> 00:42:22.650 align:center
I've actually shown
you multiple systems.

00:42:22.650 --> 00:42:24.807 align:center
I've shown you possibly
too many possibilities.

00:42:24.807 --> 00:42:27.140 align:center
But I hope you find one that
kind of rings well with you

00:42:27.140 --> 00:42:28.500 align:center
or rings well with
your subject matter.

00:42:28.500 --> 00:42:30.583 align:center
And I think a lot of it
can be more subject matter

00:42:30.583 --> 00:42:31.970 align:center
specific as well.

00:42:31.970 --> 00:42:34.740 align:center
Some systems where I showed
you specifications grading,

00:42:34.740 --> 00:42:36.870 align:center
it tends to be that, I
have these big assignments.

00:42:36.870 --> 00:42:38.328 align:center
I have these big
weekly assignments

00:42:38.328 --> 00:42:41.070 align:center
that students are spending
eight or so hours working on.

00:42:41.070 --> 00:42:43.580 align:center
And so it makes sense to have
a big specification for it

00:42:43.580 --> 00:42:45.570 align:center
and just grade according
to that specification.

00:42:45.570 --> 00:42:48.270 align:center
But if you have small multiple
choice quizzes on Canvas,

00:42:48.270 --> 00:42:51.210 align:center
maybe that's more of a
standards-based system.

00:42:51.210 --> 00:42:53.640 align:center
Or maybe it's like the final
exam clobbering, Victoria,

00:42:53.640 --> 00:42:55.268 align:center
you were suggesting.

00:42:55.268 --> 00:42:57.810 align:center
And then, Eric, do you want to
moderate from here on out, or?

00:42:57.810 --> 00:42:59.102 align:center
ERIC TREKELL: Yeah, absolutely.

00:42:59.102 --> 00:43:01.430 align:center
We have two questions in
the chat that came up first

00:43:01.430 --> 00:43:03.440 align:center
and then a couple
of hands raised.

00:43:03.440 --> 00:43:07.760 align:center
Leslie is asking about
the final exam clobbering.

00:43:07.760 --> 00:43:09.440 align:center
Do you replace the
full midterm grade

00:43:09.440 --> 00:43:13.455 align:center
with what they achieved on
the final exam if it's better?

00:43:13.455 --> 00:43:15.330 align:center
KEVIN LIN: Yeah, thank
you for that question.

00:43:15.330 --> 00:43:18.090 align:center
And I think it was Leslie
who asked that in the chat.

00:43:18.090 --> 00:43:19.300 align:center
It's really up to you.

00:43:19.300 --> 00:43:20.520 align:center
It could be multiple systems.

00:43:20.520 --> 00:43:23.473 align:center
Like Victoria suggested, it's
question-by-question basis.

00:43:23.473 --> 00:43:25.890 align:center
So maybe on the final exam you
have a very similar version

00:43:25.890 --> 00:43:26.890 align:center
of the midterm question.

00:43:26.890 --> 00:43:28.840 align:center
You're like, OK, I
can map that back.

00:43:28.840 --> 00:43:31.290 align:center
And to even be a little
bit more forward looking

00:43:31.290 --> 00:43:33.265 align:center
than the idea of
Victoria suggested,

00:43:33.265 --> 00:43:35.640 align:center
I think if you can do that
mapping for all the questions,

00:43:35.640 --> 00:43:37.348 align:center
you don't even have
to have students ask.

00:43:37.348 --> 00:43:39.682 align:center
You can just automatically
apply it in your spreadsheet.

00:43:39.682 --> 00:43:41.670 align:center
You can say, if the
number on the final exam

00:43:41.670 --> 00:43:44.560 align:center
question and the corresponding
midterm question,

00:43:44.560 --> 00:43:46.540 align:center
if that number on the
final exam 1 is bigger,

00:43:46.540 --> 00:43:48.010 align:center
then you can just
replace the midterm 1.

00:43:48.010 --> 00:43:49.885 align:center
So you can probably do
it in an automated way

00:43:49.885 --> 00:43:52.770 align:center
if you found the connections
between the different problems.

00:43:52.770 --> 00:43:53.770 align:center
That's really up to you.

00:43:53.770 --> 00:43:56.190 align:center
That's up to you
to do that work.

00:43:56.190 --> 00:43:59.220 align:center
I've also been using final exam
clobbering recently as well

00:43:59.220 --> 00:44:00.700 align:center
in addition to resubmissions.

00:44:00.700 --> 00:44:03.550 align:center
So for each of my exams,
I let students revise.

00:44:03.550 --> 00:44:05.490 align:center
And then even then,
even if they feel

00:44:05.490 --> 00:44:08.550 align:center
like they weren't able to revise
or if they missed the window

00:44:08.550 --> 00:44:11.770 align:center
for revising the exam
during the quarter,

00:44:11.770 --> 00:44:14.402 align:center
I also have the final
exam to clobber, too.

00:44:14.402 --> 00:44:15.860 align:center
For me, just to
keep things simple,

00:44:15.860 --> 00:44:20.380 align:center
I just have it be the parts
of the final just completely

00:44:20.380 --> 00:44:24.253 align:center
override parts of the
midterm if it improves.

00:44:24.253 --> 00:44:26.420 align:center
So I'm not thinking on a
question-by-question basis.

00:44:26.420 --> 00:44:28.220 align:center
But there are different ways
that you can go about it.

00:44:28.220 --> 00:44:29.560 align:center
It depends on how
much time you have

00:44:29.560 --> 00:44:31.120 align:center
to think through
it, what you think

00:44:31.120 --> 00:44:33.805 align:center
is the best way to organize
things, and the practical side

00:44:33.805 --> 00:44:34.305 align:center
of it, too.

00:44:36.750 --> 00:44:38.500 align:center
ERIC TREKELL: So we
have one more question

00:44:38.500 --> 00:44:40.000 align:center
in the chat we'll
do, and then we'll

00:44:40.000 --> 00:44:43.480 align:center
move on to the raised hands.

00:44:43.480 --> 00:44:47.230 align:center
Kevin, Dana asks, I was
interested in the model

00:44:47.230 --> 00:44:49.910 align:center
you presented from Evergreen,
in addition to the other ideas.

00:44:49.910 --> 00:44:52.300 align:center
Could you speak to
student self-evaluation

00:44:52.300 --> 00:44:54.530 align:center
and how it is related
to the final grade?

00:44:54.530 --> 00:44:57.790 align:center
I'm assuming that's
at Evergreen.

00:44:57.790 --> 00:45:01.480 align:center
KEVIN LIN: Yeah, so thank
you for that question.

00:45:01.480 --> 00:45:03.910 align:center
I'm not personally much
more familiar with Evergreen

00:45:03.910 --> 00:45:05.812 align:center
beyond the quotes that
I suggested there.

00:45:05.812 --> 00:45:08.020 align:center
The quotes that I pulled
and the information I pulled

00:45:08.020 --> 00:45:10.223 align:center
is actually public on the
Evergreen State College.

00:45:10.223 --> 00:45:11.890 align:center
Oftentimes, I imagine
they get questions

00:45:11.890 --> 00:45:15.018 align:center
from prospective students about,
this is a weird grading system.

00:45:15.018 --> 00:45:16.060 align:center
Can you tell me about it?

00:45:16.060 --> 00:45:18.352 align:center
And I think there's a lot of
potential for that grading

00:45:18.352 --> 00:45:19.080 align:center
system.

00:45:19.080 --> 00:45:21.600 align:center
The grading system itself,
it has three parts or the end

00:45:21.600 --> 00:45:22.780 align:center
of the term evaluations.

00:45:22.780 --> 00:45:25.300 align:center
It's one is a faculty
evaluation of the student.

00:45:25.300 --> 00:45:27.660 align:center
One is a student evaluation
of themselves, so

00:45:27.660 --> 00:45:29.155 align:center
a student self-evaluation.

00:45:29.155 --> 00:45:31.530 align:center
And then there's one of a
student evaluation of teachers,

00:45:31.530 --> 00:45:33.947 align:center
which is interesting because
it's like I think as teachers

00:45:33.947 --> 00:45:36.360 align:center
we're used to the student
evaluations of teaching

00:45:36.360 --> 00:45:37.270 align:center
from the students.

00:45:37.270 --> 00:45:40.060 align:center
But now it's like a
two-way communication.

00:45:40.060 --> 00:45:41.680 align:center
It's not just students
evaluating us.

00:45:41.680 --> 00:45:44.620 align:center
But we're evaluating the
students in a written way,

00:45:44.620 --> 00:45:47.310 align:center
and students are
evaluating themselves.

00:45:47.310 --> 00:45:49.780 align:center
And in the transcript
that students,

00:45:49.780 --> 00:45:51.550 align:center
if they want to send
out somewhere else,

00:45:51.550 --> 00:45:53.675 align:center
it just includes, I think,
the written evaluations.

00:45:53.675 --> 00:45:55.743 align:center
It's just a big packet
of all the evaluations

00:45:55.743 --> 00:45:57.660 align:center
that students have
accumulated over their time

00:45:57.660 --> 00:46:00.228 align:center
as a student, which
I think is amazing.

00:46:00.228 --> 00:46:01.770 align:center
Obviously, it's more
work for someone

00:46:01.770 --> 00:46:03.040 align:center
who's evaluating that packet.

00:46:03.040 --> 00:46:05.970 align:center
But I think it's so much richer
if you're thinking about, oh, I

00:46:05.970 --> 00:46:08.532 align:center
guess like undergraduate and
pass would be graduate school.

00:46:08.532 --> 00:46:09.990 align:center
Maybe those
descriptive evaluations

00:46:09.990 --> 00:46:12.405 align:center
are way more useful
than a number could be.

00:46:15.280 --> 00:46:19.505 align:center
ERIC TREKELL: Matthew, would
you like to ask your question?

00:46:19.505 --> 00:46:20.380 align:center
MATTHEW WILLIAMS: Hi.

00:46:20.380 --> 00:46:21.890 align:center
Again, my name is
Matthew Williams.

00:46:21.890 --> 00:46:24.680 align:center
I teach business law
at Olympic College.

00:46:24.680 --> 00:46:27.820 align:center
One of the questions I wanted
to figure out and piggyback off

00:46:27.820 --> 00:46:30.940 align:center
everybody or figure
out, pick your mindsets

00:46:30.940 --> 00:46:34.570 align:center
is to, when you have that one
individual, that one student,

00:46:34.570 --> 00:46:38.170 align:center
who hasn't completed
half of their assignments

00:46:38.170 --> 00:46:41.600 align:center
within the course and
getting to the midterm,

00:46:41.600 --> 00:46:45.860 align:center
they also did terribly-- so
it gets to the point of, hey,

00:46:45.860 --> 00:46:48.790 align:center
now I have to have this
stern conversation with you,

00:46:48.790 --> 00:46:53.180 align:center
But I don't want to say, hey,
don't continue your education.

00:46:53.180 --> 00:46:55.270 align:center
But like, we need
to pivot as to,

00:46:55.270 --> 00:46:57.790 align:center
how are you going to
get back on track?

00:46:57.790 --> 00:47:02.300 align:center
What would be your methodology
of trying to resolve of, hey,

00:47:02.300 --> 00:47:04.610 align:center
I don't want to advise
you to drop my class,

00:47:04.610 --> 00:47:06.988 align:center
but also we got to
get this together

00:47:06.988 --> 00:47:08.530 align:center
because you're going
to end up taking

00:47:08.530 --> 00:47:10.963 align:center
this class all over again?

00:47:10.963 --> 00:47:11.630 align:center
KEVIN LIN: Yeah.

00:47:11.630 --> 00:47:12.530 align:center
Thank you for that--

00:47:12.530 --> 00:47:14.870 align:center
MATTHEW WILLIAMS: And for my
particular course-- sorry.

00:47:14.870 --> 00:47:19.280 align:center
For my particular course,
majority of my coursework

00:47:19.280 --> 00:47:23.870 align:center
that I'm assigning is quizzes.

00:47:23.870 --> 00:47:27.840 align:center
So I have quizzes,
midterm, and a final.

00:47:27.840 --> 00:47:29.930 align:center
So how would you do that?

00:47:29.930 --> 00:47:33.650 align:center
Or how would you triage that?

00:47:33.650 --> 00:47:34.760 align:center
KEVIN LIN: Yeah.

00:47:34.760 --> 00:47:36.510 align:center
I think this would be
a good opportunity--

00:47:36.510 --> 00:47:38.468 align:center
thank you, Matthew, for
that detailed question.

00:47:38.468 --> 00:47:40.550 align:center
So I think the first
part was about thinking

00:47:40.550 --> 00:47:45.770 align:center
about how to design limits but
also how to support students

00:47:45.770 --> 00:47:49.080 align:center
and think about students who are
struggling, processes for that,

00:47:49.080 --> 00:47:50.672 align:center
thinking about systems for that.

00:47:50.672 --> 00:47:53.130 align:center
And then I'll need you to remind
me of your second question

00:47:53.130 --> 00:47:54.740 align:center
because I was spending so much
time thinking about that one

00:47:54.740 --> 00:47:55.280 align:center
right now.

00:47:55.280 --> 00:47:55.436 align:center
MATTHEW WILLIAMS: You're good.

00:47:55.436 --> 00:47:55.850 align:center
Go ahead.

00:47:55.850 --> 00:47:56.940 align:center
Go ahead with that question.

00:47:56.940 --> 00:47:57.590 align:center
KEVIN LIN: Yes.

00:47:57.590 --> 00:47:59.840 align:center
Let me first post in
the chat a quick link

00:47:59.840 --> 00:48:03.930 align:center
that can be useful as I speak
about limits a little bit.

00:48:03.930 --> 00:48:05.835 align:center
So there is this
Grading for Growth,

00:48:05.835 --> 00:48:07.460 align:center
which is written by
Robert Talbert, who

00:48:07.460 --> 00:48:09.240 align:center
is one of the
quotes in this talk,

00:48:09.240 --> 00:48:12.020 align:center
and David Clark are the
main editors for that blog.

00:48:12.020 --> 00:48:13.630 align:center
They take invited
posts, but this

00:48:13.630 --> 00:48:16.270 align:center
is one I think is written by
David that talks about not

00:48:16.270 --> 00:48:18.070 align:center
all limits are the same.

00:48:18.070 --> 00:48:21.340 align:center
And so we had some interesting
conversations, me, David,

00:48:21.340 --> 00:48:23.650 align:center
and some other people,
about that blog post

00:48:23.650 --> 00:48:26.900 align:center
and how to think about limits
and why we provide them.

00:48:26.900 --> 00:48:29.590 align:center
And you can see what we arrived
at for our intro programming

00:48:29.590 --> 00:48:32.510 align:center
courses here at UW
for computer science.

00:48:32.510 --> 00:48:34.700 align:center
We have these limits that
are, really, in part,

00:48:34.700 --> 00:48:36.700 align:center
a good part of it is
really the practical limits

00:48:36.700 --> 00:48:38.300 align:center
that TAs have time for.

00:48:38.300 --> 00:48:39.170 align:center
And you're right.

00:48:39.170 --> 00:48:40.712 align:center
I think, Matthew,
you're pointing out

00:48:40.712 --> 00:48:43.750 align:center
that if a student get started
so late in the quarter

00:48:43.750 --> 00:48:45.950 align:center
or wants to recover midway
through the quarter,

00:48:45.950 --> 00:48:48.970 align:center
it can be hard to get a passing
grade even because these limits

00:48:48.970 --> 00:48:49.970 align:center
are designed this way.

00:48:49.970 --> 00:48:52.930 align:center
And it'd be even worse for
them in typical graded course

00:48:52.930 --> 00:48:55.780 align:center
because they'd have 0s that
they could not at all improve.

00:48:55.780 --> 00:48:57.950 align:center
But here it can still be
quite difficult, I think.

00:48:57.950 --> 00:49:00.642 align:center
And depending on how you design
those limits, it can be tricky.

00:49:00.642 --> 00:49:02.350 align:center
I would say I think,
when you're thinking

00:49:02.350 --> 00:49:04.250 align:center
through those limits,
whatever you come up with,

00:49:04.250 --> 00:49:05.708 align:center
I think one of the
important things

00:49:05.708 --> 00:49:09.390 align:center
would be to have conversations
with students as early as you

00:49:09.390 --> 00:49:09.890 align:center
can.

00:49:09.890 --> 00:49:13.040 align:center
That I found to be one of
the most important things.

00:49:13.040 --> 00:49:15.530 align:center
I'll give you an anecdote
that's anecdata because there

00:49:15.530 --> 00:49:16.970 align:center
is a data side to it.

00:49:16.970 --> 00:49:19.110 align:center
When I was teaching some
large courses at Berkeley,

00:49:19.110 --> 00:49:20.568 align:center
one of the things
that we found was

00:49:20.568 --> 00:49:22.880 align:center
that there was a
surprisingly high correlation

00:49:22.880 --> 00:49:25.610 align:center
between the time in
which students turned

00:49:25.610 --> 00:49:28.010 align:center
in their first homework
assignment and their final grade

00:49:28.010 --> 00:49:28.920 align:center
in the course.

00:49:28.920 --> 00:49:31.340 align:center
So students who are just
working on the course

00:49:31.340 --> 00:49:33.230 align:center
and prioritizing the
course more or being

00:49:33.230 --> 00:49:35.970 align:center
able to turn in their work
earlier on the first assignment,

00:49:35.970 --> 00:49:37.910 align:center
even the first
week of the course,

00:49:37.910 --> 00:49:39.710 align:center
you can literally have
a pretty good signal

00:49:39.710 --> 00:49:42.210 align:center
about how much time students
are able to put into the class,

00:49:42.210 --> 00:49:45.308 align:center
about how much prior knowledge
or experience they have,

00:49:45.308 --> 00:49:47.600 align:center
things like that, even just
from things like submission

00:49:47.600 --> 00:49:49.140 align:center
time for that first assignment.

00:49:49.140 --> 00:49:51.920 align:center
So you can use things like that
to help with narrowing down

00:49:51.920 --> 00:49:53.520 align:center
like, OK, these are the people
I should keep an eye on.

00:49:53.520 --> 00:49:56.020 align:center
And then week 2 or week 3, as
we're grading the assignments,

00:49:56.020 --> 00:49:58.312 align:center
we can say, oh yeah, we should
reach out to this person

00:49:58.312 --> 00:50:00.710 align:center
and be like, hey, you've been
doing great so far on this,

00:50:00.710 --> 00:50:02.377 align:center
or this area could
use some improvement.

00:50:02.377 --> 00:50:04.377 align:center
Let's get you back on
track and make sure you're

00:50:04.377 --> 00:50:05.730 align:center
set up for success on that.

00:50:05.730 --> 00:50:08.580 align:center
I think the biggest thing is
reaching out early and using

00:50:08.580 --> 00:50:10.110 align:center
different signals
to do that, which

00:50:10.110 --> 00:50:13.830 align:center
can be even submission
time beyond the grade

00:50:13.830 --> 00:50:15.158 align:center
for the assignment per se.

00:50:15.158 --> 00:50:16.200 align:center
MATTHEW WILLIAMS: Gotcha.

00:50:16.200 --> 00:50:18.060 align:center
KEVIN LIN: Can you remind
me of your second question?

00:50:18.060 --> 00:50:20.060 align:center
MATTHEW WILLIAMS: Yeah,
you kind of answered it.

00:50:20.060 --> 00:50:24.540 align:center
And just to pick one more
thing, just for me, it is--

00:50:24.540 --> 00:50:26.190 align:center
again, I'm a brand
new professor,

00:50:26.190 --> 00:50:32.160 align:center
and so I am trying to learn
to be student and also

00:50:32.160 --> 00:50:33.010 align:center
be professor.

00:50:33.010 --> 00:50:35.730 align:center
And on top of I'm also
getting my PhD, so it's like,

00:50:35.730 --> 00:50:37.230 align:center
ooh, I don't really have--

00:50:37.230 --> 00:50:38.490 align:center
I'm running out of--

00:50:38.490 --> 00:50:41.280 align:center
my patience is running thin when
I'm sitting here writing 30-page

00:50:41.280 --> 00:50:44.410 align:center
papers, but also, I
am teaching you guys.

00:50:44.410 --> 00:50:46.140 align:center
So it's like, hey,
I need for you

00:50:46.140 --> 00:50:49.200 align:center
guys to communicate with me as
much as I'm also communicating

00:50:49.200 --> 00:50:50.500 align:center
with my own professors.

00:50:50.500 --> 00:50:52.830 align:center
So it's like I'm just
trying to just figure out

00:50:52.830 --> 00:50:57.670 align:center
the best way to trying
to, hey, Matthew,

00:50:57.670 --> 00:51:01.540 align:center
you were an undergrad
once before.

00:51:01.540 --> 00:51:06.360 align:center
Give that same grace that you
were given back in the day.

00:51:06.360 --> 00:51:10.390 align:center
So yeah, it's still a
learning curve, but hey.

00:51:10.390 --> 00:51:11.240 align:center
KEVIN LIN: Yeah.

00:51:11.240 --> 00:51:13.282 align:center
And I think, especially
if you're thinking about,

00:51:13.282 --> 00:51:15.380 align:center
Is this change possible?

00:51:15.380 --> 00:51:17.180 align:center
Do I have to get
approval for this change?

00:51:17.180 --> 00:51:18.888 align:center
I think some of the
bigger grading ideas,

00:51:18.888 --> 00:51:21.610 align:center
like having changing your
entire course grading policy,

00:51:21.610 --> 00:51:24.310 align:center
depending on your school's--

00:51:24.310 --> 00:51:27.190 align:center
the culture around
how change happens,

00:51:27.190 --> 00:51:29.253 align:center
it could be better to
chat with someone else.

00:51:29.253 --> 00:51:30.670 align:center
I know for here
at UW, it would be

00:51:30.670 --> 00:51:32.878 align:center
better for me to chat with
someone else, for example.

00:51:32.878 --> 00:51:35.462 align:center
But I think there are even small
things you can do to be like,

00:51:35.462 --> 00:51:37.970 align:center
hey oh, I noticed that I'm
already giving this assignment.

00:51:37.970 --> 00:51:41.000 align:center
Or I'm already giving this exam
or this assessment or this quiz.

00:51:41.000 --> 00:51:42.470 align:center
And if I'm reassessing
it later on,

00:51:42.470 --> 00:51:43.720 align:center
I can just do that clobbering.

00:51:43.720 --> 00:51:45.720 align:center
It can be on a quiz level,
like you were saying,

00:51:45.720 --> 00:51:47.145 align:center
I think, for your examples.

00:51:47.145 --> 00:51:49.520 align:center
So if you're asking the same
types of questions later on,

00:51:49.520 --> 00:51:51.940 align:center
you could say, hey, Could
that later assessment

00:51:51.940 --> 00:51:53.620 align:center
be used to improve
your earlier score?

00:51:53.620 --> 00:51:55.940 align:center
without really having to
do more work for yourself

00:51:55.940 --> 00:51:56.860 align:center
because, like I
mentioned, I think

00:51:56.860 --> 00:51:58.870 align:center
this could be set up as a little
spreadsheet thing where you just

00:51:58.870 --> 00:52:00.828 align:center
say, if this number is
bigger than this number,

00:52:00.828 --> 00:52:02.298 align:center
then replace it with that.

00:52:02.298 --> 00:52:04.090 align:center
So I think that's
something that can really

00:52:04.090 --> 00:52:05.450 align:center
look at your own workload.

00:52:05.450 --> 00:52:07.162 align:center
And I agree.

00:52:07.162 --> 00:52:08.860 align:center
I think it's not really on--

00:52:08.860 --> 00:52:12.000 align:center
I think you should look at your
own capacity, too, and be like,

00:52:12.000 --> 00:52:13.690 align:center
can I offer this to students?

00:52:13.690 --> 00:52:16.420 align:center
And out of all the different
ways I could use my time,

00:52:16.420 --> 00:52:18.840 align:center
what's the best way for
me to use it to benefit

00:52:18.840 --> 00:52:20.080 align:center
the students that I want?

00:52:22.745 --> 00:52:24.870 align:center
ERIC TREKELL: I do want to
be cognizant of the fact

00:52:24.870 --> 00:52:29.370 align:center
that our accommodation staff
do have other assignments they

00:52:29.370 --> 00:52:31.900 align:center
have to get to, so we've
only got a few more minutes.

00:52:31.900 --> 00:52:35.040 align:center
But, Eric you had your
hand raised, if you'd

00:52:35.040 --> 00:52:36.160 align:center
like to ask your question.

00:52:36.160 --> 00:52:39.720 align:center
And we may have to
do some follow-up

00:52:39.720 --> 00:52:41.860 align:center
emails with some questions.

00:52:41.860 --> 00:52:43.650 align:center
We'll see.

00:52:43.650 --> 00:52:44.500 align:center
ERIC: Thank you.

00:52:44.500 --> 00:52:45.340 align:center
I appreciate that.

00:52:45.340 --> 00:52:46.990 align:center
Kevin, Great presentation.

00:52:46.990 --> 00:52:48.820 align:center
It's really helpful
and informative.

00:52:48.820 --> 00:52:54.240 align:center
I'm a adjunct faculty for an
online master of social work

00:52:54.240 --> 00:52:55.180 align:center
program.

00:52:55.180 --> 00:53:01.020 align:center
And with that being said, I find
that a lot of my assignments

00:53:01.020 --> 00:53:04.380 align:center
are more writing and reflective.

00:53:04.380 --> 00:53:07.870 align:center
And I found it always
really uncomfortable

00:53:07.870 --> 00:53:11.560 align:center
grading reflective assignments
using a standard grading model.

00:53:11.560 --> 00:53:16.360 align:center
And I'm by no means an expert
in the field of education

00:53:16.360 --> 00:53:19.030 align:center
or higher ed, so
I'm curious if you

00:53:19.030 --> 00:53:25.390 align:center
have any recommendations for
an alternative means of grading

00:53:25.390 --> 00:53:30.513 align:center
reflective essays and
competency-based assignments?

00:53:30.513 --> 00:53:31.180 align:center
KEVIN LIN: Yeah.

00:53:31.180 --> 00:53:32.770 align:center
Thank you for the
question, Eric I

00:53:32.770 --> 00:53:35.810 align:center
think maybe I'll say something
that might sound a little funny,

00:53:35.810 --> 00:53:37.940 align:center
and then hopefully we can
dig into it a little bit.

00:53:37.940 --> 00:53:40.390 align:center
So the funny thing I'll
say is, have you considered

00:53:40.390 --> 00:53:42.282 align:center
not grading it per se?

00:53:42.282 --> 00:53:43.490 align:center
Because I think you're right.

00:53:43.490 --> 00:53:45.820 align:center
I think if you feel
uncomfortable or feel

00:53:45.820 --> 00:53:48.180 align:center
like it's not necessarily
aligned with rubrics

00:53:48.180 --> 00:53:50.180 align:center
or it's difficult to
figure out how to align it,

00:53:50.180 --> 00:53:51.580 align:center
maybe you don't have
to grade it in the way

00:53:51.580 --> 00:53:53.305 align:center
that you think of it
in terms of quality.

00:53:53.305 --> 00:53:54.680 align:center
You can say maybe,
did you do it?

00:53:54.680 --> 00:53:56.770 align:center
Do you have requirements,
like word count?

00:53:56.770 --> 00:53:58.520 align:center
Or does it seem reflective?

00:53:58.520 --> 00:54:00.375 align:center
It could be more coarse grained.

00:54:00.375 --> 00:54:02.500 align:center
I agree, I think it could
get really hard if you're

00:54:02.500 --> 00:54:04.520 align:center
trying to say, 0 to
10, what's the quality

00:54:04.520 --> 00:54:05.810 align:center
of this reflective essay?

00:54:05.810 --> 00:54:06.860 align:center
I think that's just a--

00:54:06.860 --> 00:54:09.432 align:center
I would not want to put myself
in that position either.

00:54:09.432 --> 00:54:11.640 align:center
So I think you can still
ask for quality if you want.

00:54:11.640 --> 00:54:13.348 align:center
You can like, hey, I
want to see that you

00:54:13.348 --> 00:54:15.480 align:center
have evidence of
self-reflection,

00:54:15.480 --> 00:54:16.313 align:center
like a 0 or 1 thing.

00:54:16.313 --> 00:54:18.647 align:center
Or it could be like it meets
a certain word count that's

00:54:18.647 --> 00:54:19.500 align:center
like a 0 or 1 thing.

00:54:19.500 --> 00:54:20.570 align:center
And that's also
easy for students

00:54:20.570 --> 00:54:21.653 align:center
to be able to self-assess.

00:54:21.653 --> 00:54:24.600 align:center
Oh yeah, I can just check my
word count, or I can check that,

00:54:24.600 --> 00:54:26.160 align:center
does it seem
reflective to myself?

00:54:26.160 --> 00:54:27.890 align:center
And hopefully it's
somewhat unambiguous

00:54:27.890 --> 00:54:30.813 align:center
or less ambiguous than a 0
to 10 quality scale thing.

00:54:30.813 --> 00:54:32.480 align:center
So I think removing
the number of levels

00:54:32.480 --> 00:54:34.605 align:center
or thinking about how can
you simplify that grading

00:54:34.605 --> 00:54:36.480 align:center
scale or even
altogether, you can say,

00:54:36.480 --> 00:54:38.100 align:center
I want you to do
this reflection.

00:54:38.100 --> 00:54:41.438 align:center
And that can be convincing to
some students but not to all.

00:54:41.438 --> 00:54:43.980 align:center
I think it depends on where your
students are in the program.

00:54:43.980 --> 00:54:45.647 align:center
If they're higher
level, like 400-level,

00:54:45.647 --> 00:54:47.580 align:center
senior-level students,
maybe they'll do it.

00:54:47.580 --> 00:54:48.360 align:center
Maybe they'll be too busy.

00:54:48.360 --> 00:54:49.050 align:center
Maybe they'll be checked out.

00:54:49.050 --> 00:54:50.790 align:center
I think have to look at your
own students and be like,

00:54:50.790 --> 00:54:52.110 align:center
what if I didn't grade it?

00:54:52.110 --> 00:54:53.777 align:center
What if I just said,
you should do this.

00:54:53.777 --> 00:54:55.495 align:center
And we'll read it, or
I'll read it later.

00:54:55.495 --> 00:54:56.870 align:center
And I'll give you
feedback on it.

00:54:56.870 --> 00:54:58.430 align:center
Maybe students will appreciate
that you're giving them

00:54:58.430 --> 00:55:01.050 align:center
feedback, even though there's
no grade attached to it, per se,

00:55:01.050 --> 00:55:03.100 align:center
the fact that you're
looking at it, that they're

00:55:03.100 --> 00:55:05.213 align:center
expecting you to
respond, you're expecting

00:55:05.213 --> 00:55:06.380 align:center
them to give them something.

00:55:06.380 --> 00:55:08.088 align:center
Or maybe you do it in
a different format,

00:55:08.088 --> 00:55:10.352 align:center
where maybe you say,
email me your reflection.

00:55:10.352 --> 00:55:12.310 align:center
And that way it doesn't
feel like an assignment

00:55:12.310 --> 00:55:13.990 align:center
in the same way, right?

00:55:13.990 --> 00:55:17.300 align:center
And it does enable you
to have a conversation,

00:55:17.300 --> 00:55:21.280 align:center
so maybe changing the ways
in which you formulate that.

00:55:21.280 --> 00:55:22.748 align:center
Those are just some ideas.

00:55:22.748 --> 00:55:24.290 align:center
I hope some of them
sound ridiculous,

00:55:24.290 --> 00:55:27.190 align:center
but I think by looking
past our typical bounds of,

00:55:27.190 --> 00:55:29.320 align:center
this is how we typically
do an assignment,

00:55:29.320 --> 00:55:30.970 align:center
I think really unleash
some creativity

00:55:30.970 --> 00:55:32.560 align:center
in terms of changing that
relationship with students

00:55:32.560 --> 00:55:33.060 align:center
on that.

00:55:35.530 --> 00:55:37.820 align:center
ERIC TREKELL: Not sure
what was going on there,

00:55:37.820 --> 00:55:39.570 align:center
but I apologize for that.

00:55:42.370 --> 00:55:43.160 align:center
Let's see.

00:55:43.160 --> 00:55:45.790 align:center
We've got about three
minutes, so maybe we

00:55:45.790 --> 00:55:48.840 align:center
can catch one
question in the chat.

00:55:51.790 --> 00:55:55.720 align:center
Kevin, Leah asked, I'm
teaching a capstone class

00:55:55.720 --> 00:55:58.250 align:center
on software design and
development at Grinnell.

00:55:58.250 --> 00:56:00.340 align:center
How do you think we
should assess student work

00:56:00.340 --> 00:56:02.090 align:center
in an upper-level
class, where students

00:56:02.090 --> 00:56:04.580 align:center
can learn different
things from each other

00:56:04.580 --> 00:56:07.183 align:center
and can learn things that
instructor didn't directly

00:56:07.183 --> 00:56:07.683 align:center
teach?

00:56:10.725 --> 00:56:13.100 align:center
KEVIN LIN: Ooh, how do you
think we should assess student

00:56:13.100 --> 00:56:14.510 align:center
work in upper-level
class, where students

00:56:14.510 --> 00:56:16.260 align:center
can learn different
things from each other

00:56:16.260 --> 00:56:18.710 align:center
and can learn things that the
instructor didn't directly

00:56:18.710 --> 00:56:19.298 align:center
teach?

00:56:19.298 --> 00:56:19.798 align:center
Huh.

00:56:22.490 --> 00:56:24.900 align:center
I have not necessarily been
put in that situation before,

00:56:24.900 --> 00:56:26.677 align:center
but that's a really
interesting question.

00:56:26.677 --> 00:56:27.760 align:center
Let me think a little bit.

00:56:38.300 --> 00:56:39.510 align:center
Yeah, creative work.

00:56:39.510 --> 00:56:42.920 align:center
I think a little bit to the
earlier point, I think Eric

00:56:42.920 --> 00:56:45.140 align:center
and I were having this
discussion in the chat here

00:56:45.140 --> 00:56:46.630 align:center
or in the meeting right now.

00:56:50.456 --> 00:56:53.895 align:center
I think if you can make--

00:56:53.895 --> 00:56:55.270 align:center
even though it's
creative work, I

00:56:55.270 --> 00:56:57.270 align:center
think there are probably
still some expectations

00:56:57.270 --> 00:56:58.890 align:center
for the format of the response.

00:56:58.890 --> 00:57:01.380 align:center
You're expecting them to
have maybe a written essay.

00:57:01.380 --> 00:57:03.130 align:center
Or maybe they're
answering some questions,

00:57:03.130 --> 00:57:04.870 align:center
or they're having
some kind of analysis.

00:57:04.870 --> 00:57:06.420 align:center
Or maybe there's
some kind of end

00:57:06.420 --> 00:57:08.950 align:center
artistic, creative product
that they're producing.

00:57:08.950 --> 00:57:10.620 align:center
There are probably
maybe at least

00:57:10.620 --> 00:57:13.192 align:center
some amount of standardization
in the format of that.

00:57:13.192 --> 00:57:15.525 align:center
So maybe you could look at
what are the elements of that

00:57:15.525 --> 00:57:17.417 align:center
that you want to see there.

00:57:17.417 --> 00:57:18.250 align:center
Now, it's up to you.

00:57:18.250 --> 00:57:20.320 align:center
You could say maybe there's
quality within that.

00:57:20.320 --> 00:57:21.820 align:center
Maybe you might say
that the quality

00:57:21.820 --> 00:57:24.180 align:center
of this particular
artistic, creative output

00:57:24.180 --> 00:57:25.590 align:center
might still be
something you rank

00:57:25.590 --> 00:57:30.463 align:center
from 0, 1, 2, or 3 or some kind
of simple rubric like that.

00:57:30.463 --> 00:57:32.380 align:center
I think that's a little
bit opaque to students

00:57:32.380 --> 00:57:33.900 align:center
because they don't
really know if they've

00:57:33.900 --> 00:57:35.983 align:center
met the quality that you're
looking for until they

00:57:35.983 --> 00:57:36.760 align:center
get that feedback.

00:57:36.760 --> 00:57:38.177 align:center
But then maybe
that it's important

00:57:38.177 --> 00:57:40.390 align:center
that you have feedback
during the term to be like,

00:57:40.390 --> 00:57:42.057 align:center
I'm actually giving
you helpful feedback

00:57:42.057 --> 00:57:45.750 align:center
to help you see where you stand
right now and help direct you

00:57:45.750 --> 00:57:48.720 align:center
in directions that will make
it so that this larger learning

00:57:48.720 --> 00:57:52.048 align:center
activity ends up panning out the
way you expect it to pan out.

00:57:52.048 --> 00:57:54.340 align:center
So I think it's giving students
feedback along the way.

00:57:54.340 --> 00:57:55.950 align:center
I think it's
thinking about, what

00:57:55.950 --> 00:57:59.070 align:center
are the elements of the
format and the requirements

00:57:59.070 --> 00:58:02.432 align:center
that can be specified, can
be codified into a rubric?

00:58:02.432 --> 00:58:03.890 align:center
But also maybe
there are some parts

00:58:03.890 --> 00:58:05.630 align:center
that aren't easily
codified, in which case

00:58:05.630 --> 00:58:07.370 align:center
you're giving them feedback
throughout that process

00:58:07.370 --> 00:58:09.290 align:center
to help students feel
prepared for what

00:58:09.290 --> 00:58:12.230 align:center
you're assessing at the end.

00:58:12.230 --> 00:58:13.230 align:center
ERIC TREKELL: All right.

00:58:13.230 --> 00:58:19.910 align:center
I do want to make sure that we
release the accessibility staff

00:58:19.910 --> 00:58:21.240 align:center
in a timely fashion.

00:58:21.240 --> 00:58:23.220 align:center
So we are going to stop here.

00:58:23.220 --> 00:58:25.583 align:center
I'd just like to thank
you, Kevin, so much.

00:58:25.583 --> 00:58:27.000 align:center
I've been looking
forward to this.

00:58:27.000 --> 00:58:29.940 align:center
I know been communicating for a
while to try to get this set up.

00:58:29.940 --> 00:58:32.210 align:center
So thank you so much.

00:58:32.210 --> 00:58:35.490 align:center
Lots of positive
response in the chat.

00:58:35.490 --> 00:58:39.750 align:center
Everyone, if you would please
complete the evaluation,

00:58:39.750 --> 00:58:42.230 align:center
the short evaluation
that we have shared out.

00:58:42.230 --> 00:58:45.300 align:center
Thank you to our interpreters,
to our CART provider.

00:58:45.300 --> 00:58:47.120 align:center
Thank you all for
joining us today,

00:58:47.120 --> 00:58:51.460 align:center
and have a good day
the rest of your day.