Data Driven: Authentic Assessment and a Data Based Business Case Study

What is authentic assessment in the math classroom? It’s probably not a math test. Tough to admit, as I give lots of math tests, but a test is so limited, so contrived, so singular. The most authentic assessment I have been part of in the math classroom was our culminating project for Data Driven this summer – a business case study presented to people in the business world.

A friend of mine from college who now works for a predictive business analytics company ran a case study on my students. The case was for a bagel store that wanted to expand – they had data on the profit of their current stores over time, and data on features of the current stores. In teams of four, students had to advise the bagel company on where the company should build 10 new locations, and what the layout should be. My friend served as the lead of the company’s expansion team – the students had a halfway call with him and could email him at any point during the week with questions or requests for data. At the end of the week, students presented (via Skype) their recommendations and defended them with questions.

skyping

Students during the final Skype presentation – feeling super awkward, but speaking with conviction about their recommendations.

HOW WAS THIS AUTHENTIC?

  • We learned a billion things and amassed a ton of data analysis tools this summer – instead of being directed what to use where, students had to sift through their knowledge to figure out what was appropriate. Though they received an initial prepackaged dataset, the problem was wide open and had very little hand holding. If they wanted to use census data about median incomes in zip codes, they had to go find that data, clean it up and attach it to the given dataset before they could use it.
  • All the math that they were doing was supporting a genuine and interesting, multifaceted problem, instead of being motivated by just being a question on the test. If they needed to do a multiple linear regression, it was because they wanted to figure out something about the data, not because a question asked them to do a multiple linear regression.
  • slides

    Slides from the final student presentations.

    In addition, it was a problem that forced them to translate their mathematical knowledge into human decisions. They had to tell the story that the data was presenting, had to make choices that didn’t have a “correct” answer, and had to defend everything they were doing in a way that a naive non-math outsider could understand.

  • Presenting to an outside audience forced them to be as prepared as possible, and also taught them a lot of lessons about communication! I wish I had taken a picture of one group when they were on a conference call with my friend. They were pacing around the room, hands on heads, brows furrowed, goofy smiles from feeling awkward – so much more learning was happening than if they were presenting to me! I also just had to sit and watch them struggle through things, like explaining what a t-test was, during their final presentation, which gave me deep insight into the results my teaching.
  • There were many points of entry and many different depths that students could take it. There were immediate things that anyone could do, and things that only a professional data scientist could have done, which made the problem perfect to test everyone, but give the students needing a bigger challenge a place to go.

HOW WAS IT STILL INAUTHENTIC?

  • The data was fake, the business fake, the audience fake. The advantage to this was that I could ensure that the math involved was the right level, and that the problem was doable, but perhaps this took something away from the motivation for the students.
  • There was no followup from the final result. Wouldn’t this have been even more awesome if they were presenting to a real company, or community organization, that was trying to make a real decision? And then they could see what the company actually decided and see what the results were.
  • There were students in each group that didn’t contribute. I don’t think anyone didn’t want to contribute, but it’s really hard to work in teams. I think that this exercise tested their collaboration skills, but perhaps didn’t assess every single student’s math skills.

THE UPSHOT:

Though this course was unique in its format (long 4 hour classes, only 12 students, no curricular pressure) and did not come with grades, there is so much from this to take to my school-year classroom. How can I include more authentic assessments in my day to day classroom life? Assessments with multifaceted, human problems that motivate great math along the way; ones with many points of entry and many places to go; and ones where they have to defend their decisions to audiences other than me.

It’s important to remember that “authentic” is not a binary designation, so my goal is to add pieces of the above to my normal classroom assessments one step at a time.

 

 

 

Data Driven: A Syllabus

As I start reflecting on the course I taught this summer, I thought I’d start by sharing my Syllabus for anyone curious. The course was a functional data course – the focus was more on being able to DO things rather than on abstract statistical work. We used data visualization software geared at businesses (Tableau), coded in R, conducted election polling, performed original research projects, wrestled over issues of data privacy, cracked codes, and put together advice for a business on how they should expand (amongst many, many other things). It was exhausting and awesome. More reflections to come!

(if that is too small below, here is a google drive link)

Data Driven Day 1: Data Speed Dating & Dear Data

This summer, I’m teaching a 5 week intensive course called Data Driven (course description) at this amazing summer program at St. Paul’s School in NH called the Advanced Studies Program. It’s an enrichment program for rising high school seniors. We are doing class 3-4 hours a day, 6 days a week for 5 weeks, with tons of time for independent work at night. The class is about creating functional data mavens – think statistics, plus data science, plus research, plus data ethics/privacy, plus cryptography, with a whole lot of reading, coding, writing, computing and interacting with the community along the way.

DATA SPEED DATING

After a quick math-themed icebreaker, we started our data class this summer with a few data themed get-to-know-you activities, the first being data speed dating. Each student picked a categorical variable and a quantitative variable that they wanted to collect from every student in the class. They then sat across from each other and “speed dated” to collect the info from each person in the class.

roundrobin

It was nice to knock out the kind of dumb and easy idea of variable types in an icebreaking activity, and it was great that every single student had a conversation with every other student in the class (only 12 students).

Then, I paired them up and each pair had to pick one of the sets of data to present visually to the class. I wanted to get them started on culling the most interesting data from a data set, picking appropriate visualizations, and translating data for others. One group did this kind of funny infographic describing how many pairs of pants were owned by people who preferred certain movie types. Problems with the visualizations, of course, but interesting nonetheless (and hey, it was the first half hour of class). In retrospect, I wish I had explicitly said “Combine TWO of your pieces of data in a visualization” because I think that would have been a much more interesting intellectual challenge (and would have led to a bunch of silly things!).

File_000

DEAR DATA

Then, I introduced our homework for the night, which fell on similar lines. It was based on the project Dear Data by two data scientists Giorgia Lupi and Stephanie Posavec. They picked a broad topic (like “laughter”, “books”, “thank yous”) at the beginning of a week, and each chose what data they were going to collect about that topic. At the end of the week, each turned their data into a beautiful visualization on a postcard, with the key on the back, and sent the postcards to each other (one was in London, one in NYC).

deardata-week06-stefanie

For my students, we picked the topic “New Encounters,” as they are all starting this program with a bunch of people they don’t know. They each brainstormed the data they were going to collect, and I gave them mini-reporter notebooks to carry around. From what I saw when they were working on them earlier tonight, some of the visualizations that the students did were just as beautiful as these professional data scientists (and some managed to collect 70-80 points of multidimensional data in a day and a half). Will post once I see them tomorrow!

AP Calculus AB Skill Drills

I have had MANY requests for the actual files for my AP Calculus Skill Drills – a 5-10 minute start to class every day for a couple of weeks leading up to the AP Calculus AB exam. Below is the file. Know that it is fairly specific to my class – they are categorized based on my standards and we voted which standards to keep reviewing that day – but still should be a decent review for anyone if you want to modify. There are 10 days of review goodness, which according to the file, I started on April 9th a few years ago. Forgive any errors of course.

Best of luck prepping kiddos for the exam soon.

<3,
Bowman

The Related Rates of an Automatic Pizza Saucer

I always struggle a bit with teaching related rates, because the focus of all textbook/AP problems is to calculate how fast something is changing in one single moment in time, which short shrifts the beauty of the topic. The main idea seems more to be thinking about how two rates are, well, related to each other. What must happen to one rate as time goes on if another rate stays constant? That’s what makes that classic ladder problem even remotely interesting (why the hell would one side be moving at a constant rate while the other is speeding up?).

I saw this gif of an automatic pizza saucer a while ago and immediately thought it would be a fabulous discussion piece for this very idea:

anigif_original-grid-image-20204-1416002502-5

Someone who designed this used some sort of calculus, even if it was the intuitive kind! We talked about this in class for 15 minutes or so, and the students that enjoy wrestling with not-so-obvious and applied situations really enjoyed thinking about these types of questions:

  • If the pizza spins at a constant rate and the sauce comes out at a constant rate, what has to happen to the speed of the arm?
  • If the pizza spins at a constant rate and the arm moves at a constant rate, what has to happen to the rate at which the sauce comes out?
  • If the arm of the pizza moves at a constant rate and the sauce comes out at a constant rate what has to happen to the rate at which the pizza spins?

We came to some interesting conclusions about the above, including that one of the situations above is not possible (I think!), which we only figured out because one student and I were vehemently defending two different and opposite things.

Anyone want to try to throw some numbers on this to figure out these very questions?

More Reflective Homework

This year, I have tried to engage my students in a more thoughtful homework process. I don’t think any math teacher, ever, has been satisfied with the way homework works in their class, and I would certainly put myself in that boat. My frustrations in the past have been that students sometimes would do something wrong and then continue to cement that wrong thing by repetition, I would get 30 homework assignments that look basically the same and spend tons of time giving useless feedback that they didn’t really even look at, and students were focused on completion over learning. I attribute this to the structure of the homework over students being their nutter butter selves. Here are the changes I made this year:

1. Every homework assignment comes with a full solution (not just answer) guide. It’s more work for me, but also makes me assign a reasonable amount of homework.

2. Students go through the assignment and do whatever they can without the solution guide.

3. Then they check the solution guide to check what they did and finish what they couldn’t. Anything they write after this point (or using the solution guide) is in a different color – which is a crucial point. They check their answers, fill in the rest of incomplete solutions and give themselves feedback on what they did well and what they did poorly.

It takes a little longer for the students, so I try to assign a little less. And some students haven’t bought totally into it yet (slash never will). But as a teacher grading it, I can see so much more. Like…

  • Where students struggled and what they still don’t understand well, which is so obvious with the colored pen. What they did in pencil is their work and what they did in pen is their work with the solution guide.
  • Evidence of learning – instead of doing something wrong over and over, they correct it and do it better the second time around, or at least know that what they did is wrong and need to get help from me.
  • Where to give them feedback on the specific things that they are struggling on.
  • Who is engaging with the homework and trying to learn from it, vs. who is just tryna get-r-done.

I also spend less time grading homework while still giving better quality feedback. I think they spend about the same amount of time doing homework but get more out of it.

The training process for this has been an investment, but worth it. I share with the class examples of things they can do to do this better, like this:

blogHW3

blogHW2

blogHW1

Feedback from students has been that they almost either really like it, or are fine doing it. They almost all indicate that it’s better for learning, which is what I care about.


How do you feel about the method of doing homework where you check your own answers?

It is very helpful XXXXXXXXXX

  • It allows you to learn the right way of doing it while it’s still fresh in your mind.
  • I like understanding what I did wrong right after I did it so that I can grasp what I did wrong.
  • Being able to look at the answer and find what I did wrong at my own pace helps me understand the problem and how I should do it next time.
  • Writing my own feedback is more helpful than skimming any you would give on homework.
  • Self check is a way to see what you did wrong right after you did the work instead of a couple of days later,

It’s fine XXXXXXX

  • I feel as though that making corrections and not totally understanding my mistakes is perhaps the biggest downfall.
  • maybe if i came back after a longer period of time it would be more helpful to me in particular.
  • I understand that it’s good to correct ourselves but I think I get more out of simply going up to you to clarify he things I’m struggling with.
  • I only feel like feedback is necessary for some problems if I really don’t get it
  • Well it is helpful some of the time but it does take a really long time to do this.
  • I think that it’s helpful like 85% of the time, and then other times it confuses me

Negatively X

Meh, I don’t really do it. XX


Still experimenting! Would love some thoughts.

Blogging About School Leadership for Klingspace

I have been quieter here than I normally would be during summer planning because I have been blogging at Klingspace, a blog run by the graduate school program from which I graduated in May. Below are the posts I have published there with a brief summary, if you are interested in reading about topics that may not be as math education focused as I usually am. 

I am excited to rejoin a math classroom in the fall and hope to re-engage in the math education discussion on this blog that I am used to!

  • 5/22 – Structure Is Not the Opposite of Autonomy – We shy away from procedures, structures and limitations in the name of creativity, but that structure can actually promote creativity.
  • 5/28 – Keeping the Change: How > What – Success naturally breeds resistance to change, which means we must be sensitive to the fact that our change-filled futures are challenges to our success-filled pasts. Give people time to process change.
  • 6/4 – Teacher Observation: Informing Practice, Not Judgment – The way most schools structure observations and evaluations make us see them as moments of judgment instead of opportunities to improve our practice.
  • 6/11 – Have a GSA? Great! But It’s Probably Not Enough – There are queer students at our schools who aren’t served by simply having a GSA. More generally, we should not assume that because we have programming for X type of students that it serves every student who identifies as X.
  • 6/28 – Using “Creative Tension” To Communicate Change – If leaders effectively show faculty the gap between their vision and the current program, faculty will be more likely to feel the need to move toward the vision
  • 7/11 – Cultivating a Growth Soulset – Just as we can always learn more with a growth mindset, we need to tend to our emotional intelligence with the attitude that we can always become more emotionally adept.
  • 7/21 – The Case Against a Linearly Sequenced Curriculum – Research about distributed practice suggests that studying something with space between is always more effective than studying it for the same amount of time uninterrupted. How can we incorporate this finding into our curriculum design?
  • 7/29 – TBA
  • 8/15 – TBA

Practicing Like an Expert (instead of a math student)

(here are some excerpts from a paper I wrote for grad school about structuring math homework for better learning – the full paper is below)

mathshampooThe traditional structure of math homework (e.g. 1-67 odd) forces students to work hard, but not effectively, as all students blindly do the same assignment consisting of a similar number of each type of problem regardless of each student’s personal weaknesses. In “Practice Perfect,” Doug Lemov’s book on how to practice more effectively, the authors compare this type of practice to shampooing your hair, something we repeat daily but probably never improve on.

Math students ought to practice math the way that experts in other fields practice. When a musician learns a piece of music, they do not just play whole piece over and over. Instead, they workshop specific parts that they need to work on. When I was learning to play piano, my teacher would have me play difficult parts repeatedly – first each hand separately, then both hands together slowly, and finally at full speed. Students do the exact opposite on math homework – they do the problems with which they are comfortable and then leave blank those they do not know how to do. Thus, they are only practicing the very material that they do not need to.

If teachers would like them to engage with it differently, they need to make intentional changes in the structure. To make math homework more like expert practice, teachers should:

Force students to differentiate their homework experience.

Though the other suggestions below would be a helpful addition to traditional homework assignments, this first one would require a more radical shift. Instead of a linear assignment that encourages students to spend an equal amount of time on each part of the course, a math assignment should consist of minimal core problems for each learning objective that each student must complete, and a bank of other problems that the student could use to remedy any misconceptions. The set of core problems should be small enough that students can complete all and still have time to tackle their weak areas. Students should be instructed that a wrong answer should be a sign to reflect for a moment about what went wrong, perhaps even formally, and then immediately try more problems until they understand. To give space for this thoughtful type of work, a teacher might have to assign less work, but the quality of the work that is completed has the potential to be much higher.
(some thoughts on grading and accountability in the full paper)

Make homework objectives transparent.

With a differentiated homework assignment that required metacognition about weaknesses, students would need the tools to pick out their own weaknesses. When picking out problems from a math textbook to assign, I have an objective in mind for each group of problems. In retrospect, it seems obvious that I should simply share these objectives with students, paralleling Standards Based Grading for the wider structure of the class. Simply grouping math problems into learning objectives would help students focus their effort more effectively by allowing them to isolate skills and measure their success.

Ensure students can get immediate and actionable feedback

But even with clear learning objectives, students can’t make progress without feedback. Too often, students will power through their entire homework doing something wrong the entire time, encoding something in their brains the wrong way. Worried that students will copy answers out of the back of the book, teachers will assign problems that do not have attached solutions. We have to get over this fear – if students cannot check their work, or are not in the habit of doing so even when they are confident they got a problem correct, they risk not knowing that they are doing something wrong. Doing one problem wrong, fixing a misunderstanding and then doing a few more correctly will lead to far better results than doing five wrong and having to unlearn something incorrect a week later. For complex problems that only have a simple answer in the back of the book, teachers could post a solution guide that details not only the answer but the process that it takes.

No idea if this will work! I’m interested to try it out when I get back to the classroom. Here is the full paper:

Teach them HOW to do homework

One of my grad school professors taught me how to read.

Okay, so I knew HOW to read (hold your snarky math teacher comments, English folk), but I didn’t realize I had no idea how to read for an academic context. For one of our very first assignments, our professor set up a very sneaky experiment that taught me I wasn’t reading very well. He told the whole class that for one of our more dense and academic readings for the next week*, one person would be randomly selected to lead a class-wide discussion. This was early on in a program with a group of 22 all-star, experienced educators. Very scary.

I was terrified into competency.

Instead of just reading it straight through and perhaps highlighting, I wrote questions in the margins, connected various parts of the text, made a list of the main ideas, pulled out quotes that could generate discussion, and generally actively thought about the content of the article.

It turned out he was bluffing, which he revealed in class the next week. Phew, *changes underwear*.  But with this exercise, he made the point to us that the way we read that article was totally different from the way we probably read most of the other stuff. And more effective. I was thankful for this because for the rest of grad school, I read much more effectively. Even if I didn’t have time to read an entire article, I would spend a bit of time diagramming, writing questions in the margins, and actively engaging with the content. Instead of expending more effort, I used my effort more effectively.

How did I make it through so many years of education without knowing how to read? How much more could I have gained from both my high school and college education? How does this apply to our math students? How many of them are trying to do better by working MORE instead of by working MORE EFFECTIVELY? What can we do to show the how to do homework?

What’s a good meta-assignment that can show students how to do math homework effectively (without making them sh*t some bricks to learn the lesson)?

 
*Alfred North Whitehead, The Aims of Education. NB: I don’t really remember what it was all about 9 months later, but hey, I guess good teaching techniques have their limits?

Reflective –> Effective

Okay. It’s time I hopped back on the blogwagon. I have been in graduate school this past year earning a Master’s degree in Private School Leadership (yeah, that exists) but I haven’t felt like I had anything to contribute to the discussion around math TEACHING up in my ivory tower. I want to end the year by reflecting on some of the things that I have been thinking about, ranging from little random thoughts to unanswerable questions, all of which I am excited to test out next year in my triumphant return to a math classroom!

So that’s what the next series of posts will be. But first… I have wondered if it was a good/necessary move for me to take a full year away from the classroom to reflect. One the one hand, it’s great to have the space and time to really delve into issues from human cognition to the use of data to improve student learning. On the other hand, I haven’t been able to workshop any of the ideas running around in my brain. Regardless, it has driven home to me the importance of reflecting, so I thought I would share the small change last year I made in my lesson planning that helped me become a more reflective, and thus more effective teacher.

You don’t need to take a year off or find more hours in the day to journal. You just need to add a column to your lesson plans.

lesson plan2

This is an actual screenshot from my Evernote planning notebook.  I chunk my class into activities, so that’s what you see on the way left. The next column describes any other necessary details or files that the activity requires. The third column is initially blank, and this is where I reflected. Every day, before planning the next lesson, I would go back to the one before and jot down a bullet point or two about each activity. Sometimes I would have a lot to say and would write some notes for my future self, but I would, at the very least, note how much time the learning activity took. Before long, this became a habit of my lesson planning that took no more than a few minutes.

I initially conceptualized this as a way to keep notes for myself in the future, should I teach the same class again, but found that I reaped the benefits far more quickly. Just sitting down for even 5 minutes to think about what happened that day started a recursive process where my reflections allowed me to make decisions in a different way in the future.

So I hope that some of my reflections from grad school might be helpful (for both readers and myself) — but I am also looking forward to reflecting next year in the classroom in a way that has a more immediate effect on my teaching and student learning.