Integrating Ethics into Data Science Courses

1

Goal for today

I want you to leave with:

2

How can we support students in becoming ethical data science practitioners?

Note

I want to begin with motivation to make it clear what my approach is.

The goal we have will shape how we get there

Focusing on practitioners makes clear what aspects are and are not important we can focus only on teaching how and when to employ ethical decision making, and how to gather information and engage in the conversations. We do not necessarily need to teach broad ethical frameworks, but how to identify when to do things.

Sources of competence

3

How we will do that?

Note

In order to take a practitioner focused approach; I’ll start with how I developed ethical thinking skills.

How I learned.

Then I’ll go through some

4

My ethical training

5

I am an engineer

6

Note

Engineering Design process

In my program our first engineering in major requirement is/was engineering design: a whole semester on this process (slide). We go through these phases in broad, minimally technical cases and we reviewed engineering failures and studies how and why they went wrong. We did this BEFORE, we took any other engineering classes; in parallel to calculus, and physics or chemistry prereqs.

7

Correctness for Safety

Note

hey constantly preached at us to both check our work throughly using the skills from our math classes for checking solutions, but also, to, more importantly, before getting into any algebra, do some napkin math. To have a few constants, a few settings of things memorized, or really to do problems until you had the intuition to look at an answer and know if it was off by an order of magnitude or not.

Context

8

Experiential Education

Note

Note: this slide serves two purposes, it fits here, in the narrative of my development and some core examples.

Also, I’ll pause to acknowledge that two of these are defense contractors and that often causes people to take pause.

moral vs ethical

2nd: context again; of these choices my coops were in 2008,9,10; recession & war

Hostpitals and military all have strict ethical guidelines and a culture of adhereing to that. Even if you don’t agree that war is ever necessary or appropriate; the military is an example of ethics which is a set of agreed upon terms. Note that this is differnt from morals, but we will come back to that. Time in these environments exposed me to the ways that ethical expectations are promulgated in these environments and on at least the small teams that I was on this was taken very seriously and led to actual conversations about what the right way to do something was and when tasks were in gray areas, that is things that were deemed acceptable, but easily mistaken for something that would not be acceptable, or unpopular very long conversatiosn about what we did and did not do or say.

CONTEXT

9

Interdisciplinary ML work

Note

In my lab, every student had a collaborator with domain expertise: I worked with pyschologists, a lab mate worked with civil engineers studying infrastructure health; other worked with physician scientists studying long time scale COPD; other with p-s at memorial sloan kettering treating skin cancer, looking for better diagnostic tools. This experience again led me to see exactly how the ethics of machine learning application needs to work in practice and form the core position that makes a workshop like this so hard.

AFOG– role of profesionaliation

medicine again; practioners interacting with …

10

Ethics requires context and is concerned with impact

11
Missing context is endemic in ML/DS/AI – even in AI ethics work

Note

This is my experience

and it was the conclusion of a meta-analysis of over 500 papers from the AIES and FAccT. Work was primarily abstract.

12

So what can we do?

13

Start Small

Note

This also gives you time to learn more; start by modeling what you know.

14
Effective, inclusive teaching is important to me

Note

Ethics as Good Pedagogy

In addition to my own ethical development, I am going to pause to note in particular my training in evidence based teaching practices.

15

Things you can do this semester

… and the next few

Note

  • Emphasize Evaluation
  • Highlight Context
  • Consider downstream effects
16

Really, this semester

These require:

Note

All of these are based in pedagogy

17

Emphasize Evaluation

in Instruction

  • Frame evaluation as competing goals

in Assignments

  • Always require more than one metric

  • Disaggregate by group in social data

Note

Evaluation allows students to think about problems more deeply.

Fairness issues “discovered” in ML through eval

It helps monitor, considering broader impact

18

Optimal is relative

in Instruction

  • Simple repeated saying

in Assignments

  • require justification on the word “best”

Note

Simple, easy to remember reminders can help students start thinking and questioning what they do.

19

Talk about where data comes from

in Instruction

  • Discuss before loading data

  • Distribute Datasheets for Datasets

in Assignments

  • Have students write data sheets

  • Ask about limits of a dataset

20

Robots.txt

In Webscraping, we take a moment and look at the robots.txt and talk about why it is there.

21

Data descriptions

22

Consider Deployment

in Instruction

  • Discuss before loading data

  • Examine Model Cards Case studies of retracted models

in Assignments

  • add reflection questions

  • write model cards

23

Deployable? Reflection

  1. Would you feel safe if your doctor used this model?
  2. Based on the exploration of the data, can you anticipate when the model that you trained might fail?
24

What will you try?

Make a note for one thing you will try

25

Looking Forward

26

Evaluation First

(changing the order)

My Syllabus

…

  1. What is a machine learning
  2. How do we evaluate them
  3. Assignment on evaluation
  4. How do we train classifiers
  5. Assignment on classification …
27

Embed activities in Ethical Challenges

Choose completely datasets that present ethical challenges for demonstration or assignments

28

General Strategies for Integrating Ethics

Note

No modules

29

Questions?

brownsarahm@uri.edu

30