Once you have a code of conduct, what comes next?

In this talk from DevXcon 2017, Sarah Sharp looks at the common and complex issues around practical Code of Conduct enforcement.

Enforcing a Code of Conduct is often complex, so this talk provides a framework for evaluating the right level of response to a CoC violation, and give lots of practical (anonymized) examples.

Transcript

Hello. I’m Sarah Sharp. If you don’t know who I am, I was a Linux kernel maintainer for five years. I have also help run Outreachy, which is a three-month internship program to get underrepresented folks in tech into open source, and then I also have my own company, Otter Tech, and I do a lot of diversity and inclusion consulting with Otter Tech. And what I found as I have looked at a bunch of different technical communities, both open source and proprietary, is that a lot of them adopt a code of conduct because of some sort of reactionary event either in their community or in a neighboring community. Maybe they’ve been following the online harassment campaigns that have been happening. Maybe someone submitted a bitcoin mining code to their CI system, and they would like to be able to block them. Maybe they just want to signal to trolls that the community is welcoming and friendly and that this is what our community values are. Some communities feel pressure to feel that they can live up to other communities. If you don’t have a code of conduct, people kind of look at you funny and go, you know, “Is your community actually friendly? I don’t know.”

Other things that have happened, people publicly table flip and leave, and sometimes that leads to customer dissatisfaction. But these kinds of reasons that communities generally tend to use to adopt a code of conduct, these reactionary things, mean that people who adopt a code of conduct for their community often don’t have a plan to actually enforce that code of conduct. And so a lot of the people that I work with in my consulting firm, Otter Tech, I try to ask them three pretty simple questions. But then once you start digging into these questions, it becomes very complicated very fast. And so since this is, you know, a practitioner’s conference, I really want to give you the meaty ugly side of this, right? The things that didn’t work and did work. So let’s dig into these questions.

The 1st question is why. Not why you as a community manager, as a developer relations person want to adopt a code of conduct, but why do your customers, and your developers, and even your boss want you to adopt a code of conduct? The 2nd question is who. Who is going to enforce your code of conduct? And the 3rd question is how. How will you enforce that code of conduct? So simple questions but gets complicated as you dig in. So the 1st question is why. Why do you want a code of conduct? Most code of conducts have a statement at the beginning that says, “We value these diverse perspectives.” So call out specific demographics, and that’s because, in general, research has shown that a diverse community builds a better product. There was a study by McKinsey about companies, and they looked at companies across specific industries, and they divided them as to how gender-diverse they were. And the companies that were in the top quartile for gender diversity were 15% more likely to financially outperform companies that were in the bottom quartile, and this gets even bigger up to 35% when you look at ethnic diversity of companies. So you want that kind of diverse perspective in your developer communities.

The other thing, of course, is that it’s not enough to just bring these voices into your community, because the problem is the bias exists. Everyone has unconscious bias, and sometimes it hurts people more than others. There was a study of GitHub. The researchers pulled down GitHub user data. They linked it to people’s addresses to their Google Plus gender, through the Google Plus API, and they looked at the women who were submitting poll requests, and what they found was women were more likely to have their poll requests accepted but only if their icon and their user name were gender neutral. So this shows that there’s bias in these developers communities. So we need to figure out how to overcome that bias. And, unfortunately, a lot of communities have, you know, people who don’t necessarily listen very well, maybe they don’t wanna confront their biases, maybe they’re even toxic, and sometimes those are our best developers. And it’s hard to justify taking these people aside and having those conversations about their bias.

However, there was an interesting study that looked at employee churn in companies, specifically in relation to toxic employees, and they found that it cost companies more in terms of hiring bonuses and trying to recruit for people who have left because of toxic workers than the top 1% of their rock stars add to the bottom line. So, if you think about this in terms your developers, yes, these developers may be adding new features, maybe they’re coding off in a corner and adding new things, but in the meantime, they impact the rest of your developers, and maybe that adds to your developer churn. So it’s something to think about. There was also a study by Google. They looked, and they said, “We wanna find out what are the most productive teams. What is their characteristics?” And the surprising thing that they found after doing all the data across like people’s education level, you know, the demographics of the team, what they found was the most productive teams are ones where people are listened to and respected, where they feel like they can talk and they can actually share their ideas and have them be listened to, which brings us back to a code of conduct.

If your code of conduct says, “We value these diverse perspectives, and we want you to participate in our community,” but that code of conduct is just some text on a website, you are giving your customers and your developers a false sense of security that they will be respected and listened to. So let’s go dig into the nitty-gritties of how you do enforcement. This is part of a much longer conversation that you need to have with your communities about your culture, but this is a very simple piece that you can take back and start having conversations with. So the first thing you have to decide with code of conduct enforcement is who. Who is going to enforce your code of conduct? Because you don’t want to have to wait until an incident occurs to have to pull together a team to work on this. It looks really bad if something is publicly happening and you don’t have a plan for it. So you need to get your code of conduct team put together, and hopefully that code of conduct enforcement team will reflect the diverse values you want to encourage in your community.

This team also needs to be available around the clock, because if you’ve got an incident that happens at 3 a.m. on the West Coast, you need someone to be able to handle it, right? So, if you’re an online community, you need to think about, “Do we have enough moderators, enough people that can help us with this?” right? If you’re running events like hackathons, a lot of time there’s alcohol involved in the evening events. Do you have someone that’s going to stay sober to be able to take an incident response from someone, or are all your organizers partying with everyone else?

The other thing to note is that not everyone in your community, or sometimes not even everyone on your event staff, is going to be trained to be able to take an instant report, and respond to it, and handle that. So you’re always gonna need to have a way to identify who on your team is available to respond to incidents. Sometimes it’s with a shirt. Sometimes it’s just by telling people how they can get a hold of you. So I have an example from the OSCON, it’s an Open Source Conference Code of Conduct and how they report. So you can see on this it’s pretty simple, it’s got, you know, someone’s title, it’s got their name, they’ve got, you know, an e-mail address and then a phone number, and they also have an anonymous reporting form. Because oftentimes with code of conduct violations, the reporter risks probably, you know, a hitch to their career, they risk being kicked out of the community, ostracized by reporting. So it’s very useful to give people a way to do anonymous reporting.

I also like this example because it explicitly calls out who is going to be on the receiving end of that report. Because what happens if one of your code of conduct instant response team slips up and they make a racist joke in conversation? You need a way to be able to report your incident response team. So it’s very useful to individually list the people on that team, so that if someone, you know, has an issue, they can contact that. The other thing you need to think about is a lot of people assume that code of conduct violations are gonna come from online trolls that will wander into your community. What is going to happen when a key developer has a code of conduct violation? What happens when they have such an egregious code of conduct violation that you need to kick them out? What if they have access to your mail server? What if they have access to your chat logs? What if they can force push through repositories? You need to think about these things, not just what’s going to happen when an online troll comes in, but what happens when one of your key developers has an issue. It’s a completely different threat model essentially, but it’s something to think about.

Another thing is the privacy of the reporter information. A lot of communities and a lot of events want to be able to pass on knowledge of incidents to the next year’s event staff or to new community mods, so that if there’s a pattern of repeated boundary pushing, you can see that pattern and, you know, notice that pattern and change your response based on that pattern. But you have to think about how you’re going to handle that data. If you’re just putting it in a Google doc, who has access to that Google doc? Could someone dig through the revision history and the comments and find out, you know, who decided particular things? Is that link shareable and editable to anyone, and could it get around? If you have a website that has your code of conduct and your reporting guidelines on there, is it git backed? Can anyone have access to those git commits? Have you talked about incidents in those git commits? And so this is what I mean by threat modeling. It’s not just thinking about how online trolls are going to come into your community and break things down, it’s how your community members may find information about this that you wanna keep private.

So let’s dig into the how of how you enforce. And I really wanna say thank you to Audrey Eschright. She built a nice website, it’s called safetyfirstpdx.org, and she’s made this particular framework which I think is brilliant for thinking about code of conduct violations. So you basically put it into risk versus impact. Risk being what is the worst-case scenario that could happen, and how likely is it to happen? And impact being how many people does this affect? Is it one person? Is it a whole room full of people? One of the things to think about with impact is that people assume, you know, if it’s just an incident between two people, well, that’s a low-impact thing. However, a lot of people don’t keep incidents confidential. You should assume that people will talk to their friends and that the incident will get around, so that changes how you evaluate what the impact for that is. So I’ve put some examples here of, you know, a low-impact example and high-risk example. And then once you’ve, you know, categorized what sort of code of conduct violation this is based on this impact versus risk, you can start to look at what your response to that should be.

So, for example, if it’s something that’s low-risk, low-impact, like someone comes to you at an event and says, to the organizer, says, “Hey, this person said something, and it was really ableist, and, like, I have this, you know, invisible disability. I just don’t have the energy to tell them they shouldn’t use that word. Could you just, like, tell them to not use that word?” that’s fine. In that case, you might go to the attendee and say, “Hey, just so you know, this is a welcoming community. We have a code of conduct. You know, we want to make sure that these kinds of people participate. Please don’t use that term. Here’s other words you could use.” So you issue a warning. If someone is in your community, and they’re on, you know, the chat channels, and it just looks like they’re having a bad day, and they’re just, like, yelling at newcomers, saying they’re asking stupid questions, maybe you wanna give them a self timeout. You know, just say, “Hey, don’t show up in the user forums for a while. I can see that you’re having some issues. Go take a break.” That’s fine. If it’s something where it’s low-impact but high-risk, so it’s between two individuals, and it’s a high-risk situation, you might do things like offer an escort service.

So, you know, “Do you feel safe going back to your hotel?” You know, “We can provide…” You know, “Building security could walk you there. We can have a conference organizer that can walk you there.” Sometimes there are interpersonal issues between two people, and so you can say, “Hey, look, just don’t contact them. Like, both of you, just don’t contact them. You can be in the same area, the same event, but as long as you don’t contact them, you can stay. But if you do attempt to talk to this person that you’re having an issue with, then we will have to remove you from the event.” If it’s something that’s low-risk but high-impact, like someone made a racist joke in a keynote, then you might have to issue a public apology. You might have to say, “We won’t have this speaker back. We’re really sorry about this. We try to make sure that our events and our community is inclusive.” If it’s something like online harassment, a stalking campaign, that’s really when you need to have a coordinated effort across multiple people in your community to respond to this sort of thing.

The other thing that people don’t think about with code of conducts is where the code of conduct applies to. So most code of conducts say, “This only applies to community spaces,” which seems like it makes sense until you start digging into the details, again, one of those complex questions.

What happens when there’s a code of conduct violation that’s very egregious in a neighboring community? Maybe it’s an open source project that you build on or another product that is linked with yours, and you know that that community member is a member of both communities. You know, what happens when you know that behavior is probably gonna be repeated in your community? Or you’ve seen borderline incidences of the same behavior. Do you just ignore that behavior in the other communities until it happens to yours, or do you proactively try to make your community safer? These are hard questions. Even harder questions are around interpersonal violence. What happens when there is a sexual assault or domestic abuse between two community members? How do you handle that? It’s something that’s happened outside of your community, and, therefore, it is not under your code of conduct, but at the same time, you wanna make the person feel safe within your community. How do you deal with that? This is a hard, hard question that many communities have to face, and I would suggest you have the conversation before you have to deal with this.

Another thing is online harassment and stalking. Oftentimes the stalking or harassment will be spread across multiple different communities, spread across multiple social media sites. Do you only look at the borderline incidences within your community, or do you look at the overall picture and realize that this is harassment as is and it should be under your code of conduct? So these are the sorts of questions that are very, like, heavy, complex questions that I feel anyone that’s in developer relations and community management needs to think about, because if you don’t think about these heavy questions, you won’t be prepared when it happens.

So, as I said, we started out with some really, really simple questions, right? You know, who should be enforcing your code of conduct? Why do your customers, and your developers, and your bosses want you to have a code of conduct? And how do you enforce a code of conduct? So these are all, you know, very simple questions, but then when you start digging into them, they get more complex. There are some resources out there that can help you answer some of these questions. There’s different consultants that have experience in this that can help you answer these questions. But I really hope that people go back and they take these questions, and they’re hard and heavy questions, and have these good conversations in your community. Thank you.