Welcome Wagon: Classifying Comments on Stack Overflow

I (Jason) wrote The Stack Overflow Comment Evaluator 5000™, a simple application that presents you with a comment thread from a post on Stack Overflow and asks you to rate each comment in the thread as Fine, Unwelcoming, or Abusive.

Prevalence of comment categories

If we take a majority vote on the rating of each comment (with ties going to the worse rating) comments on Stack Overflow break down like so…

Rating % of comments
Fine 92.3%
Unwelcoming 7.4%
Abusive 0.3%

 

According to those of us deeply involved here and familiar with Stack Overflow, about 7% of comments on Stack Overflow are unwelcoming. What did some unwelcoming comments look like? These combine elements of real comments to show typical examples.

  • “This is becoming a waste of my time and you won’t listen to my advice. What are the supposed benefits of making it so much more complex?”
  • “Step 1. Do not clutter the namespace. Then get back to us.”
  • “The code you posted cannot yield this result. Please post the real code if you hope to get any help.”
  • “This error is self explanatory. You need to check…”
  • “I have already told how you can… If you can’t make it work, you are doing something wrong.”

This stuff isn’t profane, hate, or outright abuse, but it’s certainly unwelcoming. Looking at majority voting is one approach, but the experience of being not welcomed is not a majority vote kind of thing; it’s deeply personal. What if we looked at the distributions of the ratings by individual?

 

 

.. Firstly the “unwelcoming” comments aren’t unwelcoming. They’re all valid criticisms, I imagine. You haven’t given us the context in which they’re said, which’d be extremely helpful here.

The abusive comments make up 1 in 250. Which is tiny. That’s such a small amount that I find it hard to believe you’re even worrying about it. It’s -genuinely- impressive that it’s so much lower than a great deal of other websites that seek to achieve the same thing as this one. You should be extremely proud of this. It’s never going to be perfect.

I love this site and I love what it does. It’s genuinely a great platform for doing what it does. But you should work on improving the parts of it that are lacking (like chat!) before you try to handle such a minutely small problem.

 

.. The unwelcoming comments are valid criticisms, that can be expressed in a much more welcoming manner. The problem is large enough that StackOverflow is stereotyped and memed as being unwelcoming. That warrants attention, in my book.

.. I think you’re coming at this from the wrong angle. You’re thinking “let’s calculate a metric, and if that metric is below X, we don’t have a problem. Everyone who thinks there is a significant problem is wrong.”

Whereas I think the reality is: a huge number of people think there is a problem. Women in particular, who don’t contribute because they think it’s an unwelcoming place. So I’d reframe it as “let’s calculate a metric X, and now we know that at level X, we have a problem. We don’t yet know at what level we don’t have a problem anymore, but it must be less than X’.

 

.. SO claims to be a Q/A platform for professional and enthusiast programmers and for questions about programming that are tightly focused on a specific problem. So the questions must exactly tell that specific problem. This is often nothing what a beginner can do. So in my opinion SO is not a Q/A platform for beginners in programming. So this user group will always feel not welcoming here simply because it is not. Maybe SO should providing a special beginners Q/A portal additionally?

 

.. Maybe my personal bias is showing, but these 5 example “unwelcoming” comments don’t look like that to me at all. I’ve definitely gotten way harsher ones on my posts in the past and at no point did I feel them to be unwelcoming whatsoever.

And leaving these up so readers can identify the users leaving them, or even completly citing them so they can be easily found in SEDE is a privacy violation in my opinion and nobody deserves to be put in a hall of shame like that, especially not on stackexchange where the standing policy has been to allow people to correct their misgivings in private. Disappointing.

 

.. As it was stated above, those examples of unwelcoming comments above weren’t actual comments pulled from SO; they were pieced together from little bits and pieces of unwelcome comments. You can’t identify the writers of the original comments, as they weren’t written by a user. There is no privacy violation. No one has been put in a hall of shame.

Abortion, Civility, and Rule Breaking

We probably disagree on this, but as I see it, people like Putin, Duterte, and Trump are not actually restoring law and order. I don’t think that is what authoritarian leaders do, it is what they promise. Remember that Nixon was the “law and order president”. As I see it, they are making a mockery of law and order, turning it into a tool that an authoritarian leader can use to advance his own interests, hound his enemies, protect his friends, hide his own actions, and enrich himself. I am extremely concerned about law and order, I just want to make sure that it applies to the president as well. I have some examples in mind, I want to avoid giving them now simply because I want to avoid putting hot-button issues into a post at this point of the conversation. We may well disagree on my examples, I would like to find a calm way to discuss them in order to be able to discuss and learn together.

In the long run, if I’m right and this becomes obvious to people, then they may judge both Christianity and the pro-life movement as political pawns with little moral authority if we are fiercely loyal to them. I think that’s what happened in Europe as the children of Germans realized that their Christian parents had generally supported the Nazis. I think the same might well have happened in the United States if Christians of all races had not marched together against racist laws.

I think it is really important to have some moral distance from any political leader. I’m probably a lot more concerned about Trump than you are.

.. I don’t think democracy “works” and I don’t think it’s stable in the long run. I don’t really sit around worrying about “how to make democracy work”, either.

.. I guess the difference is I’m not part of the “we” of trying to make political change happen – instead, I think political change happens because God is ultimately orchestrating the affairs of men. If men don’t voluntarily choose to stop doing a grievous sin like abortion (or world wars, or the Holocaust, or all manner of other evil), God eventually intervenes. And I don’t think God is constrained by the democratic process.

.. Ending abortion, or dealing with roving drug gangs, is definitely restoring some kind of law and order. It’s not an ideal, but it’s improvement in the right direction. And I think it’s a consequence of us more progressive-minded people being so obsessed with laws and rules that we end up letting criminals get away with turning cities into war zones because we were so fixated on the rights of criminals to due process and so forth.

.. I do pray that God will intervene, but I worry about American Christians anointing a strongman who promises to beat up our enemies. I see some religious leaders treating Trump as “God’s anointed”, sent to save America from our sins. The Trump Prophecy and many things coming out of Liberty University seem to approach idolatry, as does the billboard Jeremy mentioned.

.. If I were to advise Christians, I’d tell then to follow the Bible, which necessarily means not voting, not fighting in wars, not killing people, and so on. Regardless, abortion will only be ended through force. God is in charge of using that force. Persuasion doesn’t stop murder.

.. I think abortion will not be ended by force, unless you mean the return of Christ as an act of force. Abortion pills are too easy to transport, even if they were to become illegal.

A change of heart, that is conversion (not persuasion, although that too will help), is the best way to end abortions. Meanwhile, we can reduce abortions with easy free access to birth control, free prenatal care, free birthing centers, and reducing other financial barriers to parenthood.

The Kavanaugh hustle

While Trump is destroying the honor and reputation of the presidency, Senate Republicans are doing all they can to destroy the legitimacy of the Supreme Court.

.. They want a Supreme Court that will achieve their policy objectives — on regulation, access to the ballot, social issues, the influence of money in politics and the role of corporations in our national life — no matter what citizens might prefer in the future.

.. His confirmation will be the equivalent of handing the court over to the Heritage Foundation and the legal staff of Koch Industries.

.. It is characteristic of hypocrites to be unctuous and judgmental. How else to describe the attitudes of the GOP senators rushing Kavanaugh to the bench, and their defenders in the conservative legal academy who long to undo 75 years of court precedents?

.. when Democrats on the Judiciary Committee howled about rigged hearings and did whatever they could to bring more information to light, Republicans primly accused them of lacking civility.

.. The talk of civility is laughable in light of the Republicans’ refusal to give Garland — a judge whose quality Kavanaugh himself extolled — either a hearing or a vote. That was not just incivility, it was an abuse of power reflecting the political right’s determination to seize control of the court by any means necessary.

.. But by voting to confirm Kavanaugh, they would be ratifying everything in politics they claim to be against. This is not just about their pledges to protect the right to choose on abortion. It’s also a test of whether they mean what they say about wanting politics to be less partisan and more consensual. Caving in to the power brokers and ideologues on this will follow them for the rest of their careers.

.. If the Trump era produces a backlash so strong that a Democratic president and Congress pass breakthrough economic and social policies, conservatives will count on their court majority to block, dismantle or disable progressive initiatives.

.. This is a fight about democracy itself. Right now, democracy is in danger of losing.

How Friendship and Quiet Conversations Transformed a White Nationalist

We’d heard Derek Black, the former white power heir apparent, interviewed before about his past. But never about the friendships, with other people in their twenties, that changed him. After his ideology was outed at college, one of the only orthodox Jews on campus invited Derek to Shabbat dinner. What happened over the next two years is like a roadmap for transforming some of the hardest territory of our time.

MR. STEVENSON: In fact, we — I remember, the first time that Derek was invited over, I was very explicit with people that this was not “ambush Derek” time. This was not some opportunity to yell at him for the wrongness of his beliefs, because I knew that he would — first of all, he’d spent his whole life defending this ideology. I hadn’t spent my whole life attacking the statistics and other things that they built their ideological convictions on. And as a consequence, I knew that shouting at him — or, at least, I felt that shouting at him at something like that, or having anybody else at the table do so, would just immediately put him on the defensive, and he’d never come back. So I was very explicit that people were not to discuss his background at the table, or the white nationalism, more generally.

MS. TIPPETT: But you say it in such a matter-of-fact way. I feel like right now, in our country, we forget that if we really want people to change, that <strong>it has never happened in human history that somebody changed because someone else told them how stupid they were.</strong> What was that experience like for you? You must have wondered, when you first went, are they going to grill me about this? Or, will I be put on the spot?

MR. BLACK: I think I was less worried about being grilled than what actually happened, where I wasn’t grilled…

[laughter]

…and had to spend, ultimately, years of really enjoyable time among people who — the fact that I was friends with them was contradictory to my worldview. And that was a lot more uncomfortable than had I been grilled, because I had a background doing media interviews since I was 12 years old, where people say, “How do you believe in hate?” And I had crime statistics and IQ statistics and a history of American white supremacist statements from the founding fathers, and other things like that that tend to confuse people when a 14-year-old explains to them why all the races should be separated.

And I was pretty comfortable with that position. I thought it was important, and I knew how to do it, and if it had been a big argument, I would’ve had statistics, I would have misused social science, and I would have not changed their mind and not changed my own mind, but I would’ve at least known what was going on. I think the real thing that happened, where I was just at a Shabbat dinner for two years, and I had to say, “Well, I think my ideology is very anti-Semitic.”

.. MR. STEVENSON: I think it’s also worth pointing out that over those two years, I was legitimately friends with Derek. It was not some sabotage project where I was going undercover or something. I was legitimately — felt like I was — especially, over time, counted him amongst my closest friends, even when I frankly didn’t know exactly where he stood.

MS. TIPPETT: And you weren’t asking.

MR. STEVENSON: At first, as I mentioned, I was afraid that if I were to ask that the defenses would go up, and that that would be the end of it. Later on — well, after two years, it’s a little awkward. We’d even play games, because he knew that I knew, and I knew that he knew. It was — so there would be awkward things. I screwed with him once, at this conference that he mentioned. I knew that he was doing it, he was organizing it. So I asked him, “What are you doing this weekend?”

[laughter]

He said, “I’m going to see some family.” I said, “Where?” I said, “What are you doing?” So it was a little cat-and-mouse game.

MR. BLACK: My answer was, “I’m going to a family reunion,” which was not untrue. My entire family was there.

[laughter]

MS. TIPPETT: What else was going on?

MR. BLACK: Well, it was a seminar that I had founded the year before in response to being outed at New College. I had been very uncomfortable with the fact that so many people at this college really detested what I was representing, even though I thought it was super-correct. So in response to that, the first year, I had organized this seminar up in the mountains of Tennessee, where people, where a small group would come together, and we would talk about the best ways to argue with anti-racists and to convince people that white nationalism is correct. And this was a year later, after that initial one, and I was a lot less certain of what I believed, and I was going back to it for…

MS. TIPPETT: Moral support?

.. MS. TIPPETT: So what happened is that you never made — the Shabbat dinners never became conversations about white nationalism. But then, gradually, over a period of time, in my understanding, Derek, individuals would bring something up with you, and you’d — I don’t know. I feel like you all — you handled this so well, and feels like the campus handled it well. So you would end up taking a walk with somebody, and they would say, “I really — I want to understand this,” and that started a different level of conversation.

MR. BLACK: People I met at the Shabbat dinners — in particular, one person who did the brunt of all this labor of listening to me explain this ideology and what-all is my evidence for it, and why am I so convinced this is true; and then doing the labor to say, <strong>“You are misusing crime statistics. Here’s how statistics works,”</strong> and having that sort of conversation happen sort of naturally. It was from meeting at the dinners but then being on a small campus and doing things like, “Let’s go down to the bay to watch the sunset and just spend time as people.” And eventually, it becomes sort of awkward that “We’ve never talked about that you believe in a reprehensible political ideology, and you’re advocating something terrible, and you seem kind of nice; how do you reconcile that?”

[laughter]

MS. TIPPETT: And you knew each other well enough that they could actually say it to you that way.

MR. BLACK: Yeah, because it was lower stakes than being on an interview for MSNBC or something. It was not that I had to make my points and try to get some converts; it was that I trusted this person. I liked this person. I respected this person. And I wanted to explain why I think this is correct, because “It’s clearly correct. If you don’t want to accept that it’s true, that’s a decision you can make. But it’s an uncomfortable truth.” And that was the position I was coming from.

.. you got to a point where — when you trotted out your arguments that you were so skilled in and so comfortable with, but it did actually become a conversation. You were actually able to listen to a different way of seeing even those arguments that felt so clear to you.

.. MR. BLACK: I wanted to be someone who used evidence and believed something because it was demonstrable, not because it was some gut feeling. And if the way we were using generalized IQ statistics from around the world was illegitimate because the IQ test is culturally normed, and you can’t go around the world giving it to people, and say, “Look, I’ve discovered the different intelligence of the races” — if that’s actually an illegitimate piece of evidence, I didn’t want to use it, which is why, at the time, I thought, “I’m becoming a better white nationalist. I’m becoming better at arguing this, because I now understand how these things are being misused.

And it’s only at the end, where piece after piece after piece is removed, and all I’m left with is the fact that I think that I can be friends with Jewish students and with people of color, but my belief system says that they should all be removed from the United States, and I don’t have any support for thinking that anyone is better off — all that is, is a hateful ideology.

.. you wrote this, “I would never have begun my own conversations without first experiencing clear and passionate outrage to what I believed from those I interacted with.” And yet, as we’ve been speaking, that process of you being able to interact with them and take in that outrage was the seed that got you to that point. So — but I would like to hear about how you are thinking, these days, about this line between civility and outrage and activism.

.. MR. BLACK: I worry that my story gets told as a piece of evidence that the only way that you change people’s minds is by having friendly conversations with them, when it’s clearly not true. It’s essential that you speak up loudly and condemn something that’s wrong. That’s what happened at college. It wasn’t just these conversations. The context for those conversations was that an entire community of people that I had gotten to know for a semester before they knew who I was, and who I respected, clearly had come to a very intelligent conclusion that what I was advocating was morally wrong, was factually wrong. And I found that very unpleasant, and I didn’t want to listen to it, and it initially drove me to organize a seminar to try to make white nationalists be more confident in what they were believing.

.. do you think you — without those quiet conversations, would the outrage alone have brought you around?

MR. BLACK: No; the outrage alone would have made me a more firm adherent to being a white nationalist. But the quiet conversations couldn’t have happened without the outrage.

.. And there is a difference between being aggressive and being strong. There’s a difference between being vociferously opposed, in this case, to the white nationalist ideology and other hateful ideologies, and taking steps to harm an individual who subscribes to those ideologies. Even an ideology which is as reprehensible as most of us, probably all of us in this room, believe white nationalism to be, once you cross the line to saying, “He’s forfeited his rights as a human being; he’s forfeited his right to human dignity by virtue of having those beliefs” — maybe the Nazis said that the Jews forfeited their rights to human dignity by virtue of being Jews. Where does it end? So to be strong, no question, is important. But there is a difference between being strong and violating the humanity of another person.

.. it was grounded in empathy; that the reason why I was not willing to listen to the argument that sounded very straightforward — that we should work towards inclusion, not separation — was because I didn’t empathize with people who weren’t part of my in-group. And I thought I wasn’t necessarily doing anything bad to them, but it was also, the priority was the people who were within my in-group.

And <strong>what changed was feeling that people who were not in my in-group were being negatively impacted by my actions and that I should care about that.</strong> And trying to reconcile that I should care about people who are negatively impacted by my actions, and I’m still doing the actions, became very difficult. And it really was empathizing with people who were not “supposed” to be part of my group and increasing the number of people who were in my group — that’s the universal thing that I think came out of what I learned from coming through that, because it can — everybody has in-groups.

.. what, right now, makes you despair, and where are you finding hope?

MR. STEVENSON: Sure. So I don’t think I would use the word “despair,” because I think the word “despair” makes it seem as though there is no hope. But there is certainly a tendency, I think, increasing trend to only associating with people who agree with you, who have the same worldview, have the same opinions as you. And that’s psychologically pleasing, and it’s maybe fun, but the terrible cost of that is that you run a very real risk of losing empathy for people who disagree with you. And that’s why I see people — people who are my friends, who I love dearly, think nothing to say, “I hate so-and-so.” “I hate Republicans,” or, “I hate Democrats.” Do they know what they’re saying?

As far as hope, I think that the underlying spark of goodness that’s within each and every one of us and within everybody in the world is ultimately gonna win out; that this empathy that people can generate and feel — you can’t stop it in the long run.

.. <strong>I spent a lot of time trying to be a good activist for a bad cause.</strong> And I spent a lot of time seeing the ways that my predecessors had been successful at that, whether it’s winning campaigns or building organizations in large numbers, and so, cultivated arguments that found fertile ground. And that led us to think that we were not only right, but that with time, everybody would see that we were right, and agree.

.. And now I think I’m back to being confident. <strong>People do want inclusion; they do want to make a fair society. I think just about everybody does, wants there to be a society where we are not limited, where we’re not oppressed because of our group. And it’s just very hard to do that.</strong>