500m posts are made daily on Twitter alone. Policing them is no easy task.
Research Fellow in Informatics, University of Sussex
July 27, 2016
Leslie Jones, the actress and comedian who plays Patty Tolan in the all-female reboot of Ghostbusters, has become the latest celebrity on Twitter to be subjected to torrents of abuse. Her ordeal is yet another in a long list of people, overwhelmingly women, who have been abused online. Yet again attention has turned to focus on what steps Twitter is taking to tackle abusive trolling.
Sifting abusive posts from the roughly 500m sent per day on Twitter alone is quite a task. The hope is that the computers and software underpinning the platforms that allow us to communicate, share information and have information recommended to us based on our preferences will also help us keep the trolls at bay. Over the past 20 years, increasingly sophisticated algorithms have been developed to identify unwanted and abusive messages. Originally developed to combat spam email, the same techniques are being deployed against abusive messages.
These pattern-detecting algorithms are often referred to as “machine learning” because the software is fed examples from which it learns to identify similar messages. During a training phase the machine learns which features of any text are undesirable (however that is defined) and which features are acceptable. A piece of text will typically contain a set of such features. Once the machine has been trained, it is able to form a judgement when shown a new piece of text based on the accumulated evidence it has seen before.
These algorithms can be used to filter unwanted messages before we see them, and before they cause hurt or alarm. The hard part is defining the underlying learning model, figuring out what the features to identify should be, and determining how that evidence is to be accumulated and scored. How tricky this can be was excellently demonstrated by Microsoft, whose Twitter chatbot went rogue after pranksters subverted the machine learning process by teaching it to swear and make abusive remarks.
There are a number of approaches available to developers. This is a very active research area as machine-learning algorithms can be applied across many types of tasks, from helping computer vision make sense of what it’s seeing, to assisting robotic movements or bulk text analysis.
Commonly used approaches include Naive Bayes classifiers, logistic regression classifiers, perceptrons, support vector machines, and various kinds of multi-layered neural networks (MLNNs). MLNNs are causing the most excitement at present, not least because they roughly simulate the way in which real-life brains might be working, but also because successfully trained neural nets presently are the highest performers for many of these tasks. However, all of these approaches have their niches that depend on the precise task, the amount of training data available for learning, the time available for training, and the hardware requirements.
When it comes to intercepting trolls, machines currently can do a reasonably good job at spotting the obvious. They can spot typical patterns of insult. They might even do a reasonable job at differentiating between, for example, a message containing an out-and-out racial insult and a message that employs racial slur words in an (arguably) benign manner. For example, many short messages contain racial slur words that are being used not as an insult but as slang terms used by members within a particular social group.
But language use is subtle, and the current generation of systems struggle to identify sarcasm, reading-between-the-lines implication, or some other linguistic sleight of hand. The problem is that machines are simple-minded, based only on the form of words rather than on their underlying meanings (however they are defined).
Nonetheless, given the current state-of-the-art why is there still so much blatant abuse online? Ultimately, this is down to the choices we make rather than limitations of technology. While the virtual spaces we inhabit feel public, all are in fact someone else’s private virtual real estate. The computer servers upon which websites reside might be anywhere in the world, with what is allowed defined by legal contexts that differ depending on the country. Within those quite broad limits, the owner may define further restrictions. Ultimately, some site operators feel the benefits of free speech outweigh the benefits of a life free from abuse.
Of course, to remain popular the owners of these services need to be mindful of their users’ views, but for many domains only the tolerant and thick skinned need apply. The racist remarks, harassment and abuse directed towards Leslie Jones demonstrate how social media can degenerate into an echo chamber of hateful outbursts.
Ultimately, we need to make choices about how we behave to each other online. In the meantime, technologists will continue to develop algorithms and gradually improve machines’ ability to analyse, filter, and protect us from some of the worst aspects of ourselves.
Funny, sad, interesting, a glimpse of a new character in our internet culture: the fake person, the new agent provocateur, the troll.
You know you’ll have to at some point.
By Seb FoxAllen
He is hungry. Image via Flickr user Jan Hammershaug
Over the past few days we’ve been trying to find a better way forward from the acrid pile of burning trash that was 2015. We’ve dealt with the environment, terrorism, drug-taking, and more, but perhaps the area with the most room for achievable improvement is in the dank wasteland of the internet.
In 2015 your online friends grew more and more insufferable. All news was bad and the coverage of it worse. People you like argued ad nauseum, and ungodly promoted content took over your carefully curated Instagram feeds. One of your exes got married (lol); another got gout (lol?).
If you were a woman, an incessant stream of dudes yelled sexist shit at you from behind Twitter eggs and Solid Snake avatars. Anonymous threats of sexual violence weren’t so much lobbed in your direction as skillfully targeted at you and your platforms.
If you weren’t white, dismissals of your experiences clogged up your notifications even when you were just trying to crowdsource a substitute for honey in your homemade energy bar recipe.
If you identify as anything less heteronormative than Tim Allen, your very personhood was up for constant debate from people you didn’t know and, more jarringly, in small, subtle ways from some you thought you did.
If you belong to more than one of those groups, you might be commended for even mustering the courage to hang out online this year at all.
Will 2016 be better? Who knows! There is certainly a chance that it will be even worse. But here are a few things that could, if even fleetingly, start to make being online a bit more bearable over the next 12 months.
Trolling is a magical and valuable tactic of the internet. It uses the language and conventions of a group to coax out hypocrisy or frustration from its members. It catches the status quo in all its pomp and silliness. But trolling is also a word we use to minimize the impact of online behavior that would be criminal if it happened on the street.
Harassment online—whether it’s threats of violence, or persistent unwanted contact—is harassment, and should be described that way. Whether laws are currently equipped to deal with it or not, harassment on the internet has consistently led to offline consequences.
Language is never the be all and end all, but not trivializing harassment that happens on the internet as “trolling” (or, say, “cyberbullying”) helps people who are targeted feel heard and believed. That’s a small, good thing.
Trolls aren’t inherently bad and neither is trolling, so long as they punch up at power and privilege.
The most famous troll of 2015 was Donald Trump. His campaign for the Republican nomination for president has consistently (and sometimes spectacularly) frustrated and embarrassed the big-spending political establishment that is used to picking presidents. He engages the political system with such transparent contempt that he has proven incredibly difficult to pander against, forcing both his opponents and the Republican National Committee to respond to his silliness with a seriousness that ends up reading as even sillier.
Better examples are the South Carolina State Representative who introduced a bill forcing doctors prescribing Viagra to jump through the same ridiculous hoops that her colleagues had legislated for abortion procedures, the Australians using an iMessage loophole to hassle politicians over new cybersecurity laws, or that fake Campbell’s Soup customer service account that made fun of homophobes threatening to boycott the company on Facebook.
The difference between being a troll and being a dick all comes down to who you set your sights on.
Being told “Don’t feed the trolls” is about as useful as being told “Just get over it.” Some people process online harassment by shrugging it off, others need to hold it up to the world and light it on fire. Most use a combination of the two, but either way it comes with new consequences that obviously shouldn’t be necessary in the first place.
Feed trolls when it works for you, don’t engage with them when it doesn’t. But don’t accept that either response means you, or someone else, is asking to be targeted.
The internet, or at least the parts of it most of us use, isn’t a public space.
Hanging out on Twitter isn’t like standing on a street corner, it’s like standing in a McDonald’s. It’s a corporately-governed space that can impose rules and restrictions on what its users can say or do.
Social media companies have struggled to find meaningful ways to juggle free speech and user safety on their platforms. Twitter in particular has faced years of criticism for its unwillingness to adopt clear, useful mechanisms to protect users from serial harassment.
The same goes for other types of platforms where people are subjected to online harassment, from personal websites, to comment sections, to email clients.
For 2016, Twitter introduced changes that more concretely shape what kind of language it will allow, tackling both personal abuse and more general speech from places like the estimated 50,000 accounts linked to the Islamic State. It’s a step in the right direction, but functions like better blocking and reporting may need more attention before users really have the agency they need to use the platform safely.
What should be clear is that restrictions platforms like Twitter seem increasingly willing to entertain aren’t an affront to free speech, but rather a clear system of consequences to violating the rules its users opt in to. If we’re going to live in a corporate internet, we can insist that platforms keep up with what we want from them.
Comments are bad. All of them. They are peep shows of morbid curiosity at best, and magnets for the most pompous type of ideological grandstanding at worst.
If you scroll to the bottom of this article, you are likely to find two types of comments: (a) those you already completely agree with; and (b) those you would absolutely never agree with. They’re relics from a time when there was a genuine lack of places someone could publicly express an opinion on the news of the day. That isn’t the case anymore.
Canada’s public broadcaster closed out 2015 by announcing it would disable comments on articles about Indigenous people, which had for years attracted only the worst types of colonial backwash. In the US, major publishers like Bloomberg, The Verge, and VICE’s very own Motherboard all dropped their comments sections last year.
If someone feels a need to add comment—constructive or otherwise—to an article on your website, they have adequate means to do so across thousands of public channels. “Don’t read the comments” was 2014. In 2016, more publishers will choose not to host them.
We get it. You’re callous. You’re brash. You take no prisoners. You understand evolution and free markets and Serena Williams’s sinister agenda better than anyone. You’ve memorized all the most scathing Richard Dawkins quotes, and what you’ve read about ExxonMobil would throw people into open revolt if they only knew.
You’re a rationalist: people should be able to explain anything they say using the square equivalencies of Reason, anytime you ask them to, regardless of whether they’re busy, or don’t know you, or just don’t want to. It’s all about a free exchange of ideas, can’t everyone just realize that?
After all, you’re just asking questions. Like seven of them, one after the other. Maybe the last one was less a question and more a suggestion that that person who tweeted about salted vs. unsalted butter is a dumb bitch who probably doesn’t even like butter. All you wanted was for them to acknowledge that unsalted butter is clearly more versatile because you can always just add salt to it. It’s just logical: but they couldn’t confront the truth.
You’re no hero. You’re just acting out an age-old performance of power, maybe one you feel less and less comfortable doing out in the world these days. Those tiny cracks the Social Justice Warriors and PC Police keep carving into patriarchy and white supremacy seem like chasms from where you sit, jagged expanding holes in the way the world is, or was, or should be.
The internet of 2015, like the internet you’re reading this on today, carried all the same awfulness and injustice that persists offline, except instantly searchable and pinging you 24/7.
But you need to use it. To work, to communicate, to lulz, to connect.
Maybe all you can really do to wade through it are the same things you’ve learned to do offline: keep networks that you trust, pick fights that are worth your time and try to sidestep those that aren’t. Do what you have to do to feel safe, support others when they don’t, and keep chipping away at the structures that make any of this necessary at all.
2016, as ever.
Follow Seb FoxAllen on Twitter.