Awful people will use technology to do awful things. This is a universal truth that applies to almost any technology that facilitates communication and interaction, no matter how well intentioned it might be. Something as innocuous as Google Drive can be a vector for harassment. As we’ve recently discovered, so can video conference platforms like Zoom. Just in the past few weeks, high school classes in North Carolina and Texas, along with an NAACP meeting in California, were interrupted by racist and misogynist video, images, and text. With remote classes again ramping up all over the country, we can only expect more harm—but how much is Zoom to blame?
Last April, “Zoombombings” hit our university, and a colleague described the disturbing disruption to her online classroom, where trolls got around Zoom’s poor privacy protocols in order to screen-share pornography and scream racist and sexist slurs. Even obvious precautions, like not posting public links to meetings, are vulnerable to social engineering such as university students posting links to “come zoom bomb my class” forums. As tech ethics researchers, we did not find this surprising. However, apparently it was to Zoom’s CEO, who told The New York Times, “The risks, the misuse, we never thought about that.”
Big Tech is all about speed, especially when there is a perceived opportunity, like a pandemic forcing greater reliance on communication technology. But a “move fast and break things” mentality results in limited testing and deployment of software that isn’t ready. This is such a known problem that there’s even a term for it: “technical debt,” the unpaid cost of deploying software that will ultimately need to be fixed after it’s clear what the bugs are.
Debt accrues when these issues are not tackled during the design process. When the bugs are societal harms, however, it isn’t seen as bad tech, but rather unethical tech. “We never thought about misuse” is the precursor to another kind of debt: ethical debt.
Zoom’s “awful people” problem isn’t your typical bug, after all. When the “we’ll fix bad things after they happen” approach is about potential harms, whether individual or societal, you are failing to anticipate ethical issues. And the problem with ethical debt is that the metaphorical debt collector comes only after harm has been inflicted. You can’t go back in time and improve privacy features so that unsuspecting marginalized students didn’t hear those racial slurs in the middle of class. You can’t reverse an election after the spread of disinformation undermined democracy. You can’t undo an interrogation and improper arrest of a Black man after a biased facial recognition accusation. You can’t make people un-see conspiracy theory videos that a recommendation algorithm shoved in their faces. The harm has already been done.
Technologists can’t see the future, but they can predict and speculate. They know that awful people exist. At this point, they can easily imagine the ones who might intentionally spread conspiracy theories, who might rely on facial recognition as evidence even when they’re told not to, who might try to manipulate elections with disinformation, and that might think it’s fun to terrorize unsuspecting college students and professors. These aren’t all splashy headlines, but can also be micro-instances of individual harm that accumulate over time. As part of the design process, you should be imagining all of the misuses of your technology. And then you should design to make those misuses more difficult.
Ironically, some of the very best people to imagine things like how technology might be used for harassment are people who are often harassed. This means marginalized and vulnerable people like women and people of color—people who are underrepresented in tech. In a room of these folks, we guarantee you that “random people will jump into Zoom meetings and screen share pornography” would come up during speculation about misuse. Because many technology-based harms impact already marginalized people disproportionately, these are important voices to include in the design process as part of addressing ethical debt.
Technologists often create “user personas” during the design process to imagine how different types of people might use that technology. If those personas don't include "user stalking their ex,” "user who wants to traumatize vulnerable people," and "user who thinks it's funny to show everyone their genitals,” then you're missing an important design step. And if your response to this is, “Yes, there are likely to be these kinds of problems, but we’ll fix them after we know what they are,” start keeping an accounting book of your ethical debt.
Unfortunately, until we can count on tech companies to think ahead in this way, it is important that end users understand potential harms and take measures to protect themselves. I (Casey) narrowly thwarted an organized Zoombombing attempt in my undergraduate tech ethics class that followed my posting a YouTube video that called out Zoombombers, only because I had a waiting room set up as a security precaution. Whether you are teaching, holding meetings, or officiating a wedding, you should take precautions to protect your meetings against bad actors even when those precautions might be non-obvious or not well-designed. Fortunately, following the proliferation of Zoombombing in the spring, Zoom made some of these settings into defaults in addition to improving encryption. Though if these measures had been proactive instead of reactive, we might have avoided some harm.
The problem with debt is that it has to be paid back—with interest. For technical debt, this means that changes are harder and harder to implement once the tech is out in the world. The same is true for ethical debt, but with the increasingly high stakes of technology so deeply embedded in our lives, it’s the rest of us who are paying the price.
WIRED Opinion publishes articles by outside contributors representing a wide range of viewpoints. Read more opinions here, and see our submission guidelines here. Submit an op-ed at opinion@wired.com.
- 📩 Want the latest on tech, science, and more? Sign up for our newsletters!
- Gravity, gizmos, and a grand theory of interstellar travel
- Meet this year’s WIRED25: People who are making things better
- Dungeons & Dragons TikTok is Gen Z at its most wholesome
- You have a million tabs open. Here’s how to manage them
- The blurred lines and closed loops of Google Search
- 🏃🏽♀️ Want the best tools to get healthy? Check out our Gear team’s picks for the best fitness trackers, running gear (including shoes and socks), and best headphones