Big tech companies back away from selling facial recognition to police. That’s progress. (original) (raw)

Microsoft president Brad Smith announced on Thursday that his company did not sell facial recognition to the police, and would not until the government passes federal legislation regulating the technology. His statement follows a Wednesday announcement from Amazon explaining that the company would institute a one-year moratorium on police use of Rekognition, the company’s facial recognition software.

That change also comes after IBM’s announcement earlier this week that the company would no longer offer facial recognition products, citing the technology’s potential for abuse or misuse. As scrutiny of law enforcement ramps up, it looks as though there could be a referendum on the surveillance tools police can access, though it’s notoriously difficult to determine what surveillance systems police departments actually have. Law enforcement agencies and tech companies are typically secretive about these products, which are by no means limited to just facial recognition.

Still, this is certainly a sign that the needle of public acceptance of facial recognition is moving. For years, advocacy groups like the American Civil Liberties Union (ACLU) and the digital rights group Fight for the Future have sounded the alarm that facial recognition raises significant privacy and surveillance concerns. And researchers like Joy Buolamwini, Timnit Gebru, and Deborah Raji have demonstrated that these technologies can come with concerning built-in racial and gender biases.

The timing of these facial recognition announcements is important. The recent scrutiny of the technology comes amid ongoing anti-police brutality demonstrations, which have drawn increased attention to the surveillance tools large technology companies can and do sell to law enforcement. The news also comes as lawmakers have zeroed in on law enforcement’s access to surveillance tools and their potential use on protests, which was the subject of a letter sent by members of Congress to leaders of several federal agencies earlier this week. Now, amid several big companies making announcements about facial recognition, it’s possible lawmakers will be more anxious to regulate than before.

Amazon’s Rekognition has faced criticism and opposition for years

Launched in 2016, Amazon’s Rekognition software is part of the company’s cloud computing division, Amazon Web Services (AWS). On its company’s website, Amazon claims that its facial recognition “provides highly accurate facial analysis and facial search capabilities that you can use to detect, analyze, and compare faces for a wide variety of user verification, people counting, and public safety use cases.” Since its debut, the company has tried to push the technology to a whole host of law enforcement entities; it has been tested by police departments in Oregon and Orlando, Florida.

Amazon’s Rekognition technology specifically has been called out by critics who have shown how it can be rife with racial and gender bias. Despite the company’s initial pushback on the work of Buolamwini and Raji, Amazon is now reconsidering its plan for Rekognition.

“We’re implementing a one-year moratorium on police use of Amazon’s facial recognition technology,” the company now says. “We’ve advocated that governments should put in place stronger regulations to govern the ethical use of facial recognition technology, and in recent days, Congress appears ready to take on this challenge. We hope this one-year moratorium might give Congress enough time to implement appropriate rules, and we stand ready to help if requested.”

Notably, Amazon carved out exceptions to the moratorium. The company said it will continue to allow organizations that fight human trafficking and find missing children to use the technology. Amazon declined to comment further on the reasoning behind the moratorium, any plans to end its police partnerships with Ring, or non-police law enforcement access to its technology.

Today, the exact details of how Rekognition is being used by law enforcement remain unclear. AWS chief executive Andy Jassy told the press earlier this year that the company doesn’t know the number of departments that have used Rekognition or how they’re using it.

The technology and its use by law enforcement have received significant pushback from civil liberties advocates and researchers. That concern has roots in research published in 2018 by Buolamwini and Gebru, highlighting the extent to which facial recognition systems could have racial and gender biases, which was based on earlier work by Buolamwini. That research provided a foundation for research conducted by the ACLU, which found that Amazon’s Rekognition incorrectly matched the faces of 28 lawmakers with the faces of people who had been arrested. The organization called on Congress to put a moratorium on the sale of the technology to law enforcement.

Then, in a follow-up report, Buolamwini and Raji studied bias in Rekognition and found that Amazon’s system also had significant racial and gender bias and that it performed worst on women with darker skin. Amazon published two company blog posts pushing back against their results, which prompted a response from nearly 80 researchers, who rallied to defend Buolamwini and Raji’s research and called on Amazon to stop selling the technology to law enforcement.

Other civil liberties and rights organizations have also protested the company’s sales to law enforcement. Last year, activists from Fight for the Future traveled to Capitol Hill to conduct live facial recognition on government representatives, demonstrating how invasive the Rekognition technology could be. Amazon employees also spoke out against the company’s sale of facial recognition to law enforcement — along with its sale of AWS to Palantir, which works with Immigration and Customs Enforcement — in 2018, though the company did not accede to their request to stop selling Rekognition to police. It’s not yet clear what role, if any, these Amazon employees played in this most recent decision.

Not everyone is happy with how Amazon is addressing issues with Rekognition. Evan Greer, the deputy director of Fight for the Future, called Amazon’s announcement “a public relations stunt.” The Surveillance Technology Oversight Project, a New York-based anti-surveillance legal organization, called the company’s move “too little, too late.” Its director, Albert Fox Cahn, added in an emailed statement, “Amazon shouldn’t just end this practice for one year or one decade; it should end it forever.”

Now eyes are on Congress — and on facial recognition companies you’ve never heard of

The call for regulation shouldn’t be surprising. There is currently no comprehensive federal law governing facial recognition, and some localities have taken up the task themselves of regulating the technology. Amazon has previously floated its own recommendations for how it believes facial recognition ought to be regulated, and like many other facial recognition providers, the company also wants to avoid patchwork regulation that means different laws exist in different states. Meanwhile, critics of facial recognition technology have warned that corporate calls for regulation should be met with skepticism, as companies can push laws that are weak and ultimately defend their interests. Remember that these companies are known to have extensive budgets for lobbying.

There’s also concern that as large technology companies like IBM, Amazon, and Microsoft turn away from providing the technology temporarily, smaller providers of facial recognition will step in to take their place. Two providers of this technology, Vigilant Solutions and NEC, but did not hear back by the time of publication. Following publication, NEC America told Recode in a statement that it supported public safety agencies’ use of facial recognition.

In a statement shared with Recode, Hoan Ton-That, the founder and chief executive of Clearview AI, said that his company’s technology “is not intended to be used as a surveillance tool relating to protests or under any other circumstances” and that he looks forward “to working with government and policymakers to help develop appropriate protocols for the proper use of facial recognition.”

As Recode has previously reported, however, Clearview has repeatedly misrepresented various aspects of its work, including how well the technology works and who its clients are.

Still, that three companies as large as Amazon, IBM, and Microsoft are pressing pause on the use of this technology, however temporarily, is notable and could be a sign that norms surrounding facial recognition are changing. Eyes will now likely turn to other companies that sell the tech, as well as to federal lawmakers, who have now been given an informal deadline by one of the world’s largest companies.

“Their products have the alarming potential to infringe on Americans’ privacy rights in ways that we would have thought unimaginable not long ago,” Sen. Edward Markey (D-MA) said after Amazon’s announcement. “Pressing pause on the use of this technology by law enforcement is a positive step, but what Amazon should really do is a complete about-face and get out of the business of dangerous surveillance altogether.”

That differs from other ideas from legislators about what the future of federal facial recognition legislation should be. For instance, earlier this year, Sen. Jeff Merkley (D-OR) told Recode that the technology should be usable when law enforcement is able to obtain a warrant and seemed to draw a distinction between facial recognition that could be used for constant surveillance and facial recognition that would be used during an investigation. Some federal lawmakers have also previously proposed limiting facial recognition in federally assisted housing.

What that regulation will ultimately look like remains unclear. The fact that Amazon, at least, is only pausing its facial recognition business — Microsoft seems to be just waiting until legislation is in place before moving forward — seems to imply that some companies assume a ban on the technology’s use by police is not on the table. With or without regulation, many organizers still warn that facial recognition might simply be too dangerous to use in the first place.

Update, June 11, 4 pm ET: This piece has been updated to include more context and the news that Microsoft would not offer facial recognition to police until the passage of federal legislation regulating facial recognition.

Update, June 12, 10 am ET: This piece has been updated to note a statement from NEC America.

Open Sourced is made possible by Omidyar Network. All Open Sourced content is editorially independent and produced by our journalists.