US Special Ops Wants To Use Deepfakes To Conduct ‘Propaganda And Deception’ Campaigns

US Special Operations Command (SOCOM) wants to use deepfake videos to conduct propaganda and deception campaigns online, according to federal contracting documents reviewed by The Intercept.

While the U.S. government routinely warns against the risk of deepfakes and is openly working to build tools to counter them, the document from Special Operations Command, or SOCOM, represents a nearly unprecedented instance of the American government — or any government — openly signaling its desire to use the highly controversial technology offensively.

SOCOM’s next generation propaganda aspirations are outlined in a procurement document that lists capabilities it’s seeking for the near future and soliciting pitches from outside parties that believe they’re able to build them. -The Intercept

The plans also include hacking internet-connected devices to spy on foreign populations and assess their susceptibility to propaganda. The document describes it as a “next generation capability to ‘takeover’ Internet of Things (loT) devices for collect [sic] data and information from local populaces to enable breakdown of what messaging might be popular and accepted through sifting of data once received,” and that the ability to eavesdrop on propaganda targets “would enable MISO to craft and promote messages that may be more readily received by local populace.”

“When it comes to disinformation, the Pentagon should not be fighting fire with fire,” said Brookings Institution head of AI and Emerging Technology Initiative, Chris Meserole. “At a time when digital propaganda is on the rise globally, the U.S. should be doing everything it can to strengthen democracy by building support for shared notions of truth and reality. Deepfakes do the opposite. By casting doubt on the credibility of all content and information, whether real or synthetic, they ultimately erode the foundation of democracy itself.“

“If deepfakes are going to be leveraged for targeted military and intelligence operations, then their use needs to be subject to review and oversight,” he added.

In addition to deepfakes and hacking, SOCOM’s Directorate of Science and Technology laid out a wish list of next-generation toys to equip the modern commando and enable them to more effectively hunt and kill targets using lasers, holographs, robots and other high-tech hardware.

Last October they updated this wish list with a new section; “Advanced technologies for use in Military Information Support Operations (MISO),” aka propaganda and deception, which encompass “influence operations, digital deception, communication disruption, and disinformation campaigns at the tactical edge and operational levels.”

SOCOM also wants “a next generation capability to collect disparate data through public and open source information streams such as social media, local media, etc. to enable MISO to craft and direct influence operations.”

Though Special Operations Command has for years coordinated foreign “influence operations,” these deception campaigns have come under renewed scrutiny. In December, The Intercept reported that SOCOM had convinced Twitter, in violation of its internal policies, to permit a network of sham accounts that spread phony news items of dubious accuracy, including a claim that the Iranian government was stealing the organs of Afghan civilians. Though the Twitter-based propaganda offensive didn’t use deepfakes, researchers found that Pentagon contractors employed machine learning-generated avatars to lend the fake accounts a degree of realism.

Provocatively, the updated capability document reveals that SOCOM wants to boost these internet deception efforts with the use of “next generation” deepfake videos, an increasingly effective method of generating lifelike digital video forgeries using machine learning. Special forces would use this faked footage to “generate messages and influence operations via non-traditional channels,” the document adds. -The Intercept

This, after the US has spent years warning against the potential national security threat posed by deepfakes, which could have a ‘destabilizing effect’ on civilian populations.

Do as we say, not as we do?