Pentagon Developing Deepfakes to Deceive the Public

United States Special Operations Command (SOCOM) is “gearing up to conduct internet propaganda and deception campaigns online using deepfake videos,” according to contracts with the federal government reviewed by The Intercept.

In what many would attribute to the likely behavior of rogue regimes targeting the United States, the activities that SOCOM is carrying on overseas include “hacking internet-connected devices to eavesdrop in order to assess foreign populations’ susceptibility to propaganda,” the Intercept article reports.

The information revealed in the report is taken from a procurement document published by the Department of Defense, a sort of wish list of technological tools the Pentagon is looking to secretly deploy throughout the world.

Of particular interest on the list is a section called “Advanced technologies for use in Military Information Support Operations (MISO),” interpreted by the The Intercept as “a Pentagon euphemism for its global propaganda and deception efforts.” Here’s how The Intercept described the contents of that disturbing part of the procurement request:

The added paragraph spells out SOCOM’s desire to obtain new and improved means of carrying out “influence operations, digital deception, communication disruption, and disinformation campaigns at the tactical edge and operational levels.” SOCOM is seeking “a next generation capability to collect disparate data through public and open source information streams such as social media, local media, etc. to enable MISO to craft and direct influence operations.”

While you’d be surprised to see SOCOM — an organization comprised of elite military units renowned for their ability to work secretly and under the cover of darkness — allowing its disinformation designs to be obtained and publicized by The Intercept, the Pentagon has been hiding it in plain sight for years now.

In December, The Intercept revealed some very troubling tactics used by SOCOM to manipulate social media:

SOCOM had convinced Twitter, in violation of its internal policies, to permit a network of sham accounts that spread phony news items of dubious accuracy, including a claim that the Iranian government was stealing the organs of Afghan civilians. Though the Twitter-based propaganda offensive didn’t use deepfakes, researchers found that Pentagon contractors employed machine learning-generated avatars to lend the fake accounts a degree of realism.

So, the government of the United States, while decrying “fake news,” was itself creating fake news to foist on people turning to Twitter for unfiltered news.

Just so it’s clear and there’s no misplaced worry that the document is somehow less sinister than The Intercept’s depiction of it, here’s one a paragraph from the “Advanced technologies for use in Military Information Support Operations (MISO)” section that should remove all doubt about the purpose for the procurement. MISO will seek for technologies to:

influence operations, digital deception, communication disruption, and disinformation campaigns at the tactical edge and operational levels … seeking a next generation capability to collect disparate data through public and open source information streams such as social media, local media, etc. to enable MISO to craft and direct influence operations.

And these few paragraphs from the document are no less unnerving:

Provide a next generation capability to collect disparate data through public and open source information streams such as social media, local media, etc. to enable MISO to craft and direct influence operations and messages in relevant peer/near peer environments 

Provide a next generation of “deep fake” or other similar technology to generate messages and influence operations via non-traditional channels in relevant peer/near peer environments

Generate next generation capability to “takeover” Internet of Things (IoT) devices for collect data and information from local populaces to enable breakdown of what messaging might be popular and accepted through sifting of data once received. This would enable MISO to craft and promote messages that may be more readily received by local populace in relevant peer/near peer environments. 

Those are the goals of our own federal government. The leaders of our armed forces are seeking partners who can provide the Pentagon with the tools to carry out missions of mass deception using deepfakes and collect data from internet-connected devices used in homes, using the conversations recorded to craft messages — false messages — that the people being secretly monitored will easily accept as accurate.

The scope of the damage done by these deepfakes is familiar to the U.S. government. We know this for certain because of warnings issued by our own director of national intelligence (DNI) against such activities being carried on by foreign regimes. Here are a few selections from a DNI document entitled “Safeguarding Our Elections:”

Threat:

Foreign adversaries may use artificial intelligence to create deepfakes (high quality generated or manipulated video, images, text, or audio) to shape public opinion and influence U.S. elections

Impact: 

Undermines public confidence in political candidates and parties

Undermines public confidence in the election process

Threatens civil discourse and divides the American public

Challenges the ability of online and social media platforms to identify and manage digitally-altered content

Creates or amplifies foreign adversaries’ false narratives

In light of the plans exposed by the document obtained by The Intercept, one wonders whether this DNI presentation was less of a warning and more of a template.

At the end of the day, there is absolutely no constitutional authority for the federal government to carry out such missions of mass deception, particularly when the United States is not at war. 

Finally, consider carefully the warning given in 1815 by French philosopher Benjamin Constant, concerning what happens when a people place such immense power over the the press and public perception in the hands of government:

By authorizing the government to deal ruthlessly with whatever opinions there may be, you are giving it the right to interpret thought, to make inductions, in a nutshell to reason and to put its reasoning in the place of the facts which ought to be the sole basis for government counteraction. 

This is to establish despotism with a free hand. Which opinion cannot draw down a punishment on its author? You give the government a free hand for evildoing, provided that it is careful to engage in evil thinking. You will never escape from this circle. 

The men to whom you entrust the right to judge opinions are quite as susceptible as others to being misled or corrupted, and the arbitrary power which you will have invested in them can be used against the most necessary truths as well as the most fatal errors.