The Terrifying Truth About The Evolution Of Deep Fakes

In 1917, not long after the introduction of personal cameras, the world was rocked by the publication of photographs purported to show young children frolicking with fairies. The Cottingley Fairies, as they came to be known, were deemed legitimate by a number of experts and were popularized by none other than Arthur Conan Doyle. Decades later, the children in the photos, now adults, admitted they were fakes made with paper cutouts. Whether for art, fraud, or a fanciful prank that got out of hand, we've been doctoring images to paint a false picture of reality for almost as long as we've been able.

Our ability and tendency to create imagery from our imagination has lent itself to thriving television and movie industries that have utilized practical effects, camera trickery, and post-production modifications. As our imaging technologies became increasingly digital, that opened up a new arena for falsifying images and those tools are increasingly available to the wider public. As the ability to create false images propagated, society was bound for a little heartache.

Now, through the use of sophisticated yet readily available software, nearly anyone with the time and talent can create fabricated images or videos of almost anyone else. This has some concerned that someday soon — maybe even already — we won't be able to trust what we see and hear online.

There are all kinds of artificial intelligence, each one specialized for its particular type of work. Some are good at recognizing speech and returning relevant responses. Others are good at recognizing the environment and navigating through it. These kinds of AI lean on existing human knowledge or creation to replicate the behavior of people to a reasonable degree. The next obvious step was creating AI capable of creating new things.

In 2014, Ian Goodfellow and co-authors proposed a new framework for training generative AI. Their system, called Generative Adversarial Networks, or GAN, was published on the Arxiv preprint server. The premise of their neural network is relatively intuitive. It works by pitting two computer models against one another. The first model generates whatever you tell it to. It could be letters and numbers, or it could be human faces. The second model decides whether it believes the image generated is unique or if it is a real image taken from the training data.

If the discriminating model succeeds in identifying the generating model's creation, then the generating model goes back to the drawing board. It iterates and tries again. The adversarial nature of the paired computer models causes a creative arms race, constantly driving the generative half toward more convincing forgeries. The ability to train on real images and then generate convincing facsimiles would prove invaluable as the deep fake emerged.

In modern parlance, the term "deep fake" has taken on a life of its own. Many of us use the term without ever considering where it came from. To that question, there are really two answers. The first is simple and etymological. The name refers to a portmanteau, taking "deep" from the deep learning algorithms involved and "fake" for the obvious reason (via Grammarist). Historically, it has another origin.

In 2017, a Reddit user with the handle Deepfakes started uploading modified pornographic videos ostensibly featuring well-known celebrities. It was perhaps a natural and even expected outgrowth of the established practice of photoshopping celebrity faces onto still pornographic images but that doesn't mean anyone was prepared for it.

According to Vice, Deepfakes creations were a hit in certain subreddits and the resulting media attention helped to propel the notion of deep fakes into the public eye. In a conversation with Vice, Deepfakes noted that they used publicly available images found on Google, stock photo repositories, and YouTube videos. They then trained the AI on the target media, in this case, pornographic videos and images of the chosen celebrity. According to Deepfakes, the system worked by recognizing the target face and treating other images as distortions of that face. Once the algorithm had a good handle on the desired components, it was able to map the face of a celebrity onto the body of an adult film star, to everyone's amazement and horror.

In December of 2018, researchers from Germany and the United States revealed Face2Face in a paper published in the journal Communications of the ACM. In it, they describe and demonstrate a process which uses facial capture to modify the expression and mouth movements of a video.

Face2Face uses two video streams to achieve its effect. The first is the target video you want to change, the second is a video of you or an actor acting out the changes you want. According to the authors, the target video can be taken from any public source, like YouTube, and the second video can be captured using a webcam.

Once you have your camera on, the system tracks your facial movements and maps them onto the target video through deformation transfer. If the actor does a decent impression of the target's voice, that's all you need, but you don't even really need that. Coupled with Adobe's VoCo, creators can create audio of a person saying whatever they want, then match it up with a faked Face2Face video, allowing you to control the movements and speech of someone else like a puppet.

There are obvious concerns about how these technologies might be used in concert — not to mention how they're already being used — to create wholly fictional yet totally convincing videos. However, there are also educational and entertainment applications which could utilize these technologies in a less morally polluted way.

To see how that's being done, we'll have to go back in our minds to the peak of the Apollo program. In 1969, as Neil Armstrong, Buzz Aldrin, and Michael Collins made the perilous journey to the Moon, the United States political leaders were preparing for two potential eventualities. In one hand, then President Nixon carried the speech we all heard, one which told of a successful mission to the Moon. Though there was another speech, one which informed the world of failure and tragedy far from home (via MIT).

The folks behind "In Event of Moon Disaster" wanted to illustrate how the world might have looked had Apollo XI ended in tragedy, while also blatantly telling the viewer they're watching something fictional. This eight-minute short film takes viewers into an alternate history through the power of deepfakes.

To pull it off, an actor read the speech Nixon never gave, in order to get the right cadence. Then video and audio manipulation were used in concert such that Nixon appears to be giving the speech, despite half a century having passed.

For now, the risk of becoming the subject of a deep fake is largely isolated to political figures and celebrities. While there is some risk to the degradation of our ability to trust what we see and hear, it's unlikely that the average person will become the victim of a malicious deep fake. Whatever this says about our species, the overwhelming majority of deep fake content remains pornographic in nature, which also limits the scope of potential targets. However, given the growth of deep fakes, eventually that scope may widen.

Thankfully, all is not lost. While creators are pushing the boundaries of what the technology can do, researchers are advancing new detection methods. Until recently, the leading detection technologies could successfully identify a deep fake about 65% of the time according to a study in the Proceedings of the National Academy of Sciences. Under certain circumstances, people perform a little better. New research may close the gap.

Researchers at UC Riverside developed a method capable of detecting deep fakes with approximately 99% accuracy. It's capable of detecting not only when an entire face has been changed but also when the expressions have been manipulated. It can even identify which portions of the face have been altered. Of course, you can only detect fakes if you know to look for them and have the right tools. We'll all need to remain vigilant as deep fakes become an increasing part of our lives.