Reverse Engineering Haotian AI: An Underground Deepfake Tool Powering Romance Scams and Police Impersonations

One of my key responsibilities as GetReal’s Head of Threat Research is to make sure that we’re aware of all the popular deepfake tools out there. We look at every tool we can get our hands on in order to separate the toys from the ones that are real threats that criminals might use, and we assess how well we’re covering each of them with our detection capabilities.

Among the most elusive such tools is Haotian AI, a Chinese language tool that is marketed to criminal networks and used for a wide variety of activities, such as operating romance scams and impersonating police departments to shake people down for money. Haotian AI has been widely covered in the press, and is reported to have very sophisticated technical capabilities, but it is difficult to get access to, and ever since I started working for GetReal I’ve wanted to find a copy.

Opportunity Knocks

You can imagine how excited I was when Joseph Cox, an investigative journalist at 404 Media, contacted us to discuss Haotian AI. Joseph spent months tracking down the people who sell this tool and building a relationship with them that enabled him to buy a copy, paying with an obscure cryptocurrency on the TRON blockchain. Joseph is the first journalist to get access to this software and you can read his article about it here. 

A real concern associated with running any software of suspicious origins is that it might come bundled with malware or have built in surveillance capabilities. From our experience safely analyzing malware, we were able to provide Joseph with a safe environment to run the application on a computer with the technical specs needed to run it (most importantly an NVIDIA GeForce RTX 4080 GPU).

Haotian AI is a full service deepfake tool that is intended to be used by operators with limited technical skill. The software shipped as a set of encrypted RAR files with a copy of AnyDesk. We put the files on a computer, and gave Joseph the AnyDesk address, which he relayed to the HaoTian AI group over Telegram. They logged in and started setting up the software for us, which involved installing some NVIDIA drivers as well as Telegram, and disabling Windows Defender in order to keep it from interfering with the application. 

Getting Under the Hood of Haotian AI

Face swapping usually works by chaining together a set of AI models to independently process each frame of video. The first model identifies the face in each video frame, the next model actually swaps that face with the face in a target image, but usually at a fairly low resolution. Then there is an enhancer model that scales up the swapped face to match the resolution of the original video frame.

Haotian AI is no different. In fact, it uses a set of models that are available online and are included in many popular open source face swapping applications, like FaceFusion and Rope. The core face swapping model appears to be inswapper_128, and the package includes the GFPGAN and CodeFormer enhancers.

GetReal Security’s Protect product has robust detection capabilities targeting these popular face swapping models and readily detects the faceswaps generated by Haotian AI.

In addition, Haotian AI contains a Chinese origin commercial product called FaceUnity. FaceUnity provides a large set of beautification filters that can do things like remove acne, make your cheeks more rosey, thin your face, or make your eyes larger. We think these beautification capabilities may be used by romance scammers who want to touch up the face they are wearing on calls with their victims. Cranking them all up didn’t impact our detection accuracy in GetReal Protect, but they did produce some surreal looking faces. 

‍Our Assessment

Our assessment of Haotian AI is that real value it provides its customers isn’t coming from any special, sophisticated deepfake technology that can’t be obtained anywhere else. It comes from the way the software is packaged together and marketed to people who are interested in using it nefariously. Haotian’s full service model makes it easy for people with limited technical expertise to get up and running with these tools successfully. Joseph’s investigation identified $4 million in cryptocurrency that has been paid to Haotian AI. You can imagine that its customers have earned orders of magnitude more money using it.

GetReal’s mission is to continuously verify identity at the audiovisual layer. Understanding tools like Haotian AI is how we make sure our detection stays ahead of the threats our customers face. 

We greatly appreciated this opportunity to collaborate with Joseph Cox at 404 Media, and we hope to get more opportunities to work together in the future to understand how criminals are utilizing deepfakes in the wild.  

‍