How The Tiktok Algorithm Rates Your Face.
Tiktok, formerly known as Music.ly, has become an incredibly popular form of social media for its relatability and authenticity. Like all great things, it has its fair share of controversy and opaqueness. Reddit's CEO Steve Huffman and Facebook'sSheryl Sandberg both have their doubts about how the app stores and uses user data, with the US government even opening a national security investigation into it. Part of this fear stems from how shroudedBytedance, the parent company of Tiktok, is with their methods. With the amount of money, exposure, and fame tied to making it big on the app, it's more important than ever to really understand how virality algorithms actually work.
How do virality algorithms work?
I recently came across a brilliant video on my feed by the user @Benthamite looking into Tiktok's supposed beauty algorithm.
In his clip, he proposes that Tiktok rates videos for potential success based on the attractiveness of those in them. He also suggests a paper by Xie Et al on developing an algorithm to assess human beauty.
Algorithms to assess human beauty are nothing new. There are multiple from different Chinese universities all using the same dataset and building upon each other to produce more refined algorithms. For years, machine learning algorithms have been able be trained by humans to differentiate between attractive and unattractive. It's something humans can do incredibly easy because it takes up a relatively large, primal portion of our brain called the FusiformFace Area. We're so good at it we rate faces in inanimate objects, aka Pareidolia. For a machine, however, pictures of faces are just pixels. Facial recognition was achieved in the early2000's and typically uses Histograms of Oriented Gradients (HOG) where the processor detects sharp differences in pixel values as a facial outline or feature. This creates a bounding box that eliminates the useless background and saves on memory usage enough such that your phone can even do it in real-time.
How does this relate back to Tiktok?
With a basic understanding of how computers see faces, let's look at how scientists or more correctly, anthropometrists, the researchers who measure differences across ethnicities, see faces. In the exact same way actually. It's easy to see how identifying certain landmarks of the face, such as say nose width (alar base width) or forehead inclination, you can assess how far their measurements deviate from the average for their race. As explained in virtually every video, averageness or Koinophillia is one of the four fundamental traits for attractive faces. In fact, using these landmarks, forensic anatomists can actually predict the ethnicity of deceased remains, By allowing the algorithm to measure and then evaluate the degree of fit with these ideal averages, it can provide a 'beauty score.' A human can then evaluate if this result is accurate or not or how accurate the bounding box was at selecting the face, and the machine can tune its parameters and try again. This is the basis of facial machine learning. With Bytedance being based in a country with abundant labor and now plentiful user data, it's no surprise that their proprietary algorithm would be very accurate at picking winners.
How TikTok algorithm work in a similar way?
TikTok's algorithm is expected to work in a similar way. You give it a 2D input, it takes the geometric features of the face, compares them to anthropometric ideals. It also samples skin texture at points that most people have blemishes or irregularities in and provides a combined score of the two. However, instead of using a human rater to train it, it uses a deep learning neural network, where you identify a few top-level features of a pretty face and let the computer associate similar aspects it thinks might matter as it works its way down. Benthamite from Tiktok has actually uploaded a similar algorithm which he found online on TiktokBeautiful.com. Using computer-generated images from thispersondoesnotexist.com gave a tight range of scores from 2.5 to 3.5 out of 5, which is to be expected because that's where the bell curve of attractiveness lies for most people. Only 1 in a 100 or so would actually get a4 or 5. I'd agree with the ratings and say they're pretty accurate. In Benthamite's video, he argued that the study is based on white and Asian faces which is also true and fails to account for darker skin tones. This is likely because of the skin sampling portion of the score where darker skin has a greater range of contrast that the algorithm penalizes. Also more obviously, the system isn't trained with African anthropometric values so it's comparing them to Caucasian and Asian ones. After reaching out to him, he confirmed it used the neural network one proposed in the paper which uses a number of feature maps taken from attractive faces and compares how your face is different from it. What surprised me about the algorithm is the ability to measure skin smoothness and health. Videos with a greater momentary click-through rate and engagement, are obviously catching eyes and worth promoting to the trending page, or in Tiktok's case, the for you page. Systems try to optimize viewership and only promote high-quality content because views equal money. It's likely that this beauty score acts as one of the parameters on top of the rest of watch time, views, etc, and could help explain why everyone on your For You page appears to be so disproportionately pretty. Also, these creators get greater views by virtue of simply being eye candy, which the Tiktok machine learning algorithm, over time, would recognize as 'high-quality content'. When so much of the platform is based on the creator's personality, being attractive does help them get their foot in the door.
