Sampling the TikTok Generation: The Good, the Bad, and the Ugly
Behavioral scientists that use online surveys for (part of) their research undoubtably suffer from the declining response rates and sliming sample size of panels. Whereas former generations were susceptible to phone or email-based surveys, Generation Z is spending their time almost solely on platforms like Instagram, Snapchat, and Tiktok (Facebook is already mostly for boomers. If this is news to you, you probably are one). To keep up, academia needs to innovate and include each new generation of social media. And it has its advantages: it’s cheap, quick, and huge sample sizes are possible.
Unfortunately, focusing on the new generations is not all good. We inevitably encounter the same validity and reliability problems from the young and trendy side of the distribution as we do from the boomer side. Unsurprisingly, the nonprobability samples that, for instance, Facebook recruits hugely differ from the (inter)national average Joe in for instance age and income, and housing status. Most academics take great care to ensure a representative sample, but we need to acknowledge the limits on both sides of the distribution.
Recently, however, the ‘viral’ phenomenon has entered the discussion. Prolific, stating to fbe an improved online panel over for instance M-turk, provides easy and relatively cheap access to huge sample of participants. On TikTok, ‘sarah :)’ posted a 54 second videos stating prolific’s services are a great "side hustle" (an easy way to make “a lot of money”) to over 4 million viewers. The effect was immediate, enormous, and ugly. Prolific issued a response acknowledging that their sample was now hugely skewed towards early 20-year-old females (see figure before and after the viral post). For the first time, it’s not just the sample, but the sampling itself that’s biased by social media. However, although ugly, there is a silver lining. These changes offer an immediate threat but also a long-term opportunity, although many fail to see it: youngster are approachable and easily motivated by peers to engage!
