Two tech companies this week have taken steps toward limiting how youth interact with algorithms on their platforms. On Wednesday, Facebook and Instagram owner Meta proposed a settlement with state attorneys general in a trial over youth safety, and on Monday Roblox banned user-made games that feature scrolling video from being featured in its Kids and Select age tiers.
Meta has been on trial since last week over allegations by attorneys general across the country that its products are addictive to kids, and that it hid what it knew about this from the public. Meta denied the states’ claims. On Wednesday, Meta announced proposed changes to its platforms, writing that it will add time limits to teen accounts, block notifications at night and during school hours, and prompt teens to take breaks. The company also wrote that “Teens will be able to choose a non-algorithmic feed — one that isn’t personalized by our recommendation systems — as their default. We will periodically remind them of this option, and parents can choose to adjust their teen’s default experience to require this setting.”
If the settlement is approved by regulators, in addition to these changes Meta would pay states approximately $12.7 billion over the next 10 years to use on online safety initiatives for youth. Meta says it will pay another $5.3 billion if YouTube and TikTok “implement a one-hour Daily Limit, Night Mode, and age assurance measures” and “each pay an amount matching the [$5.3 billion] figure, with half of the remaining funds tied to YouTube’s payment and half tied to TikTok’s.”
“[T]he fact is that teens move fluidly between dozens of apps a day,” Meta wrote. “All platforms should empower parents and support teens by putting the same measures in place, because we know that when teens are restricted on one app, they simply move to another.”
While the algorithmic option isn’t the biggest part of the settlement, or the thing Meta is pushing on other companies, it’s the part that caught my eye the most. Muting notifications and adding time limits are all well and good, but to my view, they’re just band-aids over the essential hideous allure of these apps: the algorithmic feed that feels endless, that’s constantly pushing new hits of dopamine into users’ brains. Adults can barely withstand the effects of this, so I’d certainly believe it’s way worse for young people whose brains are still developing. These feeds lure people into spending more time than they intend to on platforms, and show them stuff that can feel designed to outrage or mislead them to keep them scrolling or get them in the comments. Meta, the attorneys general allege, wants this. User engagement is a success metric for all kinds of online platforms, but it can feel like algorithmic-based apps don’t care if users stick around because the products are good or enriching, just that it always feels like there’s something new beneath the next swipe.
Roblox, another company that has no idea how to keep youth safe on its youth-focused platform, has also recognized that maybe this shit sucks, or at least that it could be open to trouble over it. Kotaku reported that the company has banned for users age 5-15 games that include feeds of short-form videos that autoplay or loop and that include rewards for watching these feeds. Kotaku wrote that this is aimed at games like Steal an Egg and its clones, which allow players to level up their characters’ speed by running on a treadmill set before a screen featuring a short-form video feed. (Kotaku wrote that this feature has since been removed from Steal an Egg.)
“Rewarding users to consume a continuous feed of content without natural stopping points is not appropriate for younger users,” Roblox wrote in its announcement about the change.
I had no idea Roblox games had these kinds of features, so I’ve gone through a roller coaster of horror and relief to learn about this crackdown. But, as Meta writes, if young people are looking for a scrolling fix, there’s plenty of other places to get it. (I’m not handing it to Meta here; I doubt the company has had a Christmas Carol-style change of heart about the feature that undergirds its entire business model, and I wouldn’t be surprised if its demands on TikTok and YouTube were just to keep users from fleeing to competitors.) And Roblox itself is fueled by its own game discovery algorithm, which is based in part on player retention; the company might not want young players scrolling an endless video feed inside a game, but it still needs them faced with an endless scroll of Roblox experiences of varying quality.
This isn't unique to these companies. The whole internet, from actual websites to search engines and products, feels like it’s just designed to feed you shit now, but shit it’s hard to stop looking at or avoid. AI is bad because it produces things that are bad, but it’s also bad because it allows the worst actors to fuel this compulsive spew of crap at scale.
Trying to keep this shit away from young people is a good idea, but more widespread solutions are fraught. Banning youth from algorithms is a feature of KOSA, the Kids Online Safety Act, but KOSA’s critics highlight the ways the bill could be used to identify users, muzzle free speech, and crack down on queer content online. And countries that have attempted to ban youth from social media have seen use actually rise, and have been criticized for keeping young people from participating in a key part of society.
And algorithms aren’t the only thing keeping people online too long: While social media app Bluesky can show you only people you follow, it’s still rife with the endless scroll and engagement bait that algorithmic-based apps feed on. Maybe the problem is just the entire scrolling internet, but I don’t think that demanding everything be siloed or finite is a solution either, if such a thing were even possible. And I do think that, to an extent, youth need to be free to roam a porous, explorable internet; I wouldn’t be the queer person I am today without having spent time in AOL chatrooms I probably shouldn’t have been in as a teen. Too much oversight and restriction can keep youth from engaging with parts of themselves they might not be ready to, or that it might not be safe to, share with adults around them.
It can feel like the best you can do is just set healthy limits for yourself, something that’s hard for adults and so surely much harder for youth, especially when we’re all up against internet companies that built their success on smashing through users’ boundaries by design. Neither Meta nor Roblox is motivated by pure good will here, and at the end of the day it could be likely both just find new ways to try to keep users from logging off. But at least it’s satisfying to watch Meta tuck its tail, and I imagine it will be equally satisfying if it succeeds in taking YouTube and TikTok down a peg with it.
