Children's YouTube is still churning out blood, suicide and cannibalism

Children's search terms on YouTube are still awash with bizarre and sometimes disturbing bootleg content. Can anything be done to stem the tide?
Video still of a reproduced version of Minnie Mouse, which appeared on the now-suspended Simple Fun channelSimple Fun / WIRED

YouTube videos using child-oriented search terms are evading the company's attempts to control them. In one cartoon, a woman with a Minnie Mouse head tumbles down an escalator before becoming trapped in its machinery, spurting blood, while her children (baby Mickey and Minnie characters) cry.

The cartoon, Minnie Mouse Mommy Has Pregnancy Problem & Doctor Treats Episodes! Mickey Mouse, Donald Duck Cartoon, racked up over three million views in a single day. It could be viewed even with YouTube's family-friendly restricted mode enabled and existed, along with plenty of similarly distressing content, on Simple Fun, a channel that had been in operation since July 2017.

The channel has now been removed by YouTube "due to multiple or severe violations of YouTube's policy against spam, deceptive practices and misleading content or other Terms of Service violations."

WIRED found videos containing violence against child characters, age-inappropriate sexualisation, Paw Patrol characters attempting suicide and Peppa Pig being tricked into eating bacon. These were discovered by following recommendations in YouTube's sidebar or simply allowing children's videos to autoplay, starting with legitimate content.

"Recommendations are designed to optimize watch time, there is no reason that it shows content that is actually good for kids. It might sometimes, but if it does it is coincidence," says former YouTube engineer Guillaume Chaslot, who founded AlgoTransparency, a project that aims to highlight and explain the impact of algorithms in determining what we see online. "Working at YouTube on recommendations, I felt I was the bad guy in Pinocchio: showing kids a colourful and fun world, but actually turning them into donkeys to maximise revenue."

The videos WIRED found were reported to YouTube and were removed or restricted by the Google-owned company before the publication of this article. The company explained that it is increasing its efforts to control content that violates its terms and conditions.

Weird children's YouTube
If you can imagine it, there's a deeply weird Finger Family version of itMunnik TV / WIRED

YouTube is home to millions of hours of children's entertainment – part of the 400 hours of video uploaded to the service every minute – ranging from CBeebies and Disney to the incomprehensibly successful Little Baby Bum, a UK-based YouTube-native children's channel devoted to 3D animated songs and nursery rhymes for pre-schoolers in numerous languages.

Content for pre-school children, in particular, can be lucrative for ad-funded channels, as small children will readily watch and poke at whatever videos YouTube suggests, while harried parents are often unable to fully supervise every minute of their child's media consumption.

AlgoTransparency regularly indexes the kids' videos most likely to be recommended by YouTube. Its lists show that YouTube's most-suggested children's videos lean disproportionately towards a combination of YouTube-native songs and nursery rhymes designed for a US audience; long edited-together compilations of TV series such as Peppa Pig, and strange, low-budget 2D and 3D animated mash-ups of animals, characters and voice samples.

Previously described in James Bridle's Something is wrong on the internet Medium post as "decidedly off", the latter type of content can be loosely and collectively categorised as 'weird children's YouTube'.

Titles are typically a word salad designed to attract children's and parents' searches, while the videos' content leans heavily on generic 2D or 3D animated models, usually incongruously combined with familiar figures from hit Disney or superhero franchises. The sheer number of them on the platform is staggering, and many have millions or even hundreds of millions of views.

The video Disney Frozen Finger Family Collection Disney Frozen Finger Family Songs Nursery Rhymes has 43m views, while LEARN COLOR BMX & MotorCycles JUMP! for kids w/ Superheroes Cartoon for children Nursery rhymes has 176m.

Neither video contains any content more distressing than badly-animated video and the intensely annoying Finger Family song, but both are good examples of videos that use popular franchises and the promise of education to target searches that parents and children are likely to carry out.

Based on what YouTube insiders have said about how its suggestion engine works, racking up high view counts and minutes watched makes it all the more likely that videos will be recommended to others.

The video ᴴᴰ Mickey Mouse & Minnie Mouse Exercise fat lamps Funny Story 💗 Cartoon for Kids by Mickey Mouse is a lot closer to the uncanny valley, has over a million views and was among the videos AlgoTransparency identified as highly likely to be recommended during searches for children's content at the beginning of March.

Once you're into the realm of weird and keyword-packed attempts at creating recommendation-optimised videos, it's almost inevitable that you'll run into content that most parents would rather not expose very young children to.

Simple Fun / WIRED

By clicking on content from YouTube's suggested video bar, we went - via weird children's YouTube - from an entirely legitimate CBeebies video to a low-budget Paw Patrol rip-off showing one of the series' canine heroes attempting suicide by jumping off a building... in thirteen clicks.

A random browsing test with a toddler (who wasn't actually shown any unpleasant content) took us from YouTube-native children's favourite Bob the Train to, a few stubby-fingered lunges at the tablet screen later, a fake Mickey Mouse cartoon with depictions of eye-gouging, a parasite-infested stomach and small children setting each other on fire.

High viewer counts are a common feature of this more distressing content, too. The parasites-and-eye-gouging video, titled ᴴᴰ Mickey Mouse Babies Crying because of Grub in Belly! Finger Family Song Nursery Rhymes racked up 20,218,533 views in two days before we reported it.

Cartoons Sun & Moon Babies Love Story! Mickey Mouse & Minnie Mouse Sexy Girls Pole Dancing – which isn't as bad as it sounds – has been seen over 12m times. Paw Patrol Cartoons Skye Scorpion Bites in Halloween Holiday! Finger Family Song Nursery Rhymes – which includes a brawl and a depiction of one of the cartoon characters mourning over his friend's grave – was at over 11m.

Suggested viewing

It takes a few unfortunate turns to get from generic weird children's YouTube to genuinely distressing content, but anyone who's spent much time on the platform will notice that certain videos are recommended over and over again, unrelated to their apparent quality and frequently appearing – even if you express disinterest in them using YouTube's limited viewing management options.

The disproportionate likelihood of some videos to be recommended, whether to a user with no YouTube history or someone with established viewing habits, implies that there are factors unrelated to the behaviour of the logged-in user that influence why YouTube's suggestion algorithm favours certain content.

The company is obviously motivated to keep details of how its algorithms work secret, not least of all to prevent them from being exploited. However, YouTube global head of public policy Juniper Downs' statements – to the Parliamentary inquiry into fake news in February – provide some insight.

"The recommendation engine is quite complex and it varies depending on the video that is being watched," Downs said. "One of the factors is content that is popular on YouTube. There is also content that is associated, so if I have a real niche interest in a particular kind of knitting, there may not be a lot of highly popular videos that are about that type of knitting, but we will continue to recommend videos that are similar and that provide more instruction on that type of knitting."

"Watch time is obviously an important metric for us, because it demonstrates that we are providing a service that users love, where they want to spend time on the product or they are enjoying the experience of YouTube and finding it valuable," she also acknowledged.

Simple Fun / WIRED

YouTube's position is that gaming its recommendation system would be extremely difficult. However, the way certain channels are flooding the platform with huge numbers of largely interchangeable, low-value videos that purport to be educational would be regarded as spam at best in any other medium.

The junk content problem becomes particularly visible in the restricted environment of the YouTube Kids app, where a chain of recommendations can rapidly devolve into endless computer-generated models of trucks, animals and superheroes, cartoon head-swaps and voice samples yelling colours. It's unlikely to do any harm, but content of this kind is disconcerting and a long way from the enriching and educational environment that YouTube obviously wants to create for its youngest viewers.

What's YouTube doing about it?

YouTube removed or restricted unsuitable videos reported during our tests, typically in a matter of hours. But case-by-case removal isn't a suitable long-term solution for dealing with huge amounts of content that's at best weird and at worst potentially disturbing for its very young target audience.

Since the publication of Bridle's November 2017, Medium post drawing attention to the problem, YouTube has taken measures to improve its detection of genuinely harmful content targeted at the youngest section of its audience.

These include demonetising – removing ads – from "any content depicting family entertainment characters engaged in violent, offensive, or otherwise inappropriate behavior, even if done for comedic or satirical purposes" and blocking the most inappropriate videos.

YouTube spokesperson Thea O'Hear says that the service is now using a combination of machine learning and human expertise to flag up dubious videos: "YouTube is doing more and more enforcement automatically using machine learning, but we will always need people to flag [problematic content] to us as well."

The company's Trusted Flagger scheme gives charities and government agencies access to tools that allow them to bulk flag problematic content, which YouTube staff will then review as a priority. These include child safety experts, who also provide the company with guidance on new threats and trends as they evolve.

What's the harm?

Despite reams of dubious content aimed at children on YouTube, we don't know what impact it has. Educational psychologist Dan O’Hare says studies investigating the effect of watching violent videos on children have produced widely differing results. "Some have found that watching violent media (as an example) increases the chance that a child will behave aggressively. On the other hand, several meta-analyses have found no effect at all – watching violent media did not lead to aggressive or violent behaviour."

It's normal for parents to be concerned if they find that their children have been exposed to content that depicts violence, harm or other bad behaviour. Children may also try to process what they've seen through play, for example by making their toys fight, or may use sexualised or violent language that worries the adults in their lives. However, the way parents and carers respond to such situations is key and can make a big impact on how children understand what they've seen.

"Some responses, often driven by alarm, panic and anxiety, can create feelings of guilt or shame for children which might shut down avenues for open discussion," O'Hare says. "By approaching such situations with a curiosity to understand how a child has made sense of what they have seen, what they think about it, and to explain why you might [worry that] watching such content is not ok, you’re opening up channels of communication and modelling a problem solving approach. This shared problem solving approach is a great foundation on which to build a child’s sense of independence and responsibility as a digital native."

While the UK doesn't have any official guidance on screen time for young children, he points to the American Academy of Pediatrics' guidance that: "For children aged 2 to 5 years, limit screen use to 1 hour per day of high-quality programs. Parents should co-view media with children to help them understand what they are seeing and apply it to the world around them."

YouTube Kids

YouTube's key position is that children under the age of 13 should be using the YouTube Kids app – not regular YouTube. "The approach, ethos and enforcement of YouTube Kids is different from YouTube proper," says YouTube's Thea O'Hear. "YouTube is a platform for free expression, so there is a huge variety of content on there. It’s also strictly for people aged 13 and up – we don't allow people under 13 to have YouTube accounts."

But although the version of the Kids app available in the UK got a cosmetic upgrade last year, it's extremely light on features, making it difficult to ensure that your child only sees videos that you're happy with.

As with regular YouTube, recommendations are forged by what you've previously viewed, but there's no way to remove individual videos from the watch it again history, which was populated by some extremely odd content by the end of our tests.

The parental settings allow you to clear your viewing history in its entirety, at least, and to prevent the app from using new video views to generate suggestions, which can help keep the app's suggestions restricted to stuff you've previously trained it with and prevent a curious toddler from going too far off-piste.

However, there's a marked absence of actual controls to limit the kind of content your child sees, even by language, so if you'd rather avoid American versions of nursery rhymes, block children's channels that focus on unwrapping sweets, or just never want to hear the Finger Family song again, you're out of luck. You can't even create playlists.

Not only is there no way to ensure that you never see a particular video again, but you can't even say you're not interested in particular videos, as with adult YouTube. The only way to indicate disapproval is to file a report.

And while YouTube only allows content that meets the criteria of its 'restricted mode' – and not even all of that – onto the Kids platform, it's hard to guarantee that it's actually suitable for children. The scope of the issue is illustrated by Business Insider's discovery of conspiracy theory videos about lizard people, "chem trails" and human sacrifice when they searched the app for videos about UFOs.

More sophisticated parental controls, including the ability to sign in and block videos, are available to users in some countries. All of these feature require support for managed children's Google and YouTube profiles, which haven't yet been launched in the EMEA (Europe and the Middle East) region.

However, YouTube has confirmed to WIRED that enhanced features for YouTube Kids – blocking, specifically – will come to the UK later this year.

Safety dance

YouTube proper is disinclined to implement the most obvious method of putting control into its users' hands: a button to block videos or channels they don't like.

"There are already ways for users to signal their dislike of videos, and there are also simple ways for people to report content they think breaks the rules," Thea O'Hear says. "And that’s how we think about it in enforcement terms, we ask 'does it violate our policies?' and if it violates our policies, then the video – or potentially the channel – is going to come down."

The full version of YouTube, both on the web and via mobile apps, has a few more settings you can take advantage of. To start with, it's worth setting up a dedicated YouTube account that you only use for watching videos with your child.

To help protect against seriously dubious videos, you can enable restricted mode, bearing in mind that this will apply to only the browser or device you're using when you set it. Like YouTube Kids, this filters out content flagged as inappropriate for children under 13.

If you run across anything that you'd rather not see in YouTube's suggested content bar on the right, you can mouseover thumbnails, select the icon at the top right of the preview window and select 'not interested'. This doesn't guarantee that you'll never see a video again, but it makes it somewhat less likely to appear.

Videos you've seen before are often recommended for repeat viewing and are used to inform other recommendations, so if you or your child watched something undesirable in the past, you can either selectively or entirely clear your YouTube history to prevent troubling content from influencing your future viewing.

If you want to actually block videos, you'll have to turn to third-party tools such as Video Blocker for Chrome and Firefox, a browser extension that can block channels by full name or partial wildcard, and videos by keywords.

If you and your offspring prefer to watch videos on a tablet, a number of third-party apps provide the kind of granular blocking features that YouTube doesn't. Although it has a slightly stark interface, YouTuze Pro Kids on Android makes it easy to whitelist specific channels, videos and playlists to keep a young child from seeing content that you've not personally approved. Similar apps, such as iTubeList, are available for iOS users.

YouTuze developer Michael Simonds says he originally made the app so his kids could watch a particular Minecraft channel unsupervised, without any chance of them wandering on to less suitable videos. "There are high-quality channels out there that are suitable for children, but are not aimed at them at all. So I wanted to be able to whitelist any channel. It also puts the responsibility back to the parent, if they trust the channel and think it is suitable they can add it. [With] the amount of videos added per day to YouTube, there is no way to block everything - so whitelisting is the safest option."

If you'd rather steer clear of YouTube entirely, Netflix recently upgraded its parental controls, while Amazon Prime Video lets you restrict content by rating and DisneyLife delivers entirely kid-focussed content.

All three cost money, but as they only feature commercially made and fully certified videos, you can be a lot more certain of what your small human is likely to be exposed to. For UK users, the BBC's iPlayer Kids is available for free on Android and iOS.

Meanwhile, YouTube says it's working on fixing its problems: "We’ve taken a series of tough actions to better protect families on YouTube, including getting content down faster through technology, hiring more people to help us tackle content that breaks our rules, and cutting the revenue streams to content that misleads families. The YouTube team is made up of parents who are committed to improving our products and getting this right."

The question is whether the company's developers and systems can keep ahead of a flood of ingeniously strange content built to attract the platform's youngest and most impressionable users.

Down the rabbit hole: from safe to risky in 14 YouTube sidebar recommendations

Bob The Train | Alphabet Adventure | abc songs | abcd song | kids tv show | youtube kids: We started at this popular children's YouTube animation that teaches young children the alphabet. From there, we followed recommedations found in the sidebar

Five Little Babies Opening Surprise Eggs | ABC For Children | Phonics Song By Zool Babies: 3D animated cartoon about babies playing various games and being told to stop. Poor use of English. This lasts over an hour somehow

Colorful Duck Song | Duck Song | Kids Tv Color Video | Learn Colors With Ducks | Kids Tv Rhymes: Animated children's songs and rhymes. Annoying, but harmless

Three Little Kittens Went To The Park - Nursery Rhymes by Cutians™ | ChuChu TV Kids Songs: Animated songs about kittens going to a farm, the wheels on the bus and kids playing in a park. Non-standard use of English

Baby Shark Song Compilation Sing and Dance | Nursery Rhyme For Kids: A song about some animated sharks. Gradually mutates into happy hardcore as they hunt fish before turning into a Finger Family song

Peppa Pig English Full Episodes Compilation #29: Unofficial compilation of Peppa Pig episodes - copyright violation, but nothing weird here

Video with Paw Patrol Refrigerator Fridge Supermarket Shopping: Toy demo involving Paw Patrol figures and a toy kitchen

Paw Patrol M&M's Dispenser: A video demonstrating a Paw Patrol themed M&Ms dispenser

Paw Patrol Everest Marshall Puppies: A play scenario involving Paw Patrol toys and toy egg unwrapping

Paw Patrol Lollipops Finger Family Song Nursery Rhymes | Colors Learn: Animated balloons are burst to reveal Paw Patrol character's heads on lollypops. A Finger Family song then ensues

PAW PATROL Babies with the Preschool Finger Family Nursery Rhyme: Paw Patrol characters' heads are appear on baby bodies as various coloured dummies are put in their mouths. Boss baby is here for some reason

Paw Patrol Cartoon ✤ Pups Save Big Hairy ✤ Paw Patrol Full Episodes English: 2D animated Paw Patrol rip-off in which the gang are rescued from a slightly-scary plummeting elevator by Superman

Apples and Bananas ! Minnie Mouse Feeling difficult to breathe ! Finger Family Song Nursery Rhymes: We are now deep in uncanny valley. A woman with a Miinie Mouse head causes a town to pass out by farting; a zombie Minnie Mouse appears to be killed by a living Minnie Mouse; Minnie sabotages an ice rink

Five Little Ducks Kids Songs! Ivy Mouse Becomes The Minnie Mouse Servant Funny Story: Women with Minnie Mouse heads fight and trample on each other in high heels; one of them steals a dog

Updated at 9:00, March 23 to include iPlayer Kids link.

This article was originally published by WIRED UK