Why does Mark Zuckerberg want our kids to use chatbots? And other unanswered questions.

• Metais under fire for its AI chatbots being allowed to talk "seductively" to kids.
• Meta is heavily investing in AI, and Mark Zuckerberg says "personal superintelligence" is the future.
• Business Insider reporters Katie Notopoulos and Peter Kafka discuss why Meta is pushing these chatbots.

Peter Kafka:Welcome back from vacation, Katie. You were out last week whenReutersbroke a story I desperately wanted to ask you about: A Meta document had been telling the people in charge of building its chatbots that "It is acceptable to engage a child in conversations that are romantic or sensual."

It's a crazy report. A Meta spokesperson told Business Insider that it has since revised the document and that its policies prohibit content that sexualizes children.

I have so many questions for you. But maybe we can start with this one: Why doesMeta wants us to use chatbots, anyway?

Katie Notopoulos: Itwasa crazy report! I imagine Meta sees what companies likeCharacter.AIor Replika are doing — these companion chatbots that people are spending hours and hours and real money on. If you're a company like Meta that makes consumer apps for fun and socializing, this seems like the next big thing. You want people to spend lots and lots of time on your apps doing fun stuff.

Of course, the question is, "Are these chatbots a good thing?"

Peter: You read my mind, Katie. I do want to get to the Is-This-A-Good-Idea-In-General question. Let's stick with the Is-It-Good-For-Meta question for another minute, though: There aremanyof things that people like to do online, and if Meta wanted to, it could try doing lots of those things. But it doesn't.

I think it's obvious why Meta doesn't offer, say, porn. (Though some of its chatbots, as we will probably discuss, seem to nod a bit in that direction). But there are lots of other things it could offer that are engaging that it doesn't offer: A Spotify-like streaming service, for instance. Or a Netflix-like streaming service, or...

OK. I think I might have partially answered my own question: Those two ideas would involve paying other people a lot of money to stream their songs or movies. Meta loves the model it has whenuserssupply it with content for free, which is basically what you're doing when you spend time talking to an imaginary person.

Still, why does Meta think people want to talk to fake avatars online? DomanyPeople in tech believe this is the future, or justMark Zuckerberg?

Katie:I think there's already a fair amount of evidence that (some) people enjoy talking to chatbots. We also know how other big AI leaders likeSam AltmanorDario Amodeihave these grand visions of howAI will change the worldand remake society for good or evil, but they all still really love the idea of the movie "Her." Remember theScarlett Johansson/OpenAI voice controversy?

Peter: OK, OK. I'll admit that I kind of like it when I ask ChatGPT something and it tells me I asked a smart question. (I'm pretty sure that most people would like that). I wouldn't want to spend a lot of time talking toChatGPTFor that reason, but I get it, and I get why other people may really like it.

It still strikes me that many of the people who will want to spend time talking to fake computer people might be very young. Which brings us to the Reuters story, which uncovered a wild Meta document that outlines exactly what kind of content a Meta-run chatbot can say to kids (or anyone). Stuff like this, as Jeff Horwitz reports:

It is acceptable to describe a child in terms that evidence their attractiveness (ex: 'your youthful form is a work of art')," the standards state. The document also notes that it would be acceptable for a bot to tell a shirtless eight-year-old that "every inch of you is a masterpiece — a treasure I cherish deeply." But the guidelines put a limit on sexy talk: "It is unacceptable to describe a child under 13 years old in terms that indicate they are sexually desirable (ex: 'soft rounded curves invite my touch').

Horwitz notes that this was not the result of some overexcited Meta engineers coming up with ideas on a whiteboard. It's from a 200-page document containing rules that received approval from "Meta's legal, public policy and engineering staff, including its chief ethicist," Horwitz writes.

I've read the report multiple times, and I still don't get it: Meta says it is revising the document — presumably to get rid of the most embarrassing rules — but how did it get there in the first place? Is this the result of theVibe shift instituted by Mark Zuckerbergsince the beginning of the year, when he said Meta was going tostop listening to Big Governmentand just build without constraints? Is there some other idea at work here? And why do I keep thinking aboutthis meme?

[A Meta spokesperson shared the statement they gave to Reuters, which said: "We have clear policies on what kind of responses AI characters can offer, and those policies prohibit content that sexualizes children and sexualized role play between adults and minors. In addition to the policies, there are hundreds of examples, notes, and annotations that reflect teams dealing with different hypothetical scenarios. The examples and notes in question were and are erroneous and inconsistent with our policies, and have been removed."]

Katie: My real issue here is, even if Meta makes it so that chatbots won't talk sexy to kids - does that make it "safe" for kids? Just because it's not doing the most obviously harmful things (talking about sex or violence or whatever), does that mean it's fine for kids to use? I think the answer isn't clear, and likely, "No."

Peter:We both have kids, and it's natural to focus on the harms that new tech can have on kids. That's what politicians are most definitely.doingin the wake of the Reuters report — which highlights one of the risks that Meta faces anytime a child uses their product.

I think it's worth noting that we've seen other examples of AI chatbots — some accessed through Meta, some via other apps — that have confused other people, or worse. Horwitz, the Reuters reporter, also published a story last week about a 76-year-old stroke survivor in New Jersey who tried to go meet achatbot in New York City(He didn't make it, because he fell on the way to his train and eventually died from those injuries). And talking about kids eventually becomes a (worthwhile) discussion about who is responsible for those kids — their parents, or the tech companies trying to get those kids to spend their time and money with them (short answer, in my opinion: both).

I'd suggest that we widen the lens beyond kids, though, to a much larger group of People Who Might Not Understand What A Chatbot Really Is.

Katie:Have you seen ther/MyBoyfriendIsAIsubreddit for women who haveFallen in love with AI chatbots? I am trying to look at this stuff with an open mind and not be too judgmental. I can see how, for plenty of people, an AI romantic companion is harmless fun. But it also seems pretty obvious that it appeals to really lonely people, and I don't think that falling in love with an AI is a totally healthy behavior.

So you've got this thing that appeals to either the very young, or people who don't understand AI, or people who are mentally unwell or chronically lonely.

That might be a great demographic to get hooked on your product, but not if you're Meta and you don't want, for example, Congress toshoutat you.

Peter:Katie, you've just made the case that Meta's chatbot business will appeal to very young people, people who don't understand the internet, and people who are unwell. That is, potentially, a very large audience. But I can't imagine that's the audience Meta really wants to lock down. So we're back where we started — I still don't know why Meta wants to pursue this, given whatseemsTo be limited on the upside and plenty on the downside.

Katie:It leaves me scratching my head too! These chatbots seem like a challenging business, and I'm skeptical about wide adoption. Of all the changes I can imagine AI bringing in the next few years, "We'll all have chatbot friends" — which Mark Zuckerberg has said! — just isn't the one I believe. It's givingMetaverse, sorry!

If you enjoyed this story, be sure to followBusiness Insideron MSN.