Deal Dive: AI relationship coach Amorai offers more questions than answers

Building and maintaining relationships is hard, and COVID-19 definitely didn't help. Multiple studies have shown that adults have gotten even more lonely since the start of the pandemic.

Founders are trying to find tech solutions. There are many startups looking to combat loneliness -- some formed years before the pandemic -- including senior-focused ElliQ and Replika, which creates an AI companion, and Infection AI's Pi, an emotional support bot. But a newer entrant really caught my eye this week: Amorai.

The startup has built an AI relationship coach to help people grow and foster real-life connections by offering advice and answers to relationship questions. The company was founded by former Tinder CEO Renate Nyborg and was incubated in Andrew Ng's AI Fund. The company just raised an undisclosed amount of pre-seed funding that took only 24 hours to raise, Nyborg told Vox's Recode Media podcast back in April.

While combating loneliness is a great mission -- and some groups of people may be more open to chat with a bot than a human -- this feels like it has the potential to go so wrong so fast. But what do I know? So I pinged an expert.

Turns out I'm not the only one a little wary of this concept. Maarten Sap, a professor at Carnegie Mellon University and researcher for the nonprofit Allen Institute of AI, shared my concern. Sap's research focuses on building social commonsense and social intelligence into AI. He's also done research in the development of deep language learning models that help understand human cognition. Essentially, he knows a thing or two about how AI interacts with humans.

Sap told me that while the idea of creating a tech solution to help foster real-life relationships is admirable -- and there is definitely proof that there will be solid use cases for AI in combating these types of issues -- this one gives him pause.

"I'm saying this with an open mind, I don't think it will work," he said. "Have they done the studies that show how this will work? Does [Amorai] increase [users'] social skills? Because yeah, I don't know to what extent these things transfer over."

The biggest thing that gives him pause, he said, is the worry that this type of application will either give all of its users the same advice, good or bad, and that it would be hard for AI to get the nuances right about certain relationships. Also, would people trust advice from AI over another person anyway?

"The idea of the pickup artists sort of came to mind," Sap said. "Is this going to give you advice to tell a bunch of straight men to nag women or try to sleep with them? Or are their guardrails for this?"

If the model is designed to learn off of itself, it could create an echo chamber based on the types of questions people are asking. That, in turn, could point the model to a problematic direction if left unchecked. Bing users might have already learned this the hard way when its AI told people they were unhappy in their marriages.

Sap said that one way this could definitely work would be if there were a human touch to this. Human oversight to ensure that the app is giving the right advice to the right people could make this a powerful tool. But we don't know if that is the case because the company isn't answering questions or accepting interviews.

This round also highlights how deep the FOMO in AI really is. Someone who researches this stuff every day can't see how this company could really work, and yet Amorai raised funding in 24 hours pre-launch in a bad market.

Of course, investors know more about the company than what is released, and sure, these concerns can serve as feedback for the startup. But like a lot of AI startups, I have to assume it's building with good intentions, despite having nothing concrete to prove it.

I also don't believe this was a small pre-seed round -- something I usually assume when a company doesn't disclose the total of funding; if it was big, you'd want people to know -- but in this case, I think it's likely the opposite. It's a lot of pressure to raise a lot of money before executing or finding product-market fit.

"When I hear about these kinds of ideas and startups, it comes from a good place, but it often is just the tech solutionist mindset," Sap said.