This app will actually make you look at your millions of forgotten photos

Our digital lives are cluttered. Take a look at your photo library, and you’ll quickly realize you have more photos and videos than you know what to do with, or even remember taking.

Lucky for the e-hoarders among us, there’s a new app that helps make sense of our digital memories. Dubbed Forevery, the app sorts photos by mining the emotional connection we have to them. Every time you open it, it automatically populates new photos into folders labeled by the activities, people or places they depict. It’s like photo roulette.

“It’s geared toward uncovering memories from all the photos you’ve taken,” Matt Zeiler, Clarifai’s CEO, said. “It seamlessly makes them relivable.”

Of course, it’s not the first app to do this. Google Photos, which the search giant released in June, takes your pictures and organizes them into photo albums, gifs, collages and short videos. (But I must say, its interface isn’t as nice as Forevery’s.) Another start-up, Magisto, creates shareable musical slidestreams out of your photos and videos.

“The entire product we have is using advance techniques to make stories memorable and people love them,” Magisto CEO Oren Boiman told me recently. “It’s supposed to evoke emotion.”

Companies are realizing that tapping into people’s emotional connections with their pictures is an important, and potentially profitable, business venture. (It’s not a coincidence that the icon for Forevery is a heart.) There’s lot of powerful artificial intelligence baked into all of these products: image and facial recognition systems that can detect objects, their relation to one another, facial expressions, emotions and ideas.

That’s the really interesting—and culture-shifting—part about apps like these: the emotion and abstract-idea tagging.

For as long as humans have been around, we’ve relied on feedback from our own faulty memories and other humans to remember our past. When we looked at old photos, we relived our happy or sad memories in our heads, or in conversations with other humans. If we wanted to augment our own nostalgia, we’d pull up our favorite cry-baby music as we reminisced.

But with apps like Forevery, Google Photos and Magisto, computers are being brought into the fold. Our photos are being tagged with things like happiness, love, togetherness, isolated, cheerful, pretty, idyllic, fine-looking, empty, joy, beautiful, gorgeous, temptation and illegal. They’re being set to music that can inherently change our emotional connection to them. Google and Facebook make it easier for you to bury unpleasant memories, like photos containing an ex.

And so the question is, will these technologies alter the way we remember our lives? If so, what are the repercussions of that?

When I asked Clarifai’s CEO that question, he said that the app allowed users to talk about events in their lives in “different and meaningful ways,” thanks to the tags its algorithms used to sort photos. “Togetherness” is a good example, he said, because the app is really proficient at picking that concept out. It applies the tag when people are hugging other humans or their pets or when there’s hand holding, for example.

For me, it sorted smiling pictures of friends and family or people doing group activities. Having them all displayed at once, with the “togetherness” label at the top, gave me a sense of warmth and closeness to the people in my photos that I don’t think I would have felt had I seen these otherwise unconnected photos in isolation or even in succession. That’s a really powerful thing. This app single-handedly triggered an emotional memory that maybe wouldn’t have surfaced otherwise.

Then, I stopped to remember how all this happened. It wasn’t a serendipitous occurrence, the way a conversation with a close friend might evoke feelings of support or love. This was the work of a big-ass mathematical equation. The AI that powers the app was trained on 12 million images Clarifai got from scanning the web and from proprietary data sets handed over by some of its clients. Using 11,000 tags, Clarifai engineers taught their system what things like dogs, cat, cacti, beaches, family, adventure, affection, celebration, and vicious looked like.

When you upload your photos it uses those preset categories to sift your photos into buckets. At least at first, you’re seeing the world, largely, through the eyes of the engineers that built the system. That’s kind of jarring. You can change it, though, creating your own tags for things and people in your life; the app learns in real-time to recognize them. In other words, Clarifai’s AI gets to know you and your world, the way a good friend might. That data can eventually be used to train and improve the company’s global model—the software that powers the app for everyone. It’s feasible then, that depending on who uses the app, the concept of adventure or affection can change.

That’s cool, but also kind of creepy. These apps are part of a growing push to get machines to understand emotions. “You can’t have intelligence without motivations or emotions,” Facebook AI chief Yann Lecun, told the New York Times recently.

And that brings up another question: Can an intelligent machine emotionally manipulate us? Maybe. There’s a field called affective computing that aims to make sense of human emotions. Psychology Today writes that:

Our expressions, gestures, vocal inflection, posture and gait all say something to those around us, often in stark contradiction with our spoken words. Yet these channels of communication have been entirely unavailable to our computers. Until now. Affective computing is being used in marketing, education and behavioral therapy and will in time become a common interface for social networks and gaming platforms.

I wouldn’t say that I felt manipulated by Clarifai or Magisto, but they did shape my perception of my own life, if every so slightly. Forevery isn’t a marketing service, but Clarifai does have clients that would be interested in smarter marketing. What the app learns about people could eventually be used to improve the AI services Clarifai sells to other businesses.

Using the free version of Magisto, I  put together a short slide show of images from a trip I took with my parents to Europe this summer. I chose a “love” theme, and Magisto took that to mean romantic love. The thing looks like an outtake from a honeymoon.

It wasn’t exactly what I wanted, but even so it was still sort of magical. A computer was able to take my memories, animate them, turn up the emotional volume and help me access and share them in new ways.

For the most part, today’s computers can feel dumb, despite the fact that they can do things better than humans in some cases. They don’t understand basic human emotions—and that means that our interactions with them can feel stunted. For companies trying to sell us stuff, that’s a problem because we’re likely to disengage more quickly. At companies like Google, Facebook, Magisto and Clarifai, engagement is key. They want consumers to keep using their products.

“The emotional part is going to be crucial,” said Boiman, Magisto’s CEO. “If they agree with the way we express a feeling in that video story, they will buy it.”

As with our fleshy companions, emotions will be a way for computers to keep us hooked. Remember that.

Daniela Hernandez is a senior writer at Fusion. She likes science, robots, pugs, and coffee.