Dating : Teaching a Robot to Like

How I used Python, pynder, and Google’s Inception Network to de-dehumanize Tinder (by fully dehumanizing it).

Introduction

When friends and family ask me how I feel about my machine learning Tinder adventure, I have to tell them I’m a little embarrassed, but also a little proud despite that. After all, it worked didn’t it? Today, I’m going to share that story, and tell you about the time I trained a robot to operate my Tinder account (this violates their terms of service). It’s going to get a little weird, and I’m fairly sure there are problematic parts to this story, so let me start by clarifying something right now.

This won’t be a how-to article, for a few reasons:

• The code is ugly and bad (not at all up to my standards)
• The Tinder API has likely changed significantly since I’ve done this [June 2017] (making this method not likely to work)
• This story could also be titled: How to inject bias into your dating life!
• Like a cheesy movie, I think the real core issues I had with dating were in me, not in apps or algorithms. The power was in me the whole time!

With that out of the way, let’s get started!

My problem with Tinder

When I was using Tinder there was one aspect of it that made me more uncomfortable than anything else: how the endless swiping made me feel. I am not inclined to believe that a person can really be described even in a long long summary (or Q&A, okCupid style), especially a self-created summary. So I was already a little disturbed by how non-representative most Tinder profiles are.

I’m sure most have experienced this: you go on your date and realize the other person is not really that much like their profile. You don’t feel like you’re being deliberately cat-fished, it’s more like the Tinder profile was for their identical twin (often the cooler, more athletic one, who has a dog and smiles all the time). Knowing this happened as much as it did, it already felt weird judging people based on these profiles, knowing that I was condemning some people based on false ‘data’. My own profile was a prime example: my fiancée (we met on Tinder) tells me she thought I was more ‘red-headed’ based on my photos (no idea how), and my bio didn’t say much about me at all (it was stolen from the Wendy’s About Us page). I have no idea why, but this seemed to get me more than double the matches than a more descriptive bio. Fortunately, it was rare for people to mistake me for the CEO of Wendy’s.

Even knowing that you are swiping based on limited and potentially misleading information, Tinder forces you to engage in this process. Among Tinder hackers, it’s known that if you always swipe right you get punished by not having your profile shown to others. The longer you make these snap decisions (left, left, right, left, right), the easier it gets. Tinder would have you believe it’s a game. It’s fun, right?! But it left a bad taste in my mouth. I felt like I was training myself to judge people I didn’t know (because I was), based on purely superficial details. I felt like I was dehumanizing these people, each of whom is living their own rich and detailed life that has nothing to do with me at all, by reducing them to a few data-points and subjective feelings about trustworthiness and attractiveness.

I wanted to use Tinder to meet people and go on dates, but I didn’t want to have to spend so much time swiping and sorting people. I was much more comfortable spending more time chatting in the app, trying to feel out the other person’s sense of humor and trying to set up a date with the funny ones (likes: depressing memes). So I thought to myself, what if I trained an AI to learn how I swipe, and I had it get all my matches for me? Then, all I would have to do would be talk to people, a much richer type of interaction than judging a few photos and reading a quote from The Office.

Teaching a robot to swipe right

When I set out to do this, I didn’t really have much experience with machine learning. The most I had really done was implement some clustering algorithms during my thesis work, and use some style transfer networks to design a tattoo for myself. This time, I decided to use a neural network trained for image classification. I was treating this project very much as a jump in and make mistakes type of project, not so much a careful planning and reasoning type project.

Normally, when training a neural net for image classification (is it a hot dog or not a hot dog?), you would need many thousands (or more) of images to use for training. Since in this case training data would have to be images of people that I had “swiped on”, there was no way I was going to get enough data to train a network to predict my swiping behavior (I wasn’t about to swipe on one million images in order to avoid swiping altogether). Luckily, I didn’t have to: I used a technique called transfer learning. In transfer learning, you take a neural network that has already been trained on a lot of data, and you make it “forget” the last bits that it has learned, the part that allows it to make the final call (looks like a hot dog). Then, you retrain the network on your new task (swipe right or left), but you only train that last layer that you just reset. In effect, you are not teaching it anything new about how to see these images, you’re only teaching it a different way to interpret what it’s seeing. Since this isn’t nearly as complicated, you don’t need anywhere near as much labeled training data. In this case I was able to get by with only 2000–3000 images. So now it’s just a matter of labeling some images, which unfortunately, wasn’t as easy as I hoped.

In the first versions I made, I tried a few things, most of which failed completely:

• train on the whole profile at once, as a collage (images not consistent enough, weak signal)
• train for two labels, like and dislike (bot was very easily confused)
• use facial recognition to isolate faces (compounded errors from the facial recognition)
• using a RNN to classify bios (almost every female profile did not have a bio, or only had fewer than 10 characters in their bio)

In the end, the data labeling and cleaning was much more complicated than I expected, and took by far the longest of any step involved in creating the bot. It turned out that many profiles contain pictures of groups, or pets, or trees, or really just confusing things that I couldn’t even label myself. What I ended up needing was 3 distinct labels:

like, dislike, neutral, ̶a̶n̶i̶m̶a̶l̶,̶ ̶g̶r̶o̶u̶p̶,̶ ̶o̶b̶s̶c̶u̶r̶e̶d̶,̶ ̶N̶A̶N̶ ̶(̶a̶ ̶c̶a̶t̶c̶h̶-̶a̶l̶l̶ ̶f̶o̶r̶ ̶e̶v̶e̶r̶y̶t̶h̶i̶n̶g̶ ̶e̶l̶s̶e̶)̶

Then, after classifying each image in a profile, the overall profile would be scored, compared to previously seen profiles, and a swipe would be sent. I finally had my work-flow laid out.

How did it work?

The actual code can be found in a Github repo, here. In short though my pipeline looked something like this:

1. Using pynder, fetch enough profiles from my location to use as training data (need ~3000 images), and put them into an “unsorted” folder.
2. Shuffle all the images. This step is critical for eliminating series bias. Basically, I don’t want to like or dislike a photo simply because I recognize the person in the image as someone I previously liked or disliked.
3. Load an image full-screen, and press an arrow key to automatically sort it into a folder called “like”, “dislike”, or “neutral”. This was actually pretty important, because if it wasn’t easy to sort in a quick and instinctive way, my boredom or frustration would begin to affect my judgment, and my labels wouldn’t be as consistent.
4. In the first version, we repeat this process, but with the neutral images. Each neutral image either stays in neutral, or gets labeled as “animal” (only or mostly an animal in the image), “group” (more than one human in the image), “obscured” (very small, or face obscured, or some crazy insta filter applied), or NAN (literally a just solid color, or a tree or meme or something). Big caveat: in the end this didn’t seem to be that important, and was really tedious, so I abolished the extra labels, and just kept them all under the ‘neutral’ label (i.e. I skipped this step entirely).
5. Next, we can retrain using the retrain.py script that the TensorFlow developers provide as a tutorial, which will allow us to take the Inception Net trained by Google, and retrain it using transfer learning on our particular set of images and labels (like, dislike, neutral).
6. Finally, a scoring function is needed to decide whether to swipe left or right on a profile. Each image would be given a like score, a dislike score, and a neutral score by the AI. I would combine these values into a single number using this scoring function: score = (like – dislike²) / neutral
7. Sum these scores, and you have a final score for the profile. Through trial and error I found I could get a ~30% swipe right rate if I told the bot to swipe right if the final score was greater than 0.5, and swipe left if it was lower. I also told it to send a super like if the score was greater than 250, because why not?
8. Time to set it loose (after telling it to slow down to avoid getting flagged for bot-like behavior)

Did it work?

Yes. Yes it worked. Well, in some ways.

When it was running, it would typically swipe on around 4000 people a day. Probably 1300 or so would be a swipe right. Of those 1300, perhaps 16 on average would result in a match. Of those 16, I would manually unmatch about 4 (sorry). Of those remaining 12, maybe 6 or so would talk back after I messaged them. Finally, a small handful would be interested in going on a date (yes, I did tell them about the bot).

At its peak (and my lowest point), I set up 8 dates in a 7 day period. This went about as well as you would imagine. On the one hand, this was exhausting. On the other hand, it was amazing and suddenly very fun to have so many matches.

I consider this to have been a success for two main reasons: first, I got my wish of no longer swiping (it actually was more fun to just chat and talk to people); and second, it was the bot who swiped right on the woman I am now engaged to, and very much in love with. The bot swiped right but it was our discussion on emojis that made it clear I needed to get to know this person.

The student becomes the teacher

The primary purpose of both the bot, and Tinder, was more or less fulfilled for me at this point. In addition to the main goal however, I also learned a lot about both myself, and dating, from this process. I’ll share some of that here.

• My preferences in a partner were actually less biased than I thought (feared maybe) they would be. I did some rudimentary analysis of who the bot liked and disliked and couldn’t find any meaningful patterns in race, birthplace, age, education, etc. (it did like swim-wear though, and smiles)

• The bot (and by proxy me) was less likely to like a profile if there wasn’t at least one clear picture showing the person whose profile it was, from more or less the knees up. There were even some profiles of people who seemed perfectly nice, but the bot was still very sure I wouldn’t like them:

• Having multiple photos is really useful to create a clearer picture of what you really look like (something it can be hard to judge for yourself, I learned). This helps prevent you from being burned by a unrepresentative photo.
• Bios didn’t matter as much as I thought they would. It turns out I was far more likely to ignore bios, and only consider them after a match, deciding then if I wanted to talk to the person or not. When I labeled bios and tried training the bot on them, it actually reduced the quality of the results a little bit (swiping left on profiles I would definitely like)
• While a small number of people thought the bot was creepy, most found it interesting and expressed a wish that they had something like it for themselves.
• Tinder themselves tightly control the rate of matches that you get, in order to try to maximize long term engagement with the app. After swiping on so many profiles, my match rate remained fairly constant, even after I turned off the bot. I suspect that there is both an upper and lower limit to how many people your profile is shown to (while you’re an ‘active’ user), with the intention of keeping you interested in coming back for more.
• Lastly, Tinder is very different for straight women than it is for straight men, and men on the internet need to behave better.

Final thoughts

As I mentioned above, I consider this to have been a success. That said, I wouldn’t do it again if I was single. I was really spending just as much time and effort working on the bot as I was swiping and dating in the first place. Then I was also spending the same amount of effort on dating.

It was a fun experience, and educational, but the code is embarrassingly bad (and naive). There are many better ways I’m sure this could be done.

Finally, the bot turned out to be more important for my morale than for actually getting me matches. With the exception of that one intense week, I wasn’t really dating any more or less than before the bot. I just felt better about it, like I had more control over my own destiny. If I could go back in time and give advice to my younger self, I would have told him, “Just slow down and relax: you won’t know who you like to be with until you get to know them well, and you can’t know someone well from just a profile, some banter, and their favorite emoji (…unless?)”

I will always be glad though, I owe my old bot for introducing me to the most important person in my life. It earned its keep, and now I hope it’s enjoying its retirement.