social turkers:

crowdsourced dating

What if we could receive real-time feedback on our social interactions? Would unbiased third party monitors be better suited to interpret situations and make decisions for the parties involved? How might augmenting our experience help us become more aware in our relationships, shift us out of normal patterns, and open us to unexpected possibilities? I am developing a system like this for myself using Amazon Mechanical Turk. During a series of dates with new people I meet on the internet, I will stream the interaction to the web using an iPhone app. Turk workers will be paid to watch the stream, interpret what is happening, and offer feedback as to what I should do or say next. This feedback will be communicated to me via text message.

overview

january 4, 7:00pm, testing the system

MTurk options:

010713_interface

MTurk payment: $0.30
Number responses: 91
Avg time/task: 2:41

Avg MTurk interaction rating: 2.7

I meet him on the internet, he says, let’s cut to the chase and just meet up, and I am all about that. This is my first experiment, I don’t know if the system will technically work well, I don’t know what the worker response will be like, and so I’m pretty nervous. On top of all that, meeting new people is the #1 thing that terrifies me, which is one reason I wanted to do this project and one reason why I felt like I really wanted to vomit on my walk over. Oddly, knowing that the workers are watching me made feel a sense of reassurance, like I’m not all alone in this situation.

At first, my attention is really divided between trying to make sure the camera is well positioned and working, and trying to engage with him. Once I feel everything was set, I start to relax a little and get into the conversation. Very quickly though, he starts talking about his relationship to knives, his transcendent experience of locating, owning and maintaining them, and how it makes him feel powerful, and about his family, and I suddenly feel really guilty. Is it wrong to be exposing this guy like this? I turn the camera toward myself. I remind myself, this is a tool for myself, it’s about monitoring and determining my own interactions. After an hour and a half my phone starts blowing up with messages saying LEAVE, so I do.

The turk test I did was simple, I asked the workers to describe in one sentence what was happening, to rate the interaction, and to choose from a set of actions (stay, leave, say _____, ask _____). Based on the responses, I realized I need to be more specific in the description I ask for. I had set the system to notify me when the average leave response was > 0.75, but I realized it was easiest for workers to choose leave or stay as they didn’t require a fill in the blank. To even things out, I think I will require an explanation for these choices.

View the turk log »

january 8, 8:00pm, let go

MTurk options:

010813_interface

MTurk payment: $0.20
Number responses: 31
Avg time/task: 3:37

Avg MTurk interaction rating: 3.5

There is that moment every night, walking alone through the dark in an unfamiliar part of town to some unknown location that I just pray has 4G coverage to meet up with some guy whose name I barely remember, pockets full of devices, where I feel a little scared and wonder what exactly I’m doing and whether I actually am a crazy person. Luckily, there are enough details to distract me. I am starting to get a handle on the logistics of the system. I pause before walking in the door, hit start on my Amazon MTurk batch job, fire up the streaming app, make sure the camera is well-positioned in my purse, reapply lipstick.

I recognize him immediately when I sit down, the kind of guy I would normally walk the other direction from. But I thought, what is this if not a chance to get to know people that I wouldn’t normally? He looks me up and down and orders me a drink without asking what I want. ‘Thank you’, the workers tell me to say. My number #2 reason for doing this project and #2 biggest fear is loss of control. But who is really in control? Me, as the one using this unknowing guy for my project? Him, because I need his participation, because I need him to like me enough to stay and interact? The workers, because they are determining my words and actions? I am spewing random phrases and questions supplied by workers, I sound like an idiot, he doesn’t really seem to notice.

It seems that $0.20 wasn’t enough, I think $0.25 is a sort of cut off when workers are searching for jobs.

january 9, 7:00pm, I’ll be your avatar

MTurk options:

010913_interface

MTurk payment: $0.25
Number responses: 60
Avg time/task: 4:03

Avg MTurk interaction rating: 2.6

I had set up the system this evening to check every 5 minutes, average the responses of the workers over that time period, and text me the top voted action. It made me into less of a random, turrets inflicted person, and more like an avatar. I could work the directions into the interaction more naturally.

It starts off with a lot of “back”, “stay” and “sidestep” instructions, and things stay polite and surface. I ask him about his work, his family, his favorite spots in Portland, and I’m starting to run out of small talk topics, when I get the instruction “advance”. I move a little closer to him and he seems to respond. A few “advance”‘s later and we are talking about past relationships. He tells me he asked me out because I had such a high match percentage on the site, and I realize that he’s one of those people that’s really hopeful about finding love through these things. He’s a very nice, earnest guy. He’s not at all my type.

He suggests we leave the bar and walk a little. “Stay” was my last instruction and I say maybe in a bit. Too quickly, another “advance” comes and I say how about that walk. He ignores my apparent indecisiveness and leads me out toward the street. Another “advance” and we are making casual contact. I wonder what to do if I’m pushed further, is this wrong? I think of the times in the past when I have kissed a guy without really thinking or wanting to, and decide at least in this case I have honestly chosen to give up my agency and act based on the decisions of the workers, and in this way I’m not being false. “Advance” and we are kissing. “Back”, thank god, and I am walking home completely unsure of what to feel.

View the turk log »

january 11, 8:00pm, empty

MTurk options:

011013_interface

MTurk payment: $0.25
Number responses: 23
Avg time/task: 4:32

Avg MTurk interaction rating: 3.0

I set up my MTurk task as usual, and head to a dive bar for our date. The options are extended further this time, giving the workers more ways to control me. The guy is all over the map and difficult to carry on a conversation with, and I feel relief in knowing I have help and don’t have to handle this on my own. However, I quickly notice that the instructions aren’t coming in with the same frequency as normal. I realize too late that weekend nights require higher pay to attract workers and I hadn’t offered enough to keep them constantly working for me tonight. I suddenly feel very alone. It is just me and him sitting awkwardly in this bar surrounded by people that know each other well, laughing together and carrying on casual conversation. I hadn’t realized how much I had begun to rely on this system for a sense of security and as a guide for my actions and words.

The prompts and directions previously had made the strange setup of online dating feel like a game or performance, but now without them, it just feels sort of sad and pathetic. Is this really the best we can do? I keep the conversation going, but weirdly, now more than ever, it feels like I’m just going through motions and reciting lines. Does he feel the same? Are we all just hoping we’ll stumble across the one that makes us feel more than this? Doesn’t it take a little effort?

View the turk log »

january 12, 10:00pm, my experiment or yours?

MTurk options:

011013_interface

MTurk payment: $0.30
Number responses: 8
Avg time/task: 5:58

Avg MTurk interaction rating: 2.25

After the previous night, I offer more money in an effort to attract more workers. As usual, I get to the bar a little early to make sure everything is running smoothly. Once I’m there, he texts me saying he will be a little late, so I sit at an empty table and wait. A pair of older guys approach and accuse me of stealing their table, then “offer” to share it with me. They sit down and start hitting on me in an extremely aggressive and insulting way, but I don’t want to leave the table because it offers a good amount of space for positioning my camera. I start the streaming and figure I might as well let the workers weigh in on the situation. The instructions are helpful only in passing the time, “smile”, “laugh”, “agree” they say. But the descriptions that I view later are interesting — even in the dark and noise of the bar, a lot of them seem to pick up on the aggressive and confrontational attitude of the men.

After 20 mins of sitting there while the men calling me cold and a bitch and a tease, my date finally arrives with a friend of his. They join us at the table and the two of them sit down unnecessarily close on either side of me, sandwiching me in uncomfortably tightly. I’m a bit surprised he would bring his friend along on a date, but figured he decided to make it more of a friendly thing, and decide to go along with it. I try to bring his friend into our conversation, but the friend barely responds, staring straight ahead while trying to fondle my purse. The other men are still at our table whispering together and laughing and pointing and staring at me.

Meanwhile, my date is telling me about how he likes making videos where he goes out in public and fucks with unsuspecting people. I start to wonder if this is what’s happening right now, the situation seems a little too excessive to be real. I feel annoyed and resentful thinking about the possibility that he could be messing with me like this. I wonder how it is different than what I am doing. Maybe I am a bitch. These nonstop dates with strangers and workers are really starting to wear on me. It’s late and I’m suddenly completely tired and I make some bad excuse and flee.

View the turk log »

january 13, 12:00pm, seen

MTurk options:

011013_interface

MTurk payment: $0.20
Number responses: 32
Avg time/task: 4:22

Avg MTurk interaction rating: 2.8

This guy is cute. Though not usually one to get super nervous around guys, the added preoccupation of getting the system up and running at the start of our time together makes me bumbling and awkward. I can see how nervous I may seem and it adds to my embarrassed lack of suaveness. Surprisingly, he doesn’t seem to pick up on this too much, or if he does, he lets it go in a casual, smooth way. The workers are not so forgiving, ‘The woman seems apprehensive, guarded and not terribly interested in the man. It could just be nerves but she appears to be uncomfortable. The amount of times she goes for her beverage shows me she’s nervous also.’ I am constantly surprised by how much they are able to perceive from a few minutes observation of a low fidelity video stream. It is almost as if because they are not physically present, they are able to observe more objectively and see the obvious that might go unnoticed by the participants wrapped up in the interaction.

One aspect of this project that is particularly interesting to me is the experience for the workers, it is sort of a performance for them in a way. On Wikipedia, the search for “artificial artificial intelligence” redirects to the page about Amazon Mechanical Turk. I am interested in this idea of these workers logging on expecting to do a series of very mechanical tasks that a computer would ideally do, but are instead confronted with the request for a very human reaction. Some of their responses are amazing.

‘But I imagine that even the most affective interactions leave those involved with distaste, weakened and hungry. I said “buying items” because I would want my character to have a history/a past… Not to say that she doesn’t, and in fact I find myself relating to both characters, but the problem of the actor is negligible when a theater-goer is asleep, or when we do not understand the accents, or when we understand each character independently.’

Many of the workers are very excited about the task, and leave their unsolicited emails and contact info asking that I send them updates as the project progresses. In some weird way, I’ve almost started to feel like they are my friends watching and helping me along on each date. They are the only ones that really know what I’m doing, they are my confidantes, my collaborators.

I get through the rest of the date with only minor stumbles as instructions sometimes clash with what I’m wanting to say and do.

View the turk log »

january 14, 4:00pm, questions

MTurk options:

I asked the workers questions in real-time based on the interaction at hand.

More to come…

january 15, 8:00pm, his questions

MTurk options:

My date asks the workers questions in real-time based on the interaction at hand.

I describe the project to him and explain that he can use the workers as sort of wingmen, asking them questions and getting responses without my knowing the content of their communication. Is this a cover story, he asks. The FBI might concoct a story as strange as mine to misdirect attention, he tells me. I assure him it’s not and wonder why that is his first instinct. With this issue apparently settled, he’s interested and into the idea, but as soon as I turn on the camera he tenses up and gets nervous to think of everyone watching. This is something I’ve never felt, I don’t know if it’s because I’ve been working with and thinking about these technologies for so long, or because I feel some more control as the creator of the system. For me, the gaze of the workers seems to provide company and comfort. He asks one questions, but due to a technical issue the system isn’t working tonight. This on the fly coding I’ve been doing is catching up with me. Disappointing, as it seemed to be going somewhere interesting…

View the turk log »

january 18, 5:45pm, what is me?

MTurk options:

011013_interface

MTurk payment: $0.20
Number responses: 23
Avg time/task: 3:02

Avg MTurk interaction rating: 2.8

I set up the interface tonight to allow the workers to suggest and vote on lines for me to say, rather than having them all come through causing me to have to spew constant nonsense. I’m interested in the possibility for the workers to work together and influence each other. Previously, I could recite the lines as they came in at rapid random pace, feeling like I was just some character detached from the reality of the situation. As the lines came less frequently, I am able to work them more naturally into the conversation and use them as a guide, pushing us in unexpected directions. I no longer feel like just a character or an avatar; the distinction before between performance and reality gets blurrier for me. One of the main goals of this project is to try to create some sort of freedom from myself. As directions come to say things I wouldn’t normally, I’m suddenly made aware of how many limitations I normally have. I realize that I hold onto this self concept and consider certain actions outside the set of things I do. As I submit to the directions, it feels at first completely uncomfortable. Somewhere halfway through the date though, I realize I really can break out of myself a bit. The realization is overwhelmingly powerful.

View the turk log »

january 20, 7:45pm, mturk profoundity

MTurk options:

011013_interface

MTurk payment: $0.25
Number responses: 60
Avg time/task: 4:42

Avg MTurk interaction rating: 3.2

Best response of the evening:

‘man seems to pity her and find her exquisite at the same time. WOMAN SEEMS TO HAVE STUMBLED UPON THE WAY TO LIVE!’

View the turk log »

january 23, 3:00pm, perplexed

MTurk options:

011013_interface

MTurk payment: $0.20
Number responses: 22
Avg time/task: 4:26

Avg MTurk interaction rating: 2.8

View the turk log »

january 24, 8:00pm, say/do

MTurk options:

011013_interface

MTurk payment: $0.40
Number responses: 43
Avg time/task: 3:54

Avg MTurk interaction rating: 3.1

View the turk log »

january 25, 9:00pm, what is perceptible

MTurk options:

012613_interface

MTurk payment: $0.40
Number responses: 43
Avg time/task: 3:25

Avg MTurk interaction rating: 3.0

I’ve been surprised how much the workers are able to notice from such low quality audio and video. They pick up on the times when I am nervous, uncomfortable, bored, interested, and engaged with stunning accuracy. Tonight, feelings of interest shifted into boredom, and the workers picked up on this. Sometimes the feedback tells me to relax or smile and I read the responses afterward and they workers mention I seemed really nervous. I watch the footage and realize they are right, I hadn’t even noticed during the date. It has really shifted my perspective seeing myself through the eyes of others. After a few dates, I feel like I am simultaneously seeing this third person view of myself while I am also on the date interacting with the guy.

‘Woman: Feeling sad and searching for something. Man: Looking more happy and enjoying life.’

‘The woman doesn’t seem to be very happy. I’m not sure if she’s bored or depressed. It’s like she’d rather be anywhere but there. She listens when the guy talks to her but it’s like she’s just putting on an act of paying attention but really wishing she could just leave.’

View the turk log »

january 26, 4:00pm, hacked

012613_interface

MTurk payment: $0.20
Number responses: 63
Avg time/task: 4:45

Avg MTurk interaction rating: 3.0

View the turk log »

january 28, 8:00pm, alone

MTurk options:

011013_interface

MTurk payment: $0.40
Number responses: 49
Avg time/task: 3:54

Avg MTurk interaction rating: 3.1

After a few days in LA with friends, I’m back in Portland, feeling more alone than ever. To do this project, I moved to a place where I wouldn’t run into anyone I knew already. I’ve been spending every night with a stranger. A friend of mine told me before I started that she doesn’t do online dating anymore because she often found herself on dates feeling more alone than if she were by herself. I now understand completely. I’ve met so many people desperately searching for the one. I feel bad when I don’t like them, I feel bad when I do — I feel like such a disappointment every time. I find myself relying on the workers to stave off the feelings of isolation on the worst of dates. Knowing they are watching somehow has kept me from feeling completely alone. They are my companions, my co-conspirators, my wingmen. They are the only other ones that really knew the truth of what I am doing.

For some reason, I always feel a need to push these projects past any point of feeling rational or in control. There is something about these moments that both excites and terrifies me — driving through the dark to meet a stranger to have a potentially awkward conversation, keeping track of all the technology components I have hacked together, making sure they’re all charged, running, bug-free, worrying about whether tonight’s experiment will prove interesting or insightful, daring to believe I might meet someone really wonderful, scared of disappointment, scared to disappoint, scared of boredom, wondering what I’m doing, wondering why I’m doing it. I have systematically disrupted all my systems for understanding myself and the context I operate in.

‘The two people seem to be meeting but missing each other over and over. The woman seems lost in her own thoughts and confused. At other times, she seems engaged and excited and smiling. The guy seems interested but unsure. The rhythm keeps shifting. Neither seem to know what to think.’

View the turk log »

january 30, 9:00pm, one last dance

Early in the day I announce to the internet I will be retiring from online dating with one last date. This sparks a stream of messages telling me ‘good luck’, ‘you must be nervous’. I don’t understand these at first, I’ve been doing this for a month, by now I’ve gotten over my worries about technical glitches or the whims of the workers. What I didn’t anticipate was how different I would feel when people I know might be tuning in. Before the date I am anxiously bouncing around double checking my code for bugs, searching out the perfect bar with adequate lighting, low noise levels, optimal seating arrangements. I locate one that yelp describes as ideal and confirm with my date. Shortly before the date I decide to visit to make sure there’s enough cell coverage for a good stream and discover that it’s a complete dead zone. I write my date asking to switch to another location, but don’t hear anything back immediately, and I start worrying I will get stood up. Is there anything more embarrassing than getting stood up in front of everyone online? Maybe having your art project totally bomb in front of everyone online? Trying to create a backup plan, I turn into that girl on okcupid writing random guys, heyy cutie what are you up to tonight? Putting me out of my misery, I hear back from my date, it’s on. I take a moment to laugh at myself and the absolutely absurd situation I let myself take a little too seriously for a bit there.

By the time I arrive, I feel much more relaxed and ready to just have fun with this end to this intense month. However, I quickly realize it’s one thing to be watched and directed by anonymous MTurk workers, but thinking my friends, family, and colleagues could be out there, I feel so much more exposed. The date goes along ok, but I feel my attention pulled in different directions in a much more intense way tonight. I’m split between checking my phone trying to make sure the stream is staying up, thinking about the people out there watching, and trying to be present with my date. Compounding this is a general feeling of being all over the place that has been steadily building over this month of experiments. He is sensing my energy and responding and it’s throwing both of us off balance. The worker directions are actually holding me together tonight. ‘DO be honest’ is the cue as my date asks me about artists I admire. Well, ok. I tell him about the work of a couple of my favorite artists. The date goes on uncertainly for some time longer, then we part amicably and I’m done! It feels too good.

I return home to find an email from one of the aforementioned artists, whom I don’t know too well, letting me know he was watching. I am so busted. I feel new weird levels of exposed. Thinking more about exposure, I also realize if I thought I was hopeless with love before this, if anyone I know was even thinking of dating me, they were able to tune in and see just how completely awkward on a date I really am. I spend a short while feeling some strange soup of ecstatic to be done, uncomfortable with how much I have shared, sheepish and worried remembering one of my professors telling me once ‘lauren, nobody wants to read your diary’, high on the rush of crazy energy I’ve felt this month…

Then all at once, a moment of calm. I remember writing one of my goals for this project last december before I had figured out anything about what I would do. I wrote: I want to use technology to change my understanding of myself and my relationship to others. Then I wrote: If I already feel ok with myself, am I all right with changing that, risking possibly changing it for the worse? Then I wrote: I can’t feel ok with myself if I don’t feel like I can change.

View the turk log »

  • info

    • about
    • turk eval page
    • contact
  • experiments

    • january 4, 7:00pm, testing the system
    • january 8, 8:00pm, let go
    • january 9, 7:00pm, I’ll be your avatar
    • january 11, 8:00pm, empty
    • january 12, 10:00pm, my experiment or yours?
    • january 13, 12:00pm, seen
    • january 14, 4:00pm, questions
    • january 15, 8:00pm, his questions
    • january 18, 5:45pm, what is me?
    • january 20, 7:45pm, mturk profoundity
    • january 23, 3:00pm, perplexed
    • january 24, 8:00pm, say/do
    • january 25, 9:00pm, what is perceptible
    • january 26, 4:00pm, hacked
    • january 28, 8:00pm, alone
    • january 30, 9:00pm, one last dance