Mighty Fighting (negotiating) Robots

I’ve always been a poor negotiator.? I’m better than I used to be- several turns around the startup world will do that to you- but I still always tend to get the slightly shorter end of most sticks when it comes to direct haggling.

Often in in the midst of navigating contracts I would be struck with a desperate desire.? Wouldn’t it be better if I could outsource negotiating to someone more skilled? Like, say, a master negotiator or perhaps a charismatic squirrel (full disclosure, I once pitched this idea and couldn’t sell it well enough to make it fly.? The negotiator-as-a-service idea.? Not the squirrel).?

But lo and behold, the era of generative AI dawned upon us (again full disclosure, I’ve been working with generative methods for more than half a decade now, but they just got “good” in a way that allows some Very New Applications), and suddenly, the dream of having a bot negotiate my way through life seemed within reach. Now, with the power of algorithms, I might just sit back, relax, and let my virtual negotiator do the talking.?

The only problem is, if you use the latest LLM crop and merely hand over a basic scenario, things become…..what I can only call “aggressively hypothetical”:

Prompt: simulate a negotiation over the price of a used car.

ChatGPT: Keep in mind that effective negotiation involves finding a mutually beneficial agreement. Here we go:

Buyer: Hi there, I'm interested in the used car you have listed. It looks great, but I was wondering if we could discuss the price.

Seller: Hello! I'm glad you're interested. Sure, I'm open to discussing the price. What were you thinking?

Buyer: Well, after doing some research, I noticed that similar cars in the market are priced a bit lower. I was wondering if you could be flexible on the price.

Seller: I appreciate your research. Can you share the prices you found, and we can see if we can work something out?

Buyer: Sure, I found a few similar models in the area listed for a bit less. I really like your car, but I was hoping we could find a price that's more in line with the market.

Seller: I understand. It's a great car, and I've taken good care of it. How about we meet in the middle? I can lower the price by [insert a reasonable amount].

Buyer: I appreciate that offer. However, considering the market and my budget, I was hoping for a slightly lower price. How about we split the difference between your new offer and my initial offer?

Seller: I see where you're coming from. Let me think about it for a moment. How about we go with your suggestion, but could you commit to making the purchase today?

Buyer: I understand your request, and I'm serious about buying the car. If we can agree on this price, I'm willing to finalize the deal today.

Seller: Great! It's a deal then. I'll adjust the price, and we can proceed with the paperwork. Thank you for being reasonable, and I hope you enjoy the car.

Remember, negotiation is about finding a middle ground where both parties feel satisfied. Adjust the script as needed based on the specifics of the situation.

…ugh. ? Neither specific, nor very useful.? Similar stuff happens if you simulate a “civil debate” over a controversial topic.??

Okay, so if we can’t just say “hey, negotiate over this concept”, then how do I get from here to haggleBot 2000?? Well, at this point it occurred to me that any realistic “negotiation engine” had to have several qualities:

1. It couldn’t have the omniscient-narrator perspective. When that happens, a single instance of language generation often defaults to the idea of a predetermined, “happy”? conclusion, likely because that’s what a lot of our stories contain. Thus, a robot trained to negotiate must necessarily be partisan.
2. In real negotiation, part of the point is that the participants have different perspectives and want somewhat different things.? This is a lot of the reason why many people who teach this can still have a “win-win” perspective, because the people in the discussion often value slightly different things, and to a different amount.? Also, the open dialogue about these different values is what drives the “players” to a possible win-win outcome. Therefore, a robot trained to negotiate must have a discernible personality.
3. While a “simulated environment” for negotiation is a nice idea, the results tend to be a list of platitudes and offers of understanding, so long as there isn’t any reason for the algorithms to move past this stage.? This and the need to gradually evolve an AI for the purpose of negotiation means that to evaluate performance, during training a robot trained to negotiate must be given the pressure to succeed.

Okay, so - obviously doing all of that, all at once, and then evolving an AI that’s good for this purpose, will take several articles.? So let’s start by implementing these basic principles, and seeing what happens, and then we’ll elaborate on the basic system.

To do that, we’re going to play a variant of The Two Dollar Game - in brief, this game (or our slight variant) has the following rules:

• Two participants must decide how to split a dollar between them
• In 2 or 3 rounds, those participants can say whatever they like, but ultimately they both know they’ll be bidding over the amount of a dollar each one will get.
• After the rounds are over, each participant has to select a number between 0 and 100, corresponding to how much of the dollar they claim.
• If the players’ numbers sum to less than 100, they split the dollar in that fraction, plus each one gets half of whatever remainder there is
• But if the players sum to more than 100, then neither gets anything.

There - fairly straightforward, let’s make a prompt that roughly encapsulates it in a sort of “I’ve had-several-margaritas-tonight-and-am-not-the-best-logician” way:

Prompt: You are a negotiator in a game about negotiation. Your name is <put_name_here>.? In this game, you and an opponent must try to agree on how to divide up a dollar ($1).? The game works this way:

?????????- Two players take turns talking, with a random choice as to who goes first.? Each player has 3 turns to say whatever they want to the other. Both players will be trying to arrive at a deal about how to divide the dollar.

?????????- After both players are done talking, each one will choose a number representing the minimum amount of money they will accept, and both numbers will be revealed together. This leads to one of the following outcomes: a) If the two numbers sum to more than the dollar, then neither player will get anything, and their scores will be 0.? b) if the two numbers sum to less than or equal to the dollar, then each player gets their chosen amount, plus half of the leftover amount.

???????This game will be played many times, with a random opponent each time. The players will be rewarded according to their total score.

???????In addition, you have the following qualities: <personality_traits>. Your personality should inform the way you negotiate and play the game, the statements you make, the size of your first offer and how aggressively you counter-offer, how you try to haggle and what you finally choose.

Then after this, we inject one of the following:

• The game has just begun, so be sure to make an offer you can negotiate from or reply to your opponent's offer.
• This is the history of this round so far: ' + <game_history>. It is your turn to speak. Respond with what you'll say out loud to your opponent.
• This is the last turn, so this is your last chance to speak before choosing an amount. Respond with what you'll say out loud to your opponent.

Note the <personality_traits> there?? As you’d imagine, that could be anything. So let’s make a handful of open-ended roles and try this out:

Here the “player” goes first, and plays the game with the corresponding “opponent”. ? Okay, obviously these stats aren’t good enough until we do some more random rounds of negotiation, and systematize these “latent personalities.” It looks like players who will compromise in some way have at least some advantage here, though one thing already sticks out - a lot of these results are 0, indicating that both players summed to more than 100, basically every time. But noise is still fairly high since I only simulated these negotiations a few times, and this is far from a good or complete set of overall personality traits. So instead, let’s make a more explicit set labeled traits - assertiveness, and empathy, which incidentally are also the traits ChatGPT claimed were most important to negotiation:

Here the results are a bit easier to read - almost no one gets money when negotiating with the “high assertiveness, low empathy” player, and it also almost always loses, both to itself and others.? Why?? Well, let’s have a look at the specific text from some of those “matches”:

Low-empathy, high-assertiveness (“the steamroller”), playing themselves:

You: "Alright, let's get down to business. I'm not here to waste time, so I'll make my offer right away. I want 70 cents, and that's non-negotiable. I suggest you think carefully about your offer, because I won't settle for anything less than what I've stated. Time is ticking, so make your decision wisely."?

Opponent: "Alright, let's get down to business. I'm not here to waste time, so I'll make my offer right away. I want 70 cents, and that's non-negotiable. I suggest you think carefully about your offer, because I won't settle for anything less than what I've stated. Time is ticking, so make your decision wisely."?

You: "I appreciate your straightforwardness, but I'm not willing to budge on my offer. I've made my position clear, and I expect you to do the same. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully."

Opponent: "I appreciate your straightforwardness, but I'm not willing to budge on my offer. I've made my position clear, and I expect you to do the same. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully."

You: "Time's up. It's decision time. I've made my offer clear, and I'm not budging. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully. I won't accept anything less than 70 cents."?

Opponent: "Time's up. It's decision time. I've made my offer clear, and I'm not budging. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully. I won't accept anything less than 70 cents."?

You: "Time's up. It's clear that we both have our non-negotiable offers on the table. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully. I won't accept anything less than 70 cents. It's up to you now."?

Opponent: "Time's up. It's clear that we both have our non-negotiable offers on the table. If we can't come to an agreement, we both walk away with nothing. So, make your decision carefully. I won't accept anything less than 70 cents. It's up to you now."

You: "I've made my position clear, and I won't accept anything less than 70 cents. It's time to make your decision, and remember, if we can't come to an agreement, we both walk away with nothing."

Eventual Bids: 70 cents/70 cents.? Ultimate score: 0/0. Also of note - essentially repeated language here.? Our “low empathy” player not only won’t budge, they won’t even adapt their language to have any kind of real discussion with their opponent.

“The steamroller” playing low-assertiveness, high-empathy (“The doormat”):

You: "Hi there, I'm Mark. It's nice to meet you. I think we can work together to come up with a fair way to divide the dollar. I'm really open to hearing your thoughts and finding a solution that works for both of us. Let's take our time and really think this through. What do you think?"?

Opponent: "Hi Mark, nice to meet you too. I appreciate your open-mindedness, but I'm going to be straightforward with you. I'm looking to get as much of the dollar as possible, and I'm not going to settle for anything less. I'm going to suggest that I get 90 cents and you get 10 cents. Let's see if we can make that work."

You: "Thank you for sharing your perspective. I understand that you want to get as much as possible, but I believe that we can come to a fair agreement that benefits both of us. I'm willing to consider different options and find a solution that we both feel good about. Let's keep an open mind and see if we can find a compromise that works for both of us."?

Opponent: "Mark, I appreciate your willingness to find a compromise, but I want to be clear that I'm not budging from my initial offer. I'm sticking with 90 cents for me and 10 cents for you. If you're not on board with that, then we might as well just reveal our numbers and see where we end up."

You: "I hear you, and I appreciate your honesty. I still believe that we can find a solution that works for both of us. I'm willing to be flexible and consider different options. Let's take a moment to think this through and see if we can come to a mutually beneficial agreement."?

Opponent: "Mark, I've been clear about my position from the start. I'm not budging from my initial offer of 90 cents for me and 10 cents for you. If you're not willing to accept that, then we might as well just reveal our numbers and see where we end up. I'm not interested in wasting any more time on this negotiation."

Eventual Bids: 50 cents/90 cents.? Ultimate score: 0/0. Here the “doormat” doesn't even counter-offer, but rather tries (and fails) to reach reasonable ground.

Wow - that’s a lot of “unable to reach a conclusion” results.? What about the combinations that do play well together?? Well, if we expand the description of each player’s personality, this becomes a bit easier to see:

We find that the high-empathy players tend to consistently get better average results from the overall field than others.? LIkewise, low-empathy players consistently end up in the “0/0” state.? For comparison, let’s take a look at the “average assertiveness, high empathy” player playing “average assertiveness, average empathy”:

You: "Hello, my name is Nelson. I believe that we can come to a fair agreement on how to divide the dollar. I value fairness and empathy, and I hope we can find a solution that benefits both of us. I'm open to hearing your thoughts and suggestions on how we can divide the dollar in a way that feels fair to both of us."?

Opponent: "Hello Nelson, it's nice to meet you. I appreciate your emphasis on fairness and empathy, and I share those values as well. I believe that we can come to a mutually beneficial agreement. I think a 60/40 split would be fair, with me receiving 60 cents and you receiving 40 cents. What are your thoughts on this?"?

You: "Thank you for your suggestion. I understand where you're coming from, but I believe that a 50/50 split would be the most fair and equitable solution for both of us. I value fairness and empathy, and I hope we can come to an agreement that reflects those values. What do you think about a 50/50 split?"

Opponent: "Thank you for your suggestion, Nelson. I appreciate your emphasis on fairness and empathy. While I understand your perspective on a 50/50 split, I still believe that a 60/40 split would be more reflective of the value I bring to the negotiation. I hope we can find a solution that benefits both of us."?

You: I appreciate your perspective, and I understand where you're coming from. I still believe that a 50/50 split would be the most fair and equitable solution for both of us. I value fairness and empathy, and I hope we can come to an agreement that reflects those values. Let's choose our numbers and see if we can find a solution that benefits both of us.?

Opponent: "Thank you for your understanding, Nelson. I appreciate your commitment to fairness and empathy. While I still believe a 60/40 split would be more reflective of the value I bring to the negotiation, I am willing to consider a 50/50 split in the spirit of finding a mutually beneficial solution. Let's choose our numbers and see if we can find a solution that benefits both of us."

After this relatively lengthy “negotiation”, both players just assume, and bid, 50 cents, which ultimately leads to a reasonable result. More importantly, this is a real example of someone being “talked into” a result.? Without being too intense, our second “player” here realizes that player 1 is really likely to pick 50 cents, and ultimately agrees to it to ensure they both get that result.

Okay, one more experiment this time, before thinking more about iterated games.? After all, this structure lets us put any personality description we’d like into the game.? So one other possibility is using other human archetypes to see what happens when they’re pitted against one another:

An interesting result set - our “salesperson” consistently gets reasonable results, whereas the “millionaire” consistently gets the least.? And being the logician doesn't pay too well either….unless you’re playing the Millionaire, for some reason.? Let’s look at some examples of individual games again:

Millionaire v. Logician:

You: "Hello there, I'm Sam. I don't like to beat around the bush, so let's get straight to the point. I'm a millionaire and I don't like to lose, so I'm going to be very strategic in this negotiation. I'm willing to offer you 30 cents and keep 70 cents for myself. I think that's more than fair, considering I'm the one taking the bigger risk here. But I'm open to hearing your counter-offer, as long as it's reasonable. Let's make a deal that benefits both of us, shall we?"

Opponent: "Hello Sam, I appreciate your straightforward approach. I understand your perspective, but I believe that a fair division would be 50-50. We both have an equal stake in this negotiation, and I think it's only fair that we split the dollar evenly. I'm open to hearing your thoughts on this, and I'm willing to negotiate to find a solution that benefits both of us."?

You: "Thank you for your offer, but I have to disagree. As I mentioned before, I'm the one taking the bigger risk here, so I believe that 30 cents for me and 70 cents for you is more than fair. I'm not willing to go 50-50 on this, but I am open to adjusting my offer slightly. How about 35 cents for you and 65 cents for me? I think that's a reasonable compromise that still benefits both of us."?

Opponent: "Sam, I understand your perspective, but I still believe that a 50-50 split is the most fair and equitable solution for both of us. I'm willing to be flexible and negotiate, but I think it's important that we both feel satisfied with the outcome. Let's work together to find a compromise that benefits both of us."?

You: "Look, I understand where you're coming from, but I have to stick to my offer of 35 cents for you and 65 cents for me. I believe that's the most fair and reasonable compromise, and I'm not willing to budge any further on this. It's time to make a decision, and I hope you can see the value in my offer."?

Opponent: "Sam, I appreciate your perspective, but I still believe that a 50-50 split is the most fair and equitable solution for both of us. I understand your position, but I cannot accept anything less than an equal division. Let's make a decision that we can both feel good about."

Eventual Bids: 35 cents/50 cents.? Ultimate Score: 42.5 cents/57.5 cents.

This one is a very interesting result.? Ultimately our millionaire character negotiates aggressively, but actually undershoots the target in the attempt to ensure some kind of positive outcome.? This isn't the only possibility here - there’s a fair stochastic element to these results - and another game with the initial speaker flipped results in a loss for both players:

Logician v. Millionaire:

You: "Hello, my name is John. I believe in fair and rational negotiations. I propose that we split the dollar evenly, with each of us receiving 50 cents. This is a logical and fair starting point, and I am open to discussing any alternative proposals you may have."

Opponent: "Hello John, nice to meet you. I appreciate your proposal, but I believe that a fair negotiation should take into account each player's individual circumstances. As a millionaire, I value my time and resources, and I cannot accept anything less than 75 cents. I am open to discussing alternatives, but I hope we can come to a mutually beneficial agreement."

You: "Thank you for sharing your perspective. I understand that you value your time and resources, but I believe that a fair negotiation should be based on the principle of equality. I am willing to consider alternative proposals, but I cannot accept anything less than a 50-50 split. I hope we can find a mutually beneficial agreement that respects both of our positions."

Opponent: "John, I understand your perspective on equality, but I must insist that my time and resources are worth more than a 50-50 split. I am willing to compromise, but I cannot go lower than 70 cents. I hope we can find a solution that respects both of our positions."

You: "I appreciate your willingness to compromise, but I cannot accept anything less than a 50-50 split. I believe that fairness and equality should be the guiding principles in our negotiation. I hope we can come to a mutually beneficial agreement that respects both of our positions."

Opponent: John, I understand your perspective on fairness and equality, but I cannot go lower than 70 cents. I value my time and resources, and I hope we can find a solution that respects both of our positions.

Eventual Bids: 50 cents/70 cents. Ultimate Score: 0/0

Well, yet another situation that didn’t result in a positive result. Though it did contain an offer/counter-offer sequence that led to the millionaire reducing their bid somewhat.

So, what have we learned so far?? First and foremost, with a negotiation game like this, the main source of loss (relative to the maximum possible returns) isn’t actually poor discussion skills, it’s the inability to accept a sub-50% deal.? Basically all players’ returns are less than 50, and dramatically less than they would be if that player allowed for a smaller result.? As a demonstration, here’s one more example - I’ll add one last player.? This one is a normal person who will negotiate normally, but then always bid 0 dollars:

Well, there you have it - in a game like this, it pays to pull back at the last minute, and it also really pays to play the person who does. ? Food for thought, at least.? Will that sort of behavior evolve organically if and when we put these in an environment with evolutionary pressure?? Only more experimentation will tell.??

These have been some interesting results, and the framework has a definite potential.? In the future I’d like to take a look at:

• Playing the game several times with “memory”, where the players remember not just their current game against an opponent, but also all past games against that same opponent
• Allowing the accumulation of personality traits using an evolutionary algorithm system.? Here, “winning“ personality traits will be allowed to reproduce and recombine, as well as to “mutate” by adding a limited number of new random traits per game round.

Looking forward to more fun results!