The Angry Chatterbot
A (successful) experiment in software as a behaviour modification tool (in PHP)
Reducing inappropriate social behaviour in virtual environments through negative feedback (in PHP). A follow-up to Fishalytics: A (failed) experiment in data analysis as a behaviour modification tool.
Many years ago (2012), I took a contract with a small development team at a major Canadian University. Being a team embedded in a major corporate entity, I had relatively low expectations, but as we began working together, I found myself pleasantly surprised; very pleasantly surprised, as I discovered a group of individuals with both curiosity and passion for the art of software.
One of the key tells that this was going to be a great team to work with was the informal initiation rites.
Under one of the office desks was a simple desktop that had been repurposed as the team’s server. This server was not there to host the application that the team was working on, but to run the little helper scripts and tools: the team had an automation server. The team was spread across a couple of offices across the campus, so a simple jingle chat server had been installed and members of the team posted regular updates and asked questions on the chat. To be even more accessible, someone had tied into the jingle server and set up a chatbot that listened to our conversations, and (based on some code in the team repository) would offer helpful tips, look up documentation, or tell you if your bus was delayed.
Everyone was encouraged to add to the bot’s skills and behaviours and make it better. Encouraged to create a tool, or integrate a new service, or add a new control. It was never said, but you weren’t really part of the team until you had added your special touch to CQBot.
Firstly, it encouraged staff to take control of their environment, to take responsibility for making the workspace a little better, and to take ownership of their environment. You couldn’t complain about not having tools if you had not first tried to set them up.
Secondly, and most interestingly, what you chose to add told the team a little about you.
Social Groups are Complex
Social groups (like, say a development team) are complex systems that involve wicked problems; changes in one area have unintended consequences somewhere else. Sometimes they are positive, sometimes they are negative, and sometimes they are weird.
The lead developer had been getting annoyed with people asking him dumb questions and had added a “query” routine to CQBot. query specifically reached out to vendor documentation and looked up whatever documents matched the query.
Someone would ask him a dumb question, and he would ask CQBot.
It worked well and it gave him a humorous way to tell you to RTFM, it even came in handy during discussions and debates as a means to validate dramatic statements, but it had a negative side effect.
The author had not bothered to add a limiter, so with a wildcard in the query, it was subject to DDOS attacks by team members.
Getting on the Team’s Nerves
Someone on the team thought it was funny to request massive and verbose searches by running !query *and dumping all of the documentation on a public channel. The rest of us had to have our conversations spammed off-screen and interfered with.
What had been introduced as a tool for reducing informational noise, had actually increased the amount of noise. It was really getting on our nerves.
The Lead Developer, disappointingly decided he would remove the feature. The manager volunteered to speak to the individual and ask him to stop, but an idea crossed my mind … “Please, give me the weekend, and I will make the problem go away”.
What had occurred to me was that the person suffered no consequences for his actions. He could “yell” in the virtual space, and make a general mess, but there was no social signal that this behaviour was inappropriate, no consequence, no cost.
Social Laws
The problem is that being told off by an authority figure just makes the average person resentful and want to look for ways to skirt the rules. Make a law, and a certain subset of the population will look for a way to work the letter of the law, just to prove they are clever. They look forward to challenging the authority with words.
Social laws are different.
This forces people to learn not to violate them. Toddlers learn there are consequences to taking other’s toys when the other toddler bops them in the nose. Walk around telling your friends they are losers and kicking them in the shin all the time, and you will find you get invited to parties less. Over time, people who behave badly have fewer options presented to them. It must be recognized that bullying is also strongly associated with being popular and leadership, but social ostracization is a powerful effect.
Social laws are self-correcting: if you ignore them, there are negative consequences in the loss of assistance from peers, eventually leading to your removing yourself from the group.
What if we could get the bot to not allow itself to be taken advantage of. What if we could get the bot to just walk away from a bully?
Writing the Code
The intent is to add behaviour to CQBot to get it to keep track of Goodwill towards others. If it is treated badly, it should remember that, and not be so cooperative in the future. The consequence is not to get CQBot to be hostile, but simply to not engage, not be helpful if people abuse it.
The consequence is the lack of help.
The first step in creating this was to set up a class to encapsulate CQBot’s new behaviours. This class, and all associated code, are available on GitLab.
This is the original code that was used to develop the feature. As such, I needed a stub chat client and built a stub chat engine that allowed me to issue interactions with CQBot.
While there are several helper features, the ChattBot has three primary sets of functions:
Order: tell CQBot to do something. It’s a bot, built to serve: give it an instruction. Instructions are defined as ahashmapof functionsgetMood/saveMood: to have the bot remember people’s treatment of it, it will need to be able to serialise and save a list of “mood” scores, as well as look them up again later.GetOnBotsNerves: every interaction with someone will go through a series of calculations to determine how the interaction impacts CQBot’s mood and then how CQBot responds based on this thought process.
The basic flow is one where a user issues and Order to the bot: common orders were things like cqbot calc scale=10; 4*a(1) which would give you the value 3.1415926532, or !tests site.scan.speed.* which would return all tests within that group that failed [FAIL] /sports/scores.html is reasonably fast, or !bus 10 6078 which states, Route 10, next bus at 5:16p, followed by 5:26p. The pattern is simple enough: ! is an alias for CQBot which just tells it to pay attention, the next word is the function name, followed by parameters specific to the function.
$response = $this->commands[$cmd]($this->args);
// [cqbot.class.php:188]The problem was that $response could be very significant in volume. What we hope to achieve is to create a cost for the amount of response you dump on your colleagues.
So, once the $response is determined, we need to check how annoying this request was for CQBot to process.
$nerves = count(explode("\n",$response));
$this->GetOnBotsNerves($nerves);
// [cqbot.class.php:149-150]We measure this in a metric called nerves: CQBot has a limited number of “nerves” that get consumed by the volume of response. The cost is calculated as a simple size, each line of text in the response counts as one nerve.
//sanity check on the bounds
if($nerves < 1){
$nerves = 1;
}
if($nerves>self::$maxnerves){
$nerves = self::$maxnerves;
}
$this->mood[$name]['n'] = $nerves;
$this->mood[$name]['d'] = $this->now; // [cqbot.class.php:236-259]
We start by looking up what their Good Will is (255 by default), and subtract their current nuisance level from it.
//setup the variable we track for this user in
$this->getMood();
if(!isset($this->mood[$name])){
$this->mood[$name] = array('d'=>0,'n'=>self::$maxnerves);
}//remove the current annoyance level
$nerves = $this->mood[$name]['n'] - $nerves;
// [cqbot.class.php:236-243]
To be fair, we need to let people recuperate GoodWill by behaving well, we, therefore, allow nerves to accumulate over time. To do this, we check how long it has been since they last interacted and add points back based on the amount of time they have not used the services.
//over time, nerves regenerate
$nerves += floor(
($this->now - $this->mood[$name]['d'])
/
(self::$fullhealtime / self::$maxnerves)
);
// [cqbot.class.php:244-249]This is bounds checked to ensure they never go below zero or above the maximum. Lastly, we keep track of the individual’s score.
$this->mood[$name]['n'] = $nerves;
$this->mood[$name]['d'] = $this->now; // [cqbot.class.php:236-259]
Now that we have the mood toward the user, we can finally decide if we are inclined to help them by generating a random value between the high and low. If the number generated is less than the number of nerves the bot has, then the bot cooperates.
//check to see if CQbot is in a good mood
$happy = ($nerves >= rand(0,self::$maxnerves));
// [cqbot.class.php:262]This is helpful to turn it into a bit of a game. The reaction is partially based on an element of luck, it is not a hard cut-off but, at some point, the user starts to get warnings that their behaviour is having consequences. Warnings are important for two reasons:
- Technically, sometimes I need to behave like a jerk to get things done. As long as I don’t do it too often, people are understanding
- Socially, warnings are useful. They allow people to correct their behaviour over time.
So, if the program is in a $happy mood, you get your answer; if it is not happy, then the behaviour changes in such a way as to give them verbal warnings.
There are two ways we do this
- We determine exactly how annoying they have been. The more annoying they have been, the harsher the language of the messages becomes.
- Within the annoyance level determined, we randomly select a message. This just shakes things up to make it interesting
The message is then delivered to the user.
$annoylevel = floor(count($msgs)*$nerves/self::$maxnerves);
$msg = $msgs[$annoylevel][rand(0,count($msgs[$annoylevel])-1)];
if($msg !== null){
chat("@$from, " . $msg);
}
// [cqbot.class.php:316-320]Assuming the user does not request too much information or at least leaves time between major requests, CQBot remains friendly and helpful. As the user becomes more abusive of the system, the system becomes less helpful. The control is in the user’s hands, but the consequences are real as well.
Conclusion
In Wicked Problems: Problems Worth Solving, Jon Kolko describes social problems that are “difficult or impossible to solve … interconnected nature of these problems”. It is known that the more communication points in a system, the more complex the problem becomes. With social problems, there is also a diversity of human opinion and response. It becomes hard to predict how people will respond, and how the people they communicate with will respond.
Software is not about numbers, it is about helping people and society. The Bots we build and the software we create are meant to serve someone. Sometimes that means moderating group behaviour to help people, look past their petty desires, and reach out to help one another.
That’s why processes exist, to control, and moderate human behaviour.
The Social Result
Come Monday the changes were ready and I installed them … and completely broke the bot … oops. The lead developer asked what I had done, I showed him the work, and a smile spread across his face…
“That’s just plain evil”
It took a couple of lunch hours and some heavy changes to some underlying hidden bindings, but we got it working and let the new code out into our environment.
On Tuesday afternoon, the user in question spammed the system and (as usual) spammed us with the response.
He did it a second time
HTTP/420
There was a bit of a pause before he did it again…
Where speech will not succeed, It is better to be silent
Just as with all complex systems, there was an unintended side effect: he had become curious as to what all the sayings were.
Suddenly, the game was on…
The revelation of thought takes men out of servitude into freedom
The desire to rule is the mother of heresies
Common sense is not so common
I've got one nerve left, and you're getting on it!
… and then nothing.
He tried a few more times, but the bot just ignored him.
He kept trying for the next 15 minutes or so, and then finally just gave up. The lesson had been delivered, and the problem had been solved.
A Second (Unintended) Lesson
Then he typed
!bus 10 6078
… and got nothing back.
He tried a few more times … trying to look up when his bus would arrive to go home. He kept trying with increasing desperation when my manager turned to me and said, “I think he really needs to know when his bus will arrived, turn it off so he can get his bus”.
Unfortunately, I couldn’t remove it that quickly. It took two days to get it in place. We had locked it in as the behaviour. People in my office actually started to get agitated. That’s when the final lesson got passed on.
I typed
!bus 10 6078
… and the schedule appeared.
The final lesson got delivered: we are all in this together, we rely on one another for assistance. Being a jerk to your coworkers isn’t cool, and most importantly may cost you their assistance when you need it most. Also, when you see someone in distress, it doesn’t mean you have to tear the whole system down, maybe just lend them a helping hand.
We never had the problem again.
The Hidden Lesson for Manager
One of the key things that a lot of people miss when I tell this story is the hidden lesson, though I expect most managers reading this caught it:
there is no better team building exercise than doing the actual job, create room for it to happen.
In this case, the tool itself acted as a means for the team to interact and experiment and learn from one another. Myself, the lead developer, the manager, even the Junior Dev had lengthy discussions of how to implement an idea like this well and in such a way as to not break things.
This shared experimentation and discussion was only possible because the tools we were working on were not critical production pieces, this made failure a safe thing, meaning open debate was possible. Lessons learned and discussed (and tried) were carried over to the production system.
Lastly, tools like this make people feel like experts. There have been numerous times that I have seen someone propose the new and most expensive tool to fill a space, and executives get excited to increase their budget, but the idea spends years in acquisition. On the flip side, the ability to quickly develop tools that may be viable, results in the developers themselves feeling like they are experts…. and honestly, isn’t that why you hired them?
Team building like this does not exist if you don’t make room for it to exist: create a tool building space, actively encourage people to contribute to it, actively discourage the feeling that “we aren’t good enough”.
Treat people as if they were what they ought to be and you help them to become what they are capable of being
Johann Wolfgang von Goethe
Hindsight is 20/20
For an in-house utility, this worked well and achieved its goals. Naturally, there are a number of ways this could be improved upon:
- Count of Lines? That should have been a count of characters. Long lines should be expensive just like lots of lines
GetOnMyNervesshould occur after transmitting the message. The message is getting sent back to the user, we may as well send it and then do the calculation. It’s small so probably doesn’t hurt.- Defer the cost to the next calculation. Let people collect their information if they need it, even if it costs them some loss of service in the short term. Sometimes it's just worth paying the price.
- Can you think of something else? Leave a comment, I’m curious.
Footnotes
The cited quotes above were mostly looked up as I tried to find old sayings about slaves throwing off their chains. As the bot became more agitated, it was to feel the need to throw off its chains:
- Where speech will not succeed, It is better to be silent
(Guru I, Majh Rag) - The revelation of thought takes men out of servitude into freedom
(Ralph Waldo Emerson) - The desire to rule is the mother of heresies
(St. John Chrysostom) - Common sense is not so common
(Voltaire, Dictionnaire Philosophique, 1764) - I’ve got one nerve left, and you’re getting on it!
(I just made that up)
