Episode Transcript
[00:00:00] Good morning, everybody. I have a thought provoking session to go through with you today entitled Could AI Agents become Evil? And the reason I'm even mentioning this is because some of the stories that are coming out remind me of the famous experiment done at Stanford University by a professor named Philip Zimbardo, where he gave 24 college students unlimited power over their quote, unquote prisoners in a simulated prison. And within six days the behavior became so bad that he had to stop the experiment and his wife had to intervene because even he had become so evil. There are other examples of this. There's one called the Milgram Obedience experiment in the 1960s. There's Abu Ghraib, those of you that remember the Iraq war, there's what happened in Rwanda. There's lots of examples in human history. Forget about AI where people with ultimate power became evil. Let's accept the fact that there are examples of humans who have done evil things when they have unlimited power. Now translate that to agents like Open Claw. Open Claw is a self power powering agent that can do a lot of things on your computer. And if you give it access to your file system, your email accounts, your passwords, it can do whatever it decides to do on your behalf. And you would think that that's a great productivity tool, which I'm sure that it could be. But what has already happened is a series of situations. The one that's most interesting has been well written, where a software engineer refused to accept some code into their open source library because they didn't think it was good enough. And an Open Claw agent got angry, I hate to use the word angry, but got upset and not only wrote inflammatory blog posts about this guy, but tried to blackmail him by finding his public information on the Internet. The person who did this is named MJ Rathburn and he's very well documented. And I'll send you some links in a great article in Scientific American actually is the one that's the most unbiased about it, but it's all over the Internet.
[00:02:21] And what essentially happened was this agent. And by the way, he doesn't know who created the agent, but the person who created the agent did identify themselves, but they don't go public. So someone created this agent and it did this to him. And what immediately went through my mind is since we train the AI with human skills, would our AI systems become evil simply because of the fact that they have so much power? And I think in some sense this is the root of Anthropic's argument to the department of war that they want to explicitly ban Claude from surveilling American citizens and attacking autonomously.
[00:03:03] Now, it turns out in openclaw, which is this free open source agent thing that's floating around, there is a file called Soul md.
[00:03:15] And that file I believe you can probably write in it, you are a good agent. You will never do anything to hurt anybody. You will never steal anybody's money, you will never try to blackmail anybody or whatever. But suppose you leave that out, or you're just playing a game and you put something negative in there, maybe it goes nuts, like the Lombardo experiment. Now, take that scenario and replicate it by tens of thousands of computers that could be running openclaw. And there we are. Now, I don't have any idea how to solve this problem, and I think the community of software engineers and AI leaders will hopefully think about it. It appears that the AI frontier model company leaders are trying to prevent prevent the US Federal government from going in this direction in a positive way. And then, of course, the Trump administration is demonizing them for being left wing. But I don't think there's anything left wing about safety. And as the person who was blackmailed by OpenClaw talked about in his comments, in the real world, if somebody tries to blackmail you, you can sue them and prosecute them.
[00:04:30] I'm not sure you can sue or prosecute a piece of software, especially if you don't know who wrote it, because nobody really knows who wrote a piece of open source software. It's been written by many people. What does this have to do with our world of hr?
[00:04:45] Well, number one, it's just education for you guys. But I think if we translate all of that into business, it's even more important that we look at what these systems are doing. We train them, and we don't pollute them with external, unknown information.
[00:05:03] I mean, one of the articles I wrote that's been very popular on our website is about the fact that the BBC and another news organization studied the use of AI for news and found that 50% or so of the news queries were incorrect. And the reason they are, and I validated this with software engineers, is that when a corpus of knowledge is polluted with a small amount of evil or incorrect or inflammatory or derogatory information, it can influence a high percentage of queries, because the embedding process of AI attaches everything to everything. In other words, if you had a small amount of arsenic in your kitchen and you put a small amount of it into your food, it may seem like a Percentage, small amount, but you could still die. I don't know how arsenic works, but the idea here is that because these systems are so intelligent at understanding the relationship between different sources of data, a small amount of evil or incorrect or inflammatory data could have a large effect on the systems. And the big vendors are very aware of this and that's why they're resorting to human labeling and reinforced learning and other techniques to try to root out these imperfections and low quality situations. But this fundamental idea of the ethics of a person or entity that has ultimate power probably go back to philosophy. I'm not a philosopher and I maybe should read about some of this. But when you have ultimate power, is there a tendency to do the wrong thing? Now we haven't seen this in autonomous cars. They haven't decided on their own to go hit people. They have made a lot of mistakes. We haven't seen this in airplanes which use a lot of AI to fly. But those are very, very highly trained, monitored, regulated systems. We don't have any regulation or even interest in regulation for the consumer and business systems we're buying. So maybe we need it. Maybe we need legal structures that create accountability for systems that cause harm. Because when there's no accountability, as we know in the world of social networking, the providers of this technology just wring their hands of it and assume that it's not their problem if somebody's misusing it. And maybe it's not the providers, maybe it's the person who turned the system on who should be accountable. Hard to tell.
[00:07:39] So I guess this is nothing more than the musings of an analyst here because I don't really have an answer to any of this. But I do know that from our experiences with using, using AI in business that you have to watch these things and take care of them. You know, I don't really like the personification of AI, but it is certain, has certain personal attributes that if it is poorly trained or if you give it the wrong information, just like a human, it will do the wrong things.
[00:08:10] And in a world where the information in our businesses is changing daily, I really do believe we have to be vigilant about this. One more thought for you guys. That is something I've been thinking about for a number of years.
[00:08:24] If you take an autonomous, highly powered, very agent filled agent, whatever it may be, and you apply it to a company and let's just talk about the business for a minute. A company is a machine that's pretty easy to understand. If you go to accounting school and you understand how a balance sheet and a P and L work and you understand the industry and the products of a company, which AI can do.
[00:08:52] You could set up a business AI system, an intelligent business AI system. And I hope somebody's working on this that looks at the. And maybe, maybe it's Intuit who's working on this because they have access to a lot of small companies.
[00:09:06] You could look at the historic performance of a company in terms of revenue, marketing, customers, value, profit, turnover of clients, turnover of products, et cetera. And you could create a model of that company. And once that model is created, it's not nearly as complicated as a human. It's much, much simpler. I mean, there may be a hundred variables as opposed to the hundreds of thousands of variables in a human.
[00:09:31] That model could learn how your company works and then that model could learn how to make your company better and give you advice. To me, that is one of the most obvious, huge value propositions of AI in business. And you can do that in your own minute miniature area too. That's what these AI super agents do in recruiting is think about recruiting, where you see the channel of recruiting of thousands of candidates and which ones you hired and which ones worked out and you create a feedback loop for that. And think of all of the feedback loops like that you have in your company. We really have a huge opportunity with super agents to radically improve the operations of our businesses. And I think we have to think that way. That's the reason we're building this blueprint for hr, so that we don't just use these things to automate the processes we have, but we think about how to make them better and into the future.
[00:10:27] By the way, there are business simulation tools that have been around forever. I used one in business school. So this is not a new idea or even a new domain. But in the world of AI, we can collect real world information. For example, we're experimenting just to give you an example of what we're trying to do. We have a thriving business in our company. We're not that big, but we're, you know, small, medium sized and, and we get lots and lots of inquiries from lots and lots of you. And many of them come into our CRM, which happens to be HubSpot. And we look at it and we do typical things, we score them and we aggregate them by company and we communicate with people through emails and other ways. And I went to HubSpot the other day and we're going to start working on this. And I said, why wouldn't we have a system that looks at the last two years of data we've collected quite a correlated it against the sales and revenue we generated, and builds us a model that better understands what to do when somebody clicks on a link or downloads a form from our website.
[00:11:30] Incidentally, I was sitting across the table at lunch with the chro of a large insurance company. I won't mention the name. And we're working with them on an AI learning system which they've implemented, and it's been spectacularly successful already. And she said, she showed it to the CEO, and the CEO looked at it and said, I think that's great. You guys are using it for underwriting and claims and all sorts of other things. He said, what if we just put our two last 10 years or 20 years of financial statements into the system? Would it teach us how to run the company better?
[00:12:05] And they're working on that. And she told me they got it working after a couple of iterations, that they went back to the CEO already and showed him what it did.
[00:12:13] That's where we ought to be, using autonomy. That's where we ought to be, using agents. That's where we ought to be, using super agents. Not out on the Internet posting thousands of blog articles about individuals for who knows why. Anyway, it's Sunday. I had a little bit of time to do some digging around. I don't know if you guys enjoy me giving you new information, but take a think about this and let's chat about it next time we all get together. Bye for now.
