Episode Transcript
[00:00:00] Speaker A: Good morning, everyone.
[00:00:01] Speaker B: I am in India, in Singapore this week at a few conferences over here and meeting a bunch of clients.
And I felt it was time to try to shed some light on the lawsuits, one against Workday, another against Eightfold by job candidates about AI based recruiting. And so what I did was put together a fairly detailed, not super long discussion of what AI based recruiting is and its history and the two lawsuits that are certainly pending, but may or may not be relevant if they don't result in any damages. Workday was sued by a candidate who has claimed that he was discriminated in many job searches by companies who happen to use Workday's platform, somehow indicating that it may be biased against him or her because of he or her race or other characteristics. The second one against Eightfold is a lawsuit about explainability where some job candidates are claiming they were not adequately explained why they were not given certain jobs, and they're using a law for explainability in credit scores as an example. So you'll hear me explain it in more detail in a minute. I think in both cases, these are legitimate fears. After all, we're all watching AI scanning protesters in the United States and find vast amounts of information about us from the federal government for security. When you go through an airport line, it's just amazing how much data about people is floating around in these AI systems. So of course we have to expect that recruiters are going to get access to this through various vendors. But as you'll hear about in the podcast, it's not as simple as identifying somebody's experience. There's much more detail needed to assess job fit. And even if an AI does give a score, most employers don't consider that to be a final decision. They're only using it for screening. And let me also remind you that any questions or concerns you have about AI, I would double or triple or maybe tenfold those concerns about human recruiters because humans are biased and, and humans can't explain why they make decisions either. At least sometimes they can't. So my personal opinion is this is a very thorny problem. The vendors have worked very hard to make their systems explainable. And you can read Eightfold's position on this and Workday's position on this. I'll put some links in here. But if one of these lawsuits results in a significant amount of damages, there will be a new flurry of activity to validate or explain how these systems work. I do believe in the next year or so we're going to have to learn how to create explainability in these systems. It turns out in the more modern AI systems, you can ask ChatGPT or any of the other tools why it made a decision and it will actually.
[00:03:05] Speaker A: Explain it to you.
[00:03:06] Speaker B: The Workday system and the Eightfold system are much more complex and they're much more customized and they may or may.
[00:03:11] Speaker A: Not have that feature.
[00:03:12] Speaker B: But anyway, that'll be coming. Here's the podcast.
[00:03:15] Speaker A: I want to give you some perspective.
[00:03:17] Speaker B: On AI based recruiting, which is a.
[00:03:20] Speaker A: Controversial topic now supported by two lawsuits, one against Workday, one against Eightfold, by job seekers who feel that it's biased and unfair. And I'm not going to take a position on those particular lawsuits, but I do want to give you some context for this because it's not going away and it's a really important topic. So if you go back in time 30, 40 years, just for a minute, to the way recruiting used to happen before the Internet, we would literally mail a paper resume to a company, we would wait to hear back and maybe we would get a letter of interest and we would fly or drive to a job interview. We would go through the interview, maybe we would be selected, maybe we would go through some testing and maybe we would get a job.
[00:04:10] Speaker B: And so what that meant was that.
[00:04:11] Speaker A: The job seeker, the job candidate was very much unempowered. It was hard to find jobs. You had to actually read about them in paper newspapers. So there was a job advertising business, pretty big one actually. And we used to scan the want ads. I mean, I used to, when I was a young man looking for part time work, I would literally scan the want ads to see where I could find a job. Okay, so that was fine. It just was what it was. And then along came the web and the Internet and monster.com, which is now by the way, part of a company called Raidency, which I'm going to be talking about later. And later LinkedIn and others started to put jobs on the Internet. Employers didn't really have tools to put jobs on the Internet. Their, their job posting systems were called applicant tracking systems, but they were mostly stored in databases. And little by little through pioneers by, like Doug Berg, who I'm going to interview later, you could put these things on the Internet and so you could, you as an employer could post a job and it would go onto the Internet and then it would go onto these posting networks like Monster Career Builder indeed, which didn't exist yet. LinkedIn and people could find it. And those of us looking for jobs, which we all do at various points in Our lives would use these networks to find them. And a whole bunch of tools were created for candidates. And so the balance of power shifted from employer to candidate. We also had a war for talent and a shortage of skills in the middle of all this. So there was a sort of an arms race on both sides, but particularly on the employer side to build better and better tools to find the people or source them rather than post ads and pray that they would come now. You know, in the old days when I got out of college, the employers would go to universities and recruit people. That continued too, but it trimmed down a lot because it was easier and easier to get people online. And then the sourcing industry suddenly got smarter and smarter and smarter because AI, even long before ChatGPT, could scan through all this data about people, which was now on the Internet in LinkedIn or some someplace else. And there's an. And by the way, there's a massive industry of companies selling profile data of workers to employers, which I've also written about. So, so you could take AI, or what was not called AI at the time, and you could scan through these names, job histories, locations, degrees, certifications, credentials, comments about their resume, et cetera, and you could try to infer what this person might be good at and whether they would be good for a job. And the primary link or scoring factor was the words or language that the person used to describe the jobs that they had had in the past compared to the job that you're posting. And by the way, one of the things that's changed a lot is in those days, job titles weren't changing every day.
[00:07:10] Speaker B: They were pretty fixed.
[00:07:11] Speaker A: So sales representative was not a wildly changing role. So if somebody had sales representative on their resume, you could assume that they kind of knew what that job was all about, even though you have the issue of what they're selling and who they're selling it to, et cetera, inside versus outside sales, et cetera. So anyway, so we had all that and you know, one of the companies that kind of pioneered this searching technology was Eightfold and they got a 2 billion valuation in their early rounds. Then came along Bimmery, which was using this kind of technology for candidate relationship management, Seekout and many others, including LinkedIn, who can search through your resume to try to match you to a job. And we were all pretty excited about it because as a job seeker, you, you would start to get recommendations or in app suggestions in one of these apps about jobs that seem to be relevant to you. Of course, you know the bad side of that from a job seeking standpoint is you're getting it and so is a million other people, so are a million other people. So you know, you're not, it's not a very exclusive process, so you still have to put a lot of yourself into it. On the employer side, this technology was fascinating. And so companies started to buy these tools to, to find rare skills, very unique roles, technical specialists, people in different countries that were hard to find, or people that were leaders or leadership potential. And so the AI tools like Eightfold worked on things to assess leadership, to assess technical skills and so forth. And what they mostly were good at was figuring out that this individual has done this job before. They've worked in this industry, they've worked in this size company, they've used these kinds of technologies and so they're probably likely to have the, these skills. And the word skill, which I have been a little bit unhappy with from the beginning, became the credentialing factor that these systems use. So what they sort of turned into or were positioned as is skills, inference systems.
[00:09:12] Speaker B: Now, you know, if you look at.
[00:09:13] Speaker A: My resume or anybody else's, my skills are not obvious from my resume. My experience is interesting to some people, but nobody really knows why or how I did the things that I did unless they know me or people who know me. And they may find out that my skills are mostly somewhat technical, even though it may look like I'm some sort of a business guy, which I sort of am, but not at my root. So these were always sort of problematic in the sense that even if somebody looks like they have the right skills until you meet them and talk to them and hear what is really going on under the covers in their history and what happened in these experiences, it's a little hard to tell. But we were so excited about this and I mean everybody that we all bought this stuff, workday, went out and bought, hired, score. A lot of companies started to buy Eightfold and vendors like Bimary and Phenom and many others sold skills technology into the recruiting market in many forms, including.
[00:10:15] Speaker B: Analysis of, even analysis of video interviews.
[00:10:19] Speaker A: Supposedly you could listen to what somebody said in a video interview and understand their level of expertise. And then of course there's this peripheral industry for AI based assessments too. But I'm going to leave that out for now. So we had all this searching and indexing and inference technology and for the.
[00:10:35] Speaker B: Most part it was adding a lot.
[00:10:37] Speaker A: Of value because many companies could find qualified roles much, much faster and more deliberately than ever before. And you could use this data to analyze the job market and say, look, there's a trend for people in this industry to hire these kinds of skills. What about us? Why aren't we using those skills? Or why aren't we hiring in this location where all these people appear to be when we're located over here versus over there and many, many things like that. It's very valuable information. And then companies like Lightcast, Skyhive and others techwolf entered this space to add credentialing and labeling and organization to this information.
You could even look at trends in job titles and say to yourself, wow, look at all these job titles we don't have in our company. Why don't we do these things that these people do? So it's very interesting, very good stuff. But it was using AI and as you know, AI is not perfect and it infers things and it makes mistakes and it's all dependent on how it's been programmed. So, you know, I worked a lot with Eightfold, still do, and they built a lot of models under the covers, including looking at somebody's prior history to see their relationship between where they worked and when they worked there and what might be relevant to other factors of that. Because, I mean, the goal, to me, the essential opportunity here, which may actually be happening in the federal government unfortunately, but should be happening in the private.
[00:12:05] Speaker B: Sector, is if we really had good.
[00:12:07] Speaker A: Information on people's experiences and then we looked at data about the companies they worked in and the departments and roles, we could learn a lot about what they did in these various roles to not only help employers find candidates, but also to develop them, give them new career opportunities and many other things. And this is all slowly happening in the new world of Gen AI. However, if you're a job seeker and you're getting rejected from a bunch of job websites on different companies and nobody's calling you back, you might say to yourself, what is going on here? Is it my race? Is it my age? Is it my history? Am I being discriminated against? And of course that's illegal. There are certain factors you cannot discriminate against in at least in the US and most countries. So the legal issues have been there. And so the vendors that have been selling this stuff have really worked hard to unbiased their systems by testing them. You know, Ashutosh Garg at Eightfold has Multiple, has a PhD in this to make sure that they're not biased. By the way, a lot of the root technology ideas here are the same technologies used at Google to recommend an Ad or Meta. When Google and Meta send you an ad, they're not only looking at your clickstream, they know who you are, they know your gender, they know your political affiliation, if you've expressed it, they know your age, they know your buying history, they, they don't. They may or may not know where you worked, but they know the kind of work you do. And so that technology is very heavily engineered and lots and lots of people are working on it and it's not going away.
However, as with all AI, including Genai, it's limited because the data isn't complete. If you look at again your LinkedIn history as an individual, my guess is you probably haven't updated it for a while. I haven't updated mine for years. So there isn't that much in it about me as a person. There's just a lot of stuff about what I maybe the kinds of roles I've had, not really the things I've done. And I've always felt that, you know, the most valuable database that could possibly be created would be an experience database of what projects you did, not what companies you worked for, what job titles you had. But anyway, that's hard to get without an interview. Well, along comes this massive new market now facilitated by the LLM vendors for data labeling. Data labeling is a vague concept, but here's what it is. You're OpenAI or you're another foundation model company and you're trying to make sure that your AI is really good at welding, answering questions to welding. And so if welders need help on different kinds of materials and temperatures and techniques for welding, you want your AI to be the welding expert. You find out that it's not very good because it's picked up most of its information on welding from safety videos by some hack on YouTube or Reddit, and it's incorrect. So you hire 20 people who are experts on welding, including metallurgy experts, heat transfer experts, people that worked in oil refineries and policies of different materials. And you say, go through all of the queries on welding and fix them, label the answers, tell the AI through the tools that we built for you what's right and what's wrong. You know, score it or write an answer or fix what's wrong. And sure enough, six months later, through the help of these experts, your LLM or chatbot is now a world class welding advisor because it's been trained by actual real human experts. And you pay these experts 100, $200 an hour. So they think it's great, they Get a great gig job here and they feel like they're helping to build the new intelligence of the Internet. Believe it or not, this is a massive industry. There's a company called Mercour which just had an article written about it in the Financial Times. There's a bunch of these companies, scale AI that do this. Now this started out as a very simplistic market where the labeling industry was mostly used to get rid of porn or violence or horrible things that people have posted on the Internet to try to get that out of these LLMs. And that was done at by more like commodity prices. But now these companies are hiring scientists and doctors and business specialists and analysts and financial panelists to train their models to be experts at different things. We know a lot about this because this is what we've been doing with Galileo. Galileo is I believe, trained to be the world's authoritative, most comprehensive agent in the areas of business oriented HR and all of the aspects of that. And we've literally trained it and continue to watch it and improve it every single day. So if you're a oil company and you're recruiting welders and you happen to use this vendor's product that's really good at welding, you know, it might be able to identify from resumes or backgrounds, important criteria on what a good welder would look like on the Internet so you could go find them. That's probably not the perfect example. But if you're looking for a computer scientist or a biologist or a space rocket engineer, or a fluid mechanics expert for designing a jet engine or power plant expert or whatever, if the data is well labeled, going to be more accurate. The analogy here is a recruiter and I was just talking to find them about this and I'm going to talk about find them in a minute. If you are an oil and gas company, you're trying to find petroleum engineers, PhD petroleum engineers, you're not going to take a sales recruiter off the street and send them on that gig. They're not going to be very good at it. They're not going to know what to ask people, they're not going to know where to look. So similarly, why would you use a generic AI that doesn't know anything about oil and gas, petroleum engineering, or those kinds of careers, or those kinds of universities, or those kinds of degrees? It wouldn't really know much and it wouldn't be that helpful. And that's what's been happening is these fascinating AI search engines have turned out to be great until more recently where we have Much more need for much more refined data about people. Enter a new vendor and others will be coming called findom. Findom, who just received a big chunk of VC money, decided a year or two ago, maybe longer, to label human capital data in different industries and different domains. And if you think about, you know, someone like me or anybody else you're trying to hire, what you want to know is what they've done and what they've experienced and therefore their skills and capabilities in quite detail. I want to know if they worked for a startup, I want to know if they worked for a big company, I want to know if they worked for a global company, I want to know if they worked in this industry versus that industry, what period of time. And so what Find Them did is they built this very interesting metadata model of the types of data we want to get out of job profile information. And they've been labeling information and it's turning out that with the label data from Findom, this is not going to be the only company that does this. It's going to be a big deal. It's way more accurate and way faster and way more predictable and much higher quality hiring system. The initial use case that we've explored with them and we're talking to them about a bunch of others is in the military. If you are looking to hire leaders, managers, operations people and you want to hire people who have worked in the military, the language and the jargon and the job titles and the skills in the military are very different than in the civilian sector. I don't know the military very well, but they've built a labeled version of the job profile or the, the candidate database of the world for military recruit.com I believe is the name of the company we're going to have an interview with with them that's really good at finding jobs for people who've been in the military because it knows based on the experience these people have had exactly what skills they bring to the market. So that's great for them, it's great for recruiters, great for everybody. So this new world of labeling data intelligently is going to improve the capability and fine tune and the quality of this recruiting stuff. By the way, this is the same thing in your own internal AI systems. You're going to find in your company that you're going to want to label data too.
Obviously credit card scoring companies have been doing this a long time. We'll talk about that another time. Oh, and by the way, this also means that the scoring tools like Hired Score and most of the ATS and any of the others that you have are going to slip and fall behind the labeled tools. Just like OpenAI has to label data now to keep up with the quality demands of the users for the consumer AI market, we're going to have to do the same thing in recruiting. So these vendors like Finum and others are going to be, I think, very important in the market and they're going to grow and there's going to be others and they will tap into this bigger data labeling market, which has lots and lots of suppliers already. I mean, there's a philosophical question in my mind as to why you, as an Expert would make two, would want to spend your time making $200 an hour to train an agent to take your job away and destroy your entire profession and career. But, you know, some people don't think that way.
Okay, now we get to this issue of the legal ramifications of this. You know, I don't know a huge amount about the law. I read it and try to keep up with it a little bit. I think what's going to happen here is probably as the quality of these systems goes up, the attention or perspective or focus on quality of hire by candidates is also going to go up. It really depends on whether the workday or Eightfold lawsuits go anywhere. It's interesting. The workday lawsuit is by either one or multiple individuals who feel that they were personally discriminated by workday after applying to a whole bunch of companies that had workday recruiting software. The Eightfold lawsuit is a little bit different, where the candidates want to know why they want transparency. Why were we screened out? And they're using as a legal foundation the rules about transparency in credit reporting. When you, when your credit rating goes down, there's a certain amount of detail you're entitled to knowing as to why it went down. So it isn't a black box. Whereas this particular group of people also believes there should be explainability in the AI for recruiting. And I would say that is a good idea. And data labeling is actually going to make that easier because you will have more obvious visibility into why an individual was not selected. Now, what does the individual do about that? Can they ask the vendor to update the database like you can ask a credit card reporting company to update your credit if it's incorrect? That's probably where this is going to go. But that's a big, pretty big problem given the fact that there are 4 or 5 or 6 billion people looking for jobs, each of whom have some complaint about why they're not getting credentialized correctly.
But anyway, this is where this is going. Let's make this all relative to you as an HR person, as a business person, as an employer, you need to be aware of this. Look at the labeling vendors like findm and others. Ask your recruiting technology providers what their perspective is on this if they're working on it. And be careful that if you buy an AI based recruiting system that it is trained and knowledgeable about the domains and the industry and the specific jobs and the specific technologies and the specific skills you need. Because a generic AI used to be fantastic, it no longer really is because we're all becoming so sophisticated about using these tools. Personally, as an analyst, I think this is a really valuable direction for the market. The downside, of course, is this sense of monitoring and lack of anonymity for all of us. But that's to me, that cat's sort of already out of the bag. And we're going to do a lot more research on this and we're going to give you many more case studies on it. And I think it's really going in some interesting directions. Okay, that's it for now. Talk to you guys later. Bye.
