Episode Transcript
[00:00:00] Speaker A: Okay, you guys, today I'm not going to talk about HR or the market or the economy or jobs. I want to talk about language. This is something I've been meaning to talk about and write, but I found that it might be easier to talk it through. The most difficult transformation we're going through in this new era is the words that are being created around AI because they're very vague and very confusing and, and people are making stuff up all the time, like the word harness or the word surface or the word fabric or the word mesh. So I want to spend a few minutes, not too long, just explaining what some of these words are. And certainly the most. The one that. The ones that I think are the most important so you understand this enormous new space. Okay? The first word is the word model. The model is the brain of the AI. It's. It's the large language model, the LLM, that takes data and information from text or images or graphics or voice and turns it into something that it can understand. And there are also what are called frontier models. And the word frontier isn't really defined, but I suppose it means the models that are moving the fastest, that are on the edge. And we tend to categorize the bigger vendors who are putting a lot of R and D into models as frontier vendors. Now, my experience in tech is in the 1980s and 1990s when there was a different form of technologies being developed. We had a very similar situation when databases were moved over to Unix and we had open systems and then we had client server. And, you know, many, many of the things we take for granted now went through the same evolution. But so that's, number one is the model. And you could call it a platform because it is a place you stick things on top of, but it's really not just a platform, it's a brain. The second word is the word context.
And context is a layer, and it's often called the context layer of software that makes sense of the information that is being consumed by the model. Now, the best way to think about context is think about your own brain. When you go for a walk and see a flower or a bird or a person or. Or a car or something that reminds you of something else, that's because your brain has a lot of power to understand context. We learn from many, many things during our lives, including genetically.
And as a result, when we see things or experience things, they have meaning to us. In the context of our history or our innate learning, the machine doesn't have any of that. So it needs A reference point or a way of putting things into context. Now, Theoretically, if the LLMs lived for 100 years and kept accumulating data, they would have historic context, but they don't live very long and they don't have a lot of history yet. So we give them context in a context layer and in the business world, in the enterprise world, the context layer we build is a layer of information about our companies, our business rules, our business and our culture. So at Deloitte we had a lot of rules about what we did and didn't do and why we didn't didn't do things because of risk. And in highly innovative company there would be a lot of context around what does innovation mean to us. In a highly hospitality centric company there would be context around that. Then this can be very practical. So in a logistics company like FedEx, they've built a context layer. It's called a layer, but it's really just software that defines what a package is and the characteristics of the package that's tracked in various systems. So that you can go to the context layer through a chatbot and ask it questions about a package and it knows what you're talking about. The word package may not mean anything or it may mean something else, but in the context of this company it means a very particular thing. So one of the things we do as a, as we build out these agentic HR 2030 systems is we have to build a context layer and there's no reason to put the context into every agent because that's too much work and somebody needs to keep track of the overall context, not, you know, the individual context of you as a team or you as a business unit or you as an individual. So this context layer is very important. We also have found, believe it or not, that Galileo belongs in the context layer because Galileo has tens of thousands of research assets that define words like leadership, management goals, performance, succession, pay equity, diversity and so forth in a very neutral, research based way. So we did it. We've been doing work with Microsoft and they have put Galileo into the fine tuned version of the Copilot and they found that it had 75% higher quality responses for many employee inquiries because it knows a lot about management and work and HR stuff. So you will find things that belong in the context layer that are relevant to your company. If you're a law firm, there could be certain things about the legal profession. If you're an oil company, there could be safety things. And that's going to be a big area of R&D for all of us is to build that thing. Okay, so that's the second word. And then the word that often goes with context is semantic layer. So semantic just means language from language. The third important word is memory. We think of memory as, like, how much hard disk we have or how much RAM we have in our computer. But in the AI world, it's different. When you go back to an AI a week later, a day later, it remembers things from before.
And the memory capacity of these models is getting bigger and bigger. What that means is that over time, you're going to have your own personal AI and it's going to know a lot about you and it'll be a big help to you. Just like we have our phones that have tons of stuff about us on our phones, and our computers do too, our browsers, all that stuff. The AI is going to have even more because it'll have your meeting recordings, your emails, your text messages, and whatever else you decide to give it.
So this memory capacity keeps going up. Of course, that's one of the reasons we're building all these data centers. So. So at some point we will have an LLM that could store the entire financial history of your company. Now imagine if you took the last 10 years of the detailed P and L of your company and you're a big company and you just stuck it in there and you said to it, I want you to analyze sales over the last five years and tell me what the variations are by season or by geography or by product line or by economic cycle. You'd have to put the economy data in there too. So this memory thing has a lot of potential. And I really do think the reason the agent world is so different from the old world is we're going to be modeling enormous numbers of things. We actually have a really great demo you're going to see on Galileo, where we've modeled an entire company in Galileo. Very cool. Okay, that's number three. Fourth word is rag, R, A G. Retrieval, augmented generation. I think that's the definition. Anyway, rag. RAG is the technology that intakes data into the LLM. So when you give it a picture or a document or a report or a spreadsheet, it has to read it, decode it into tokens. If it's a picture, it breaks it into tokens that are basically bits, chunks of bits. If it's a spreadsheet, you need to tell it what the spreadsheet means, because if it sees a spreadsheet and there are no titles on the columns, it doesn't know what to do with the numbers.
So RAG is the technology that brings things in, but you sometimes have to label the data clearly so that the model understands what it's seeing. We have a lot of tabular data in companies, transactions, sales, revenue, invoices, number of products, features, inventory levels, and so forth. So RAG allows you to take all that data and load it into the LLM. And what the LLM does is turn it into vectors, which I'll talk about in a minute. Okay, next word is tools. So tools are the skills that the model obtains. So if you take a bunch of stuff and you throw it into OpenAI or Claude or whatever, and you say, build me a PowerPoint version of this, it's got to manipulate the data to build it, but then it has to create a PowerPoint data object, and it needs to know what PowerPoint is. So it needs a skill that it takes through the tool, it's called tool gathering or tool taking, to make it capable of using that skill or that tool to build a PowerPoint. So what this means is that we can give or teach our LLMs different things by building skills that it treats as tools. So very flexible way of thinking about how we evolve these things. And many of you will build skills that will turn into tools for the models that you use in your company. Next popular word is mcp. MCP stands for Model Context Protocol. It's actually pretty simple. It's just a way. It's a client server way of agents or LLMs, talking to each other. It has been implemented now in a lot of systems. And what it functionally allows you to do is to take an intelligent system of type A and plug it into system B. So, like, you could take Galileo and plug it into Claude, and all of a sudden, Claude is a genius about HR and management and leadership. And it's becoming more and more popular and more interoperable and kind of a plugin, sort of a little bit the way USB works for hardware, where you can plug things into each other and use one interface with a different set of intelligence and data behind it. Okay, next word is the vector or embedding.
Now, the most amazing thing to me about the LLM technology is this idea of an embedding. An embedding is a very, very long vector. A vector is a long series of numbers that represents a relationship between many, many things. And what these embeddings do is they relate every token to every other token, which is a bizarre idea, but mathematically, imagine you had a token, a thousand objects and each of those thousand objects knew what its relationship was to all the 999 other objects. And now multiply that by about a million.
It's a very complex mathematical thing, but it's actually not that complicated. But if you do linear algebra, you'd know what this is. But basically, these embeddings mean that very, very long vectors define how the system works. And these vectors are stored in a vector database. So we now have data technology platforms and chips that are designed to manipulate these very, very, very long strings of numbers to perform the inference calculations we do when we ask a question or we tell the LLM to do something. And so you probably don't need to know a huge amount about it. But if somebody talks about embeddings or vectors, they're talking about the mathematical entities that allow the corpus of knowledge to work.
Like your brain, as I said, you know, your brain does this. Our neurons, somehow, magically, I don't really know the chemistry behind it, relate everything to everything else. I mean, you have a dream and you think back about your dream and you realize that it had to do with, you know, some thing that happened to you in your childhood compared to something that happened today. And, you know, it's all making sense. Well, the LLM has to do that all the time. So that's the vector database and the embeddings.
[00:12:13] Speaker B: One more important set of words I left out is the neural net and the weights. So in addition to the embeddings and
[00:12:20] Speaker A: the vectors that are used to represent
[00:12:22] Speaker B: data, the LLM uses what's called a neural network, which is a series of calculations that strings across millions and millions and millions of vectors to compute what the relationships will be between these entities.
And the way that works, you can look at some videos that actually show it on YouTube. There's a lot of them that actually display it, but there are weights, in other words, numbers like 1, 2, 3, 4, 5, that weight or tune the neural network to behave in a certain way. So when the neural network is starting, it trains itself, quote unquote, by creating weights based on the data that's in it. And then as you use it, or it makes mistakes, you want to change the weights to tweak it, tune it to become better and better at whatever you want it to do.
[00:13:16] Speaker A: And because the weights are numbers, they
[00:13:17] Speaker B: can only take one state. You can't change it from six to four, depending on the question you ask. So the model becomes biased in some directions. Because we decided that it was good at this, we assumed it doesn't have
[00:13:31] Speaker A: to be good at that.
[00:13:32] Speaker B: Now, there's a bit of a mathematical debate out there about whether a large model is better than a small model. But my own personal experience with Galileo is that a narrow model that's really, really good at a subset of stuff is much more useful than a broad model, because that's the way. The reason we talk to college professors who specialize in different topics, or doctors who are specialists in different things, or engineers that are specialized in different things.
[00:13:59] Speaker A: It's very hard to know everything because
[00:14:01] Speaker B: you can't vary the weights to be good at everything. I'm probably oversimplifying something that's much more complex than that, but those are words that come up a lot too. And the way the big front, the big vendors, the frontier vendors manage this is they retrain the models periodically and they go through a massive re balancing and re weighting every now and then, which costs a lot of money to change the weights based on the experience and intelligence that they want to optimize.
[00:14:32] Speaker A: Lots and lots of interesting math and
[00:14:34] Speaker B: science and engineering behind that.
[00:14:36] Speaker A: Another confusing word is the word orchestration.
Orchestration is the layer of management that sits on top of these models. It's typically not in one model, it's in a prompt or in another software tool that sits on top of it. Because the orchestration tells the model the order in which to do things and what to do. So if you're building a recruiting agent, agent means that it can have agency, it can do things. And you say, first I want you to review the job description and ask the hiring manager a bunch of questions. Then I want you to source the job using the following tools. Then I want you to qualify the candidates based on the quality criteria. Then I want you to screen the candidate based on another set of criteria. Then I want you to create interview scripts. Then I want you to interview these people. Then I want you to summarize the interviews, blah, blah, blah, blah, blah. Forty steps later we have an answer of who to hire. Well, those steps require orchestration. You could write it all down in one very long prompt, but then you got to edit the prompt every time you want to make a change. So why not take that orchestration and stick it somewhere else so that you can manage the orchestration separately from the intelligence in the system itself, because you're going to be also adding new sources of data into the system as it operates. So the orchestration layer is like the context layer. It's a tool you either build or buy that tells the agents what to do in what order and what we want to do today versus yesterday and why. And when you see the HR 2030 architecture, you'll understand why this is so important. Because with the super agents that we're building and talking about, the super agents orchestrate the other agents to do very, very important things that we find quite difficult to do as human HR people. Okay, next word that's bizarre is the word fabric or mesh. Now, I don't know where this word came from, but fabric is like a piece of cloth where it has layers within it and it has threads and we can stitch things into it. And this, these words are used a lot in databases and data because we have data coming from all these different systems and all these different formats, and some of them are relational and some of them are flat files, and some of them are images, and some of them are scans and other things. So the word fabric is used to describe the software that connects together various forms of data and it gets thrown into product names. The word mesh is also used for that. I'm not sure if there's a big difference between the two words. There's probably a technical definition, but that's another one that keeps coming up. Another word that's interesting is the word harness. A harness is a phrase that some engineer must have come up with. That's the software that sits around the model, the LLM. So when you go into Claude or you go into Microsoft Copilot and you see the user interface around it, and the fact that even Sana has this, that you can do things in the surrounding interface and change models within there, that's the harness that allows you to do things and use the model in different ways. So there will be many, many, many harnesses built by different vendors that will make your agent do the thing that it's supposed to do. Galileo has a particular harness that has hundreds of predefined prompts and agentic consulting prompts and things built into it. But there could be harnesses that do other things. Like the copilot has various harnesses to do things that are Microsoft graph related. Next word I want to talk about is governance. So it's not governance like Washington, D.C. this is the topic which is big of how do we maintain the quality, integrity and accuracy of these stuff that we're building? Who owns the quality of this? Who owns the quality of that? Who owns making sure this data is up to date? Who makes sure that the policies are up to date? Who will check the results and fix them by telling the system or teaching the system, things that it needs to know. That is an issue of governance and ownership. And there's the issue of rails or guardrails. Who or how do we decide what the agent has access to or what it is, is, or is not allowed to do? Does it need permission for this or that, or can it autonomously do this on its own? Those rails or guardrails, some come from the HCM or the erp, but the HCM and the ERP is designed for people and what data people can access. The agent might be a super agent that does work that one person never would have done. It would have been done by a group of people. So we're going to need rules agents, I believe, although this will obviously be a debate of agents that enforce rules. So you can go to the rules agent and update the rules without having to go into each independent agent and tell it, oh, we just changed the policy for this. Go change all the agents to behave differently. Okay, that's sort of the big stuff. I mentioned tool calling, I mentioned vector databases, model context. I'll keep up to date and talk about these things on this podcast, but I hope you found this educational and a little bit demystifying. These words keep popping out of nowhere, and a lot of them are authored by vendors or technologists.
And because the whole space is so new, you know, words like artificial general intelligence and so forth keep getting redefined. But at least for now, I hope these words help you understand a little bit more about what's going on, and we'll do our best to keep up to date. Thanks, everybody. Have a great weekend and have a great week.
