Reimagine Financial Intelligence with AI

<script src="https://fast.wistia.com/player.js" async></script><script src="https://fast.wistia.com/embed/65ftfpggr0.js" async type="module"></script><style>wistia-player[media-id='65ftfpggr0']:not(:defined) { background: center / contain no-repeat url('https://fast.wistia.com/embed/medias/65ftfpggr0/swatch'); display: block; filter: blur(5px); padding-top:56.25%; }</style> <wistia-player media-id="65ftfpggr0" aspect="1.7777777777777777"><div class="wistia_preload_transcript_outer_wrapper" style="width: 100%; height: 100%; display:flex; justify-content:center; align-items: center; margin-top:-56.25%;"><div class="wistia_preload_transcript_inner_wrapper" style=" overflow: auto;"><p class="wistia_preload_transcript_text" aria-hidden="true" tabindex="-1" style="text-align: justify; font-size: 5px !important;">The Teamwork Bankers Association is pleased to welcome you to today's webinar from automation to autonomy, how agentic AI turns intelligence into action presented by Amdocs. My name is Isabelle. It is my pleasure to facilitate today's event. Thank you for joining. Please note we are recording and all participant lines are muted. If you have any trouble, please email conferencesconsumerbankers dot com or send a message in the Q and A box. This presentation will last up to sixty minutes and will include question and answer opportunities at the end. You may submit a question at any time by entering the questions into the Q and A box on the bottom of your screen. As a reminder, the views expressed in this webinar are those of the presenters and do not represent the views of CBA or its members. It's now my pleasure to introduce our speakers, Guy Hilton, VP of strategy in JTM at Amdocs, and Paul Saleh, principal consultant at Amdocs. Guy and Paul, welcome. Thank you very much, Isabelle. Alright. So hello, everyone. Thanks for joining us. So, Isabelle, introduced us. I'm Guy. This is Paul. And what we wanna go over in the next probably hour or so is a bit of what we see in the market regarding the adoption of AI, Gen AI, agentic AI, some things we talk to CIOs about, the CDOs, some best practices, and some things to think about as you continue your journey towards the AI and agentic AI. So, Paul, why don't you go ahead and first introduce yourself for a couple of minutes? Yeah. Thank you, Guy. So Paul Solay. I'm managing principal consultant over in the Amdoc Studio. So we focus on being able to work very closely with our customers and provide services solutions inside of their environments. I've been working in technology for twenty years now across cloud, data, digital, and most recently, generative AI, working across the FSI space as well as other markets as well. And I'm Guy Hilton. No connection to Paris or to the hotel chain. I run strategy and go to market for Amdoc's Amdoc's studios and industry verticals with specific focus on financial services. Been doing it for also the better part of twenty odd years. Background in technology, moved at some point to the business development and strategy area, and this is where, I've been doing my thing for probably the last ten or fifteen years. Yeah. Fantastic. Thanks so much for coming. So, look, I think this is gonna be a bit of a back and forth today. I have some questions. I know you have some questions for me, so let's go ahead and kick off. I'll throw out the first one. So if you had to summarize the Geniei maturity curve today, we're in the bank where is the banking industry, really? Okay. AI, that's a very good question. And and before I answer it, maybe we spend, like, thirty seconds in defining when we say GenAI, what what it what does it mean? Because it means a lot of different things to different people. So if you think about AI in general, there's like a maturity curve. Or in the old days, we had RPA, robotic process automation, but these days, talk about GenAI, which is more around where I can generate content, code, or insights instantly from natural language prompts and agentic AI, where I can understand business goals and I can autonomously manage workflows of, you know, business processes to achieve them. Then there's general artificial intelligence all the way to up to artificial super intelligence where a lot of people are working at it right now. In the current market, especially in banking, we're somewhere between Gen AI and agentic AI. Meaning, Gen AI in the form of chatbots that we see either in POC stages or some of them move to production, and agentic AI in the sense of automating workflows through an agentic force. Now where is the banking industry? I think that when you look at statistics, the majority are still in the POC stage. So different, you know, different analyst firms are covering this, but probably more than sixty percent today are investing in in GenAI and Agentic AI. The majority, so probably fifty out of those sixty, are probably still in the POC stage. Eight to nine percent are moving types of use cases to production. Yep. I absolutely agree with those stats. And, look, I think it's a great way to put the scale or the maturity curve from Jenny and I back towards robotic process automation and towards superintelligence. That's definitely where the industry is looking to go. I don't know when and where they'll get it and what it'll look like when we get there, but that's definitely an exciting thing. Coming back to banking, though. So from what you're seeing in the market, what are the high value use cases that are gaining traction in consumer banking? So across customer experience, risk ops, compliance. Okay. I'll just probably say that what we see, again, is mostly still in p different different stages of, POC stages with different levels of maturity, but it's more around content generation, content management, some synthetic data generation, knowledge search, and so on. Specific ones, you know, if I I think I'll I'll try to sort of maybe build those into specific categories or subsegments. So there's a lot around customer engagement and enablement. I think one of the key things everyone's talking about is personalized banking offers. We'll talk about probably, I guess, more, in the in the in the next couple of minutes. Things like financial health tools, things like loan portfolio management. There's a lot around operational efficiency or operations automation, anything from exception handling, payment data analytics, financial contract management, and so on, all the way into the risk management and also compliance reporting, things like fraud detection, things like suitability complaint, liquidity and cash forecasting, risk model training, and so on. So there really is a very wide net of different use cases. It really depends on where the business challenges of that specific company bank and so on are actually at or where do they they feel they wanna start. I think there's a majority currently towards back end type of use cases versus front end, but literally, it's they're testing it all. Yeah. No. It's wonderful. I definitely see the same things in customers that I'm working with where a lot of the technology is focused on the back end, like you said, where we're doing enablement only because it's less risky than putting a a gentic workforce in front of customers directly today. So and that risk comes, and I I'm sure we'll talk about it during this webinar. The risk comes a lot from lack of governance or immature governance frameworks and, you know, the data not quite being where it needs to be. Yeah. And, you know, so maybe a follow-up question for you actually on that one because I know you're seeing a lot of our customers, especially on, you know, on the US side. And there's, like, a perception that most Gen AI initiatives are still sort of trapped in POC stage. So I know I I see it, you know, globally, but do you think it's also true, when you talk to our customers? And if you do, do you have any, like, data points or ratios that you're actually seeing across the industry? Yeah. Absolutely. So, look, that's POC purgatory. It's a very real thing inside of banking, but not for the reasons that most people think. So across the programs that we support, we see about sixty to seventy percent of initiatives stuck in the POC or pilot stage. Now it's partially because things are stuck or because they're moving so quickly that there's a lot in pilot that they haven't had the regulatory mandate to move up the stack a bit. But twenty to thirty percent is in limited production or in one business line. We're finding that areas like capital markets tend to move a bit faster than the enterprise banking. Again, points back to a lot of regulatory controls, also points to just the nature of their business. And then five to ten percent are truly scaled across the enterprise. So very few use cases have actually been fully adopted across all areas inside of banks. But the bottleneck isn't mono performance. It's data readiness, governance, integration with real workflows. Most banks can get a chatbot or summarize your live in weeks, but what they struggle with is operationalizing the reliability securely and repeatedly across the bank. So what's changing now is that leadership teams are asking, what is the ROI? So they're looking for things that are generating actual revenue, actual return for the work that they're doing. You know, I mentioned that a bit of the work that we're doing is in enablement, and it's in enablement because we're shifting from legacy systems into modern systems on the back end to support this generative AI push within their business. But at the same time, we're using generative AI, we're using Agentic specifically to help support the refactoring of these legacy systems. Because it could be a twenty, thirty, forty year old systems that the developers that originally built it are long gone from the business, and there's no real documentation that follows. So traditionally, taking time to analyze those systems, build a plan for them, start migrating the code and refactoring into something modern would take years to accomplish. These days, it's taking months, not years. Alright. You know, you mentioned ROI. So I wanna maybe just expand on that a bit, maybe zoom out. And instead of ROI, maybe I'll call it value. So what's, like, the biggest misconception that you see today, either from banks or from vendors, doesn't matter, about where the Gen AI value truly lies? So the biggest misconception I see It's a it's a it's a very big big question. It's a big one. It's a big one, but I like it. The biggest misconception that I've seen is that the value is from better answers or friendlier chatbots. So the idea of hallucination comes up over and over and over, where when you're using chatbots directly, hallucination's an issue. Right? If you have ninety five percent accuracy of a model, five percent of the time, it is making stuff up just to satisfy you, and that's where hallucination where hallucination comes from. But it's not where the real value lies. The real value in generative AI and agentic, even moving to AGI and ASI, it's from the operational leverage. So what can Gen AI do in an organization? The idea of hallucination, just coming back to that quickly, can be diminished with the way you architect these systems, the way you put them together, the controls that you put in place, even having what we tend to implement a lot is agents validating agents. So you're not getting the first shot response from a question that you ask. You're actually getting the ninth shot response from the question because we've put it into an iterative loop to validate itself. And that might be using the same model for both agents. It might be using different models for both agents because each of them has a different capacity, a different ability. But we're putting those checks and balances in place, Very much like if you bring on a brand new college graduate to do an analyst job and they're working through an analysis, you're not gonna take that analysis and put it in front of a customer. You're gonna have multiple checks and balances before the output of a college graduate ever gets in front of a customer because you're gonna wanna teach the college graduate or that brand new intern how to do the work, but you're also wanna make sure that what he's doing, you're leveraging, improving, and validating, and putting in front of the customer. So we're taking the same approach to the way we're implementing these agentic systems or agentic pods. So I love the few vendors. Comparing you're comparing agentic to college graduates. I think that's an amazing analogy. Sorry. Go on. It it it's it's accurate. It's accurate. And we're finding that it's very accurate. Because day one, these agents, they have the ability to reason. Very much like you come out of college with a degree, and really all it tells employers is you're able to think. Like, you have the capacity to think. Pretty much what a college degree does today. But they have no context of the business. They have no context of the industry. They have no context of anything. They can just reason. They can just think. If you give them a problem, they'll give you a solution. Whether it's right, whether it's fit for your company, all that still needs to be taught into them, and it's the same thing with agents. Right? Alright. So, you know, let let's speak a bit about agents. So we talked a bit about trends and what's happening in market. I wanna maybe start deep diving into the topic of why we're all here today. And that's a good bit about, you know, when you try to think about implementations of agentic AI. So when a bank decides to to move from Gen AI chatbots to Agentic AI system, what fundamentally changes? I mean, what are, like, the nonnegotiables that we need to think about? So first and foremost, it's the mental model changes. Okay. You know, I mentioned earlier about how we have to switch from thinking about friendlier chatbots that have better single shot responses to actually trying to figure out where the value is in changing the operational leverage and implementing it into real workflows in the business. Well, that is the first thing that needs to change. So that mental model, you're no longer designing prompts. You're no longer looking for the best explanation of something when you're designing a system, when you're implementing this stuff. You're now looking at LLMs not as a tool but as an orchestrator of tools, as an orchestrator of workflows. So what you're doing is instead of designing a single prompt to get the best answer, you're designing a whole system. So policies, routing, logic, memory, verification, escalation paths. Escalation is a very important thing in these systems because when hallucination does come up or when there's a problem that your virtual college grad can't answer, they need to be able to escalate it and bring in an expert in the loop to give them the right answer and guide them in the right way. So the requirements are really shifting from accuracy of a response to reliability and governance. So some of the things that we're implementing for guardrails, or just in general in our systems, actually, are deterministic guardrails. So we know what good looks like. Let's put the guardrails in place. Let's have a validation bot checking the responses so that we can get to the right outcome. It might take three iterations, it might take six iterations, but we will get there in a time faster than typically a human would be able to generate the same outcome. Transparent reasoning chains. You know, I mentioned trust and governance earlier. I've talked way too much about the history of ML lately, but the first five years that ML was out and it was providing outcomes, it was providing predictions, people didn't trust it because it was a black box. So no one actually scaled ML for the first five years until they realized that the ML was providing a better prediction than the systems they currently had. So we can easily fall into the same trap now with Agentic and with Agents, but what we're doing to go around that is having the transparent retraining chains, being able to enumerate why the agents are making the decisions they're making. So it does two things for us. One, it allows us to follow the logic and see if the logic is sound. If the logic isn't sound, then we can always tell them where the logic fell over, tell them to remember how we fix the logic so that they can get the answer right in the future. So it comes with clear audit logs, being able to see all that, having identity and proper permissioning, and then fallback logic when the confidence drops. So what we typically implement with ours is a confidence threshold. So depending on what they are executing on. Let's say it's a low risk activity. Let's say it's a code refactor, as an example. We will set an eighty percent threshold, which means that if the agent is eighty percent confident based on its, its cycle that the deterministic plan that it has created, excuse me, is going to work, it will execute it autonomously. If it's seventy nine percent, it will escalate it. And that way Okay. We have confidence, and we know that the agent has confidence in the actions that it's taking. And if it doesn't have that confidence, again, it's escalating to a person. And then I think the last thing is what really becomes the hard part is integration. You know, we have between twenty to fifty existing upstream and downstream systems between CRM, case management, core banking, know your customer, claims, loan servicing. You know? And these are systems that, in today's day and age are very heavily controlled. So coming back to that trust, coming back to that governance, coming back to the observability and the safety that we have to have around the agents, all that needs to be in place to even integrate into these systems. And without that integration, we're not gonna see true value, true end to end value from having these agents working in environments. I I agree. And maybe I would add one more thing. The more agents you have, you start you need to start probably also thinking about access control and who gets, you know, who gets, entry to what type of information and so on as you would in any other enterprise scenario even today without Agentic. No. Absolutely. Great. It's interesting. That challenge is one that we've faced into quite heavily. So traditionally, when we're talking about agents and I can't even say years ago, three to four months ago, we were talking a lot about rag based systems. Right? And rag based systems are great. They give the data to the agent to be able to do its job, bring in all of this information so that it can quickly respond. The problem with rag based systems tends to be that we are softer with our security controls around this huge rag database than we are around the source system data, and it creates a huge attack plane for bad actors. Yeah. So what we've actually been shifting to with a lot of our systems, we still have RAG, and we have RAG for data that is nonsensitive but also not fast moving. But for anything that is fast moving data, because you don't wanna have live streaming to your RAG vector database, anything that's sensitive, because you don't wanna be pulling finance data, trade data, HR data into this Rag database because then it be just becomes, you know, a definite attack surface for people. We've gone to multi agent Rag. So keep some of that Rag data in there if that makes sense to keep there. But then for the areas like HR, like finance, like trade detail, actually develop an agent that sits in front that is your guard or your access control agent that that agent has access to the sensitive system, but then is summarizing at a level that makes sense for the request and the user behind the request. So that way, you're controlling that access, and you're controlling that data flow, and you're auditing that data flow as well. Yeah. Just adding that extra level of security between the user and that sensitive data so that organizations can, one, pass regulation, but also be confident that what the agents are working with is what they should be working with. Yeah. Makes sense. Okay. So so, look, on the topic of data, what are the true, like, data and cloud table stakes for Agentic AI? And which of those are hygiene versus which are true dent differentiators for organizations today? Okay. That's actually a very good question. Let's start with hygiene. That's probably easy. You know, one of the key probably the key questions that a lot of CIOs and CDRs are dealing with today, is is our data estate ready for prime time? Prime time being Agentic AI in our case. Right? And the answer is that the data state today, and I don't think it comes to a surprise to any of us, is fairly fragmented. There's machine learning tools. There is a lot there's a bunch of, generative AI beginning of of, you know, starts and so on. There's a b BI tools there. There's streaming applications. There's data warehouses, data lakes. In fact, you know, one of the CDOs probably two weeks ago said, I have several data lakes. In fact, some of them have actually dried up. Now they're data swamps. And all this needs to be in a certain level of quality and confidence before you apply an agentic sort of flavor to it. Because, you know, in the in the data world, we used to say garbage in garbage out. But, honestly, in the agentic space, it's garbage in a thousand times garbage out. And it's a sort of a nightmare scenario for CDOs to have agent an agentic implementation act upon bad data. So hygiene is, you know, is key. Now we can we can look. Do we need hygiene in a clear quality of data for a specific use case? Do we need it across the entire state if we wanna go full scale enterprise and so on? These are discussions that are happening literally in rooms as we speak. But I think it goes beyond that. Right? Because if you think about what needs to be in place in order for an agentic implementation to be successful, then there's data itself and the hygiene and the quality of it. But we need to think about a lot of things. I think you touched upon a few, in in in your previous answer, but things like you wanna have processes and tools that can manage the AI agent teams, whether it's security tools, data tools, and so on. You wanna have some sort of a management platform. You wanna be able to fine tune the AI teams and roles as you actually go for it, and that's all done through data. The the key thing that I think, at least we've been talking to to our customers about is the governance and and the AI lineage. Because as you said before, these are sort of college grads. They're gonna make mistakes. The question is what's gonna happen when that mistake is there? How can you roll back quickly, trace where the mistake was, and retrain the model? If it needs to be retrained again and again. I think that when you look at this and maybe have even agent driven governance, that has human oversight, that's a key thing that is a differentiator between a good implementation and one that would probably be challenged down the road. And the last thing is what type of data, once you have it, is accessible to the AI teams? Is that, like, a data lake and lakehouse model for structured, unstructured data? Is it a comprehensive data model and, you know, in a tool catalog that you can leverage? So it goes beyond is my data, you know, at a certain quality or not? Also, what's the surrounding element? And I'll throw another one at you because we when we talk about data, we seem to sometimes forget what's you know, what data resides on, and that's actually cloud. Now cloud supposedly is not an issue anymore. Right? Because, essentially, everyone's on cloud, microservices, you know, they've modernized the applications and so on. But I would argue that when you look at the cloud platform maturity, is there a developer environment? Do you have your cost and FinOps and observability figured out? Is there, like, integration and delivery figured out, centralized monitoring, logging? You know, you wanna have that cloud that cloud space with, like, unified control plane, maybe a global policy engine and stuff like that. You wanna have those in place as the data resides upon them. And I think these two, like, are the bundle that then makes the the agentic implementation what it is or what it should be and gets you that level. Because if you have that right, then you can have then on top of it a data and ML layer, and on top of it the agentic framework itself. And then you have this trio of layers working in Unition. I think this is where the actual competitive advantage is. It gets you the speed, the accuracy, the quality, and the ability to fine tune and roll back quickly if and when you need it. Yeah. Fantastic. And, look, I completely agree with you. Cloud is fundamental to to getting all of this right, to getting data right, to having all that automation around it. What I find interesting and enlightening, especially around FSI and it's not just, honestly segmented as FSI. It's across most industries today. There is a shift towards on premise, and it's not on premise as it used to be. It's on premise in a highly automated way. So trying to get what they used to call private cloud, which used to just be a legacy on premise Yeah. Into the future state, which is a highly automated state to almost match the public cloud providers. Right? Yeah. And I I agree with you. Things like global policies around security and governance, but that means having that policy stretched across public cloud, but also your own private cloud and your own automation. Because Correct. Like, in FSI, one of the interesting caveats is the security frameworks tend to be more focused on on premise or private cloud can touch your public cloud, but not the other way around. The other way around is always an exception. So there is definitely, like, that need in the cloud space to have that highly automated, rigorous path to production, but also having the right things in place, having prod like, parallel data between production and nonproduction to sanitize, being able to use Agentic to build those datasets in nonproduction to match the production volumes, types, and and variability so that you can do proper testing with confidence go to production. So Alright. Yeah. Absolutely interesting. Look. We're, like, thirty minutes into our discussion. Yes. Let's put some examples in there. Okay? Because I think we're we're probably sharing a lot of theories and what we see in the market, but, you know, you've been dealing with probably most of the cutting edge type of projects that we do in that space. Can you maybe share or walk us through, I don't know, a recent example where, like, we had, agent based automation that materially changed the outcome. So it wasn't just a nice to have. It really made sort of a dent in the way we work today. Absolutely. There's a few different use cases that I could run through, but I think the one that I want to today is vulnerability management. So vulnerability management is one of those things that's been foundational for ITSM for a long time. But what we find is that large enterprises tend to deal with the really critical stuff, but everything else gets put on the backlog and move through. So one of our clients today has around two point five million vulnerabilities across their entire estate. Wow. Working with they're working with, governance organizations to work through it. They have a moderated plan to work through it. They have regular checkpoints. But obviously, all this adds overhead, risk. They have millions of vulnerabilities, and they're continuing to push some of these and progress some of these. You know, some will be replaced as they upgrade applications, but new vulnerabilities will be added. So we've recently gone through a piece of work with them just to identify what the ROI looks like to work with them. And one of the eye opening things was if you assume that you could take three hundred fractional employees, so FTE, so full time equivalents. Right? It might be spread over a thousand developers. It might be spread over three thousand developers. Those three hundred FTE would take something like seventeen years to actually resolve the amount of vulnerabilities they have in the environment, which doesn't make sense. Right? Correct. Totally. What we did was we worked with them on a model where we can scale up the agentic pods to focus on vulnerabilities and not even go, yes. We could take a hundred percent vulnerabilities because that's not accurate. Right? So we assume that we could start at thirty percent of the low hanging fruit, the things that are defined, the things that are known, looking at infrastructure, looking at middleware, and ultimately scale up as we continuously train the agents, as we refine the agents, as they get to know the environment better because we've implemented a shared memory for those agents to write back into so they continuously learn off of the work that they're doing, the vulnerabilities or closing, whether it's been successful, whether it hasn't been successful. So we have a model where it scales from thirty percent benefit up to seventy percent benefit still with a workforce that is scaling back. And what we found is by year six, we could actually close every single vulnerability that they have while scaling that FTE workforce down from three hundred equivalent to sixty equivalent. So significantly dropping the amount of human capital that needs to be invested in just closing vulnerabilities, but also resolving their entire backlog and going into a maintenance mode rather than a put out fire mode. So that materially impacts the way that they're working with their regulators. It materially impacts the amount of effort that they're putting towards this initiative because today, you're basically moving three hundred FTE to deal with vulnerabilities, tomorrow, what they'll be able to do is take those pardon me, my math is slow this morning, two forty freed up FTE and focus them on high value outcomes, focus them on development efforts, focus them on future product enhancements rather than closing security gaps. So productivity goes up in a in a big way, essentially. Massively. Yeah. I mean, you're essentially freeing up two hundred forty example of a of a material of a material change, yeah, or a material outcome. It it's definitely a material outcome, and that's really just from the productivity point of view. But having the regulators come off and go, yeah. We're comfortable. You no longer have to report to us. You no longer have that overhead. Don't have that overhead. You don't have the fines from the regulators that you're continuously paying because you're not technically in compliance. Right? So there's there's a few different buckets that that ROI comes from, which which is awesome which is awesome. It's great to see that scale of change just by implementing this one use case. Like, it's just vulnerability management. And then there's opportunities, obviously, to scale that across. So look at before vulnerabilities even occur, you know, how to use Agentic to validate code, how to use Agentic to red team the environment to identify vulnerabilities that they don't even know about and close potential security gaps that bad actors could take advantage of. Right? So there's multiple areas that we'll be scaling into just taking that ROI from this one area, which is an enormous ROI, and being able to shift some of that value into developing more use cases and developing in areas that materially shift the opportunities that the bank has. Yep. So look. One of the things that we run into, and I'll be completely frank with this, is governance. Like, governance is difficult. Yes. Governance often lags innovation. However, governance with agentic AI is a key topic across more organizations today. So what role do you see governance playing in agentic, and how can organizations hedge their governance up with their innovation? Look. Governance and innovation in the same sentence is always, you know, is always a challenge. Okay? Let's be honest. I would say, though, that especially in agentic AI, when you when you're really pushing the envelope of the organization, if you have innovation and you don't have governance think of it like innovation without governance would spell chaos. Right? So, usually, governance in at least every innovation project I've been part of comes at the end as a bolt on. Right? So you run the whole innovation thing internally. Everybody gets excited. And before it goes, you know, goes public or a couple of months before, you start introducing to the compliance teams, governance teams, and so on, and then you get, you know, some feedback. You have to go back, change a few things, close a few loops, and so on. It won't work like that with Agentic. Right? With Agentic, I think that what we wanna look at is you need to bake governance from the get go. It has to be a place where, essentially, you think about governance, not as a bolt on, but something that is already there from the get go. You wanna define the ownership. So, for example, who approved the data sources, the prompts, the models, the actions? Right? What's, like, your model risk management? Right? So do you have an inventory versioning, validation cadences, you know, clear accountability in in these types of elements? This is crucial. Do you look at ethics and transparency, which is also a big part of governance, especially when you talk about agentic force because there's an there's a bias there. There's a there's an inherent bias in in an LLM that you need to teach again and again. So is it traceable? Is it explainable? Users basically should know, you know, why a decision was made and then act upon it. So, specifically, in the case of Agentic AI, I try to think of governance as, like, the, the best analogy I can give is governance is is like the oxygen that lets innovation breathe safely, and then you can really do a lot of stuff, and it has to be there from the beginning. Otherwise, you're gonna you're gonna get yourself into a whole world of challenges as you're about to launch a new agentic experience. No. Thank you for that. I completely agree with you. Like, governance governance, if you do it right, almost protects these initiatives and lets them grow. Like, it is that oxygen. It is that food for them. It's funny, though, what you said at the beginning about initiatives getting to the point where they're almost in production and then go into a governance board and security board. I I feel like I've been through that fight so many times where do do you remember Monty Python, the holy grail, the black knight, where he just gets Yes. Kept kept getting the lips cut off? Was like, this is just a flesh wound. I felt like that early in my career that I was just like, this is just a flesh wound. Now I know a lot better, and definitely security governance needs to come in day one and be foundational to anything that we're delivering. So besides governance, where do you see banks typically getting stuck when trying to scale Agentic beyond the POCs? Like, what organizational blockers matter more than the technical blockers, and what best practices have you seen to overcome them? That is an excellent question. I I'd start by saying that we're all very focused on the technology. How are we training the models and what we're gonna do and what the agent's gonna look like and what does a two a look like and what are the access controls and security. We keep we kept talking to for the last, I don't know, forty minutes or so just in our discussion. But if you try to think what makes a good implementation really scale across the enterprise, there are there are a few more items there. So one is data. We talked about that. The other one is governance and everything that has to do with that. And the third one, I think, is people. And I don't think we spend enough time thinking about the human fact. And I'll give you a couple of examples. Right? Think of it this way. If you look at technology adoption and you don't have human adaptation, it's gonna fail. So if you really wanna make this a success, you need to think about things like upskilling. Right? You wanna upskill your people on, I don't know, prompt design, agentic orchestration, evaluation frameworks. You have to build those fusion teams where you have on one end the technical resources, right, the data scientists and so on. But you wanna bring in the domain experts and the compliance teams and the operations, and you really co create those use cases. And then you wanna probably have AI champions. I think that's, a very good best practice I saw in several of the leading banks across the globe that they have AI champion inside the business units who connect the innovation with the real workflows. Because as we said, ROI and you wanna we wanna actually come up with a use case that means something that creates that material impact, these are the key things that you have there. And, you know, the best gen a Gen AI systems or Agentic systems aren't really smart. They're also being trusted by the people that shape them because there's a high level of confidence in how you actually, how you actually develop and deploy them. And there are two other items there still in the sort of human realm. One is what you don't measure, you can't really scale, especially with with everything that has to do with, with Agentic AI. There's always the question of, so what did they do? How how well did it work? Especially by the executives. Where's the business case? Where's the ROI? So you wanna actually define those success metrics early, whether it's productivity, whether it's experience and engagement, whether it's risk and compliance, whether it's you know, there are now we're starting to see, and I'm happy to talk about it maybe at at another time, but agentic specific KPIs, things that are relevant not just for in in the class of things that you think about NPS or AHT or stuff like that, but really agentic KPIs for the agentic era. And then you wanna make sure the executives see the Gen AI impact in the same dashboards that show them, revenues and risk and so on. The last thing here probably, and and this is the best step I I I think I got from one of the CIOs in in Asia Pacific. I just came back from a two week roadshow there. He said, dude, you don't go from zero to autonomous overnight. And that's something we it's not a flick of a switch. There's there's a whole sort of we use the word journey a lot. Right? You start with assist. So think like, read only chatbots for knowledge retrieval. Then you move into Orchestrate, which is kind of what we're talking about right now, which is agentic workflows, workflow automations, and so on. And then you move to bounded autonomy, which is like policy anchored agents with guardrails that run specific tasks. And each time you wanna pass between one of those phases, it's not just a technology gate. It has to be some sort of a combo or combination between a technical and ethical, human and operational. And only when these align, then you really get that progress. It's when trust and maturity actually align together is where you get that clear indication that you can really move to the next stage. Absolutely. And look, there's two common approaches. Most of the organizations I'm working with are either focused on use case led approach or a framework led. So do we get ROI day one and then figure out how to build what's behind it, or do we build everything you know, you build your guilty castle and then start building use cases on it? How should a bank decide which path is right, and when do the approaches break down? I don't think one is right or wrong. I think they're both super valid. It really depends what you're aiming at. Right? The use case approach, I think it's good for testing the water, show the capabilities, get a quick win with the business teams, without boiling the ocean. So if you think about the amount of effort required to, you know, get a use case up and running from a data and cloud perspective and governance, everything we just talked about, is significantly lower than going on a full blown framework implementation that then builds upon it or so on. Having said that, if you think about framework approach, then you're looking at something that actually scales. Right? It creates consistency. It's easily redeployed. It's it's really a if you think about it, the foundation across many use cases. That's super important as you wanna go into a full blown enterprise deployment. Now if you wanna do just use cases, and I think this is where most most of the POCs are today, they're looking at either a a use case or several use cases under one category and they're running those, which is great, by the way. I think that's good because it creates a learning curve. But as you do these and then you do another use case and another use case and there's no commonality or framework that actually helps you, you know, create that consistency, what you're actually doing is you're creating the silos of tomorrow. Because in a couple of months, years, depends on how quickly you wanna progress with this, you do want these use cases to start connecting to one another. You want these specific agent that were purpose built to start talking to one another. And then you have to break some of those silos in order to do it, which is, again, the the classic game of let's build something then break the silo because we need now consistency. If you're looking at Agentic framework from the get go, I think it creates that foundation. And if you agree on the basics, which is your master prompt contract and the way the foundation is and the customizations and how you actually create that agent to agent interaction regardless of what the agent does, by the way. Right? And you apply the governance and everything else we just talked about. On top of these, you can run practically any use case, and it's pretty much guaranteed that they all work in Univision if and when you're gonna need them to. Now there's another interesting thing that if you zoom out even more because we're super focused on how do we do this in our organization, front office, back office, operational efficiency use case, revenue generating use case, doesn't matter. It's all within, you know, our own capabilities as an organization. But what's gonna happen pretty soon, if you think about it, is think about something called collaborative autonomy. That's some that that's the term I use a lot. So as you zoom out from your own specific bank, you're gonna be doing things with other banks. Think like post trade exception management. It's probably the one of the classic operational efficiency, use cases that we see out there in the market. You need to talk to a counterparty who's in a different bank or a different counterpart. You might be talking to a fintech. You're you're talking to your customers. There's always a regulator somewhere in, you know, in in that ecosystem. So essentially, what you're creating would be beyond what you're doing in your own organization, you have agents talking to someone else's agents, talking to the regulator agents or whatever. It's a whole new ecosystem. If you're going with the framework approach, it's gonna be easy for you to actually be flexible and move that around. If you're going at the end of the day, long term strategy, use case by use case, you're gonna be somewhat more rigid, and that might be a big challenge if you wanna work and be a may a meaningful player in this now ever evolving ecosystem. Yeah. I love that guy. A lot of the conversations I've had recently with customers has been and I repeat this word for word. Use the tactical to build a strategic. I just pick those initial use cases to get ROI, to get funding back, and then leverage that funding to go build a strategic framework. Because where the value that I've seen really comes from is start getting this to your frontline. You know, the technologists in the back office, the managers, they know twenty percent of the problems that are actually being faced by the frontline. The other eighty percent of use cases live on that frontline. And if we're not putting it in the hands of the actual operators of the frontline employees, then we're losing a lot of that value that could be had. But in order to do that in order to do that in a way that it's not just a POC, it's not just an unregulated pilot, we need to have that framework, and we need to have that path to production even for the frontline use cases. So I love that. Alright. So, you know, I'll I'll connect that with probably my my question for you. And, again, I'm looking for best practices. So I'm a bank. Right? I wanna be production ready with a I had that specific comment made to me by, one of our customers. I wanna be production ready with the Agentic AI within twelve months. What are the and here's where I'll add my question. What are the, I don't know, four, five things that are must have you need in place in order to meet, you know, set your timeline? Just just throw that one out at me, guys. But the first thing is something that you touched on earlier, which is a unified data layer with explicit contracts. So you don't have to have perfect data. That's my view. A little bit of hallucination, a little ghost in the shell isn't going to break these more advanced agentic workflows. But you do need to have accessible data, governed data, catalog data. You need to have lineage PII class classifications. You need to have multi agent systems in front of it to make sure that the access controls are being respected, and you have to have business semantics aligned to target use cases. So when you start having that, then you can finally say, yes. I have a unified data layer that's supporting my agentic workflow. Once that's there, because that's foundational, you then have to build an agent safe architecture. So have isolated execution layers, role based access, deterministic guardrails, model observability, and those policies for escalation to bring humans and experts into the loop. So having observability is a big thing in my view because of the trust that it builds. You know, we talked about security and governance earlier and the idea that governance is that air that allows for this innovation to grow. And it's one hundred percent true. Without that governance, without observability, without the right architecture to give that, then trust starts eroding in these agents. And then once trust erodes, usage drops, right? Trust needs to be there for adoption. So without the right agent safe architecture, you won't have adoption, you won't have trust. Regulators won't allow it in some areas where you'll lose huge benefits that would potentially be there. I think the next aspect so once you have your unified data layer, you have your agent architecture that you can scale on, then it's about ownership. So bring that people back into the loop. It's not just about the technology. You need to have business, technology, and risk and compliance all at the table, having the conversation together. So business needs to find the outcomes. You know, what are we looking for? What are we looking to automate? What flows in our business are incredibly manual but low low criticality and low expertise manual. Like, where can we add this automation that isn't RPA, that isn't fragile like RPA has been, but is robust and can do these jobs and actually think about what the right outcome is. That's what business' role is to really define those things that are gonna push the needle. Tech needs to build the system. Tech ultimately needs to own the architecture, own the automation, build what needs to be built in order for this to remain safe because that's where tech has played for a very long time. And then risk and compliance needs to set those operating boundaries. You know, I talked a little bit about the confidence thresholds. Those weren't set arbitrarily. Those were set in compliance or in conjunction with the business and with risk. And risk set the absolute minimum thresholds. Business when, yeah. No. We can't trust it yet, so let's lower that threshold a little bit. And, ultimately, they'll raise it as the models prove themselves, as the agents prove themselves, as automation continues to happen. They'll loosen the grip a little bit to raise that threshold so they don't have to be inserted as often as they are today. So most failed programs skip that alignment. They skip that alignment between the business and the tech and the risk and compliance. They either approach it from a tech side ignoring risk and compliance in business or business decides to go out, with a credit card and go, yep. I'm gonna buy this system. I'm gonna build my use cases on it, which is that dark IT or the shadow IT that's been happening for years anyways. So now we have alignment. The next thing is the repeatable factory model. So as I said, most of the value you get comes from these things being in the frontline employees' hands. So the ability to automate their own processes, to automate what those flows are. That factory model needs to be in place so it's repeatable, so it's governed so that you can quickly get from idea to pilot to production with these ideas and start getting real value from them. My recommendation for every customer I talk to is figure out three to five high volume use cases. You know, you might have trade exceptions, know your customer, claims, collections, lending documentation, whatever it may be. Focus on those at first. Work with compliance. Work with the business. Focus on mapping all that out, automating them, recognize the ROI from them while you build the strategic layers underneath. Because the strategic layers underneath are gonna be what you need to scale everything, but at first, you can only do so much with what you have until people are ready, until your framework is ready, and then you can open the floodgates. I have a customer that recently opened the floodgates just for one of their divisions. And, within, I think, three weeks had eighteen hundred new use cases built on their agentic framework. Now not all of those are in production yet. They're still going through, like, pilot and POC and validating and tuning, but that is what it looks like when you open to the front line, that you have a lot of these use cases immediately pop up that you can then work through and identify the ROI and drive from the frontline so that tech can focus on the repeatability, the framework, the governance, the security, the controls, while letting the frontline business do what it needs to do to automate away a lot of its more tedious work. Yep. I I I agree, and I would add also, you know, business sponsorship is huge. At the end of the day Yes. We're not doing this as a technology exercise. We're doing this to improve either operational efficiency, productivity, and bottom line that way or create new revenue streams the other way. So whichever it is, there's always gonna be a business involvement significant to to to look at. And the earlier we get them excited, the earlier we have, you know, this meeting of the minds on what really are we looking to work with, I think that would go a long way in in making that implementation also very successful in the period of, let's say, twelve months. Yeah. Absolutely. I'm just Companies I work with no. I was gonna say companies I work with all the time, executive sponsorship is incredibly important. If it's not coming from the top down, then groups like risk and compliance are going, no. But we haven't had a mandate to actually do this, so we're just going to put the brakes on everything. And then the business goes, we don't have time to deal with giving us back more time because we have our day to day tasks. Right? So I agree. Like, it needs to be from the top down mandated, but then it needs to be from the bottom up adopted. I like that. I really like that. You know, I think that's a great that's a great summary. Just looking quickly at our sort of q and a. Guys, if you have any questions, now would be a a good time to ask them. We probably have, like, five minutes till the end of the, of the panel. So if there are any questions you wanna ask or things that you think we've missed out on, happy to get those comments and questions in. Alright. In the meantime, until we get those questions, I'll just Can I yeah? Go ahead. Can I can I expand on one of the ones that was answered and you quickly messaged back? So another agentic AI will be implemented for the governance. Thank you for submitting that. This is something I'm actually really passionate about, the idea of having supervisory agents that sit across the entire stack. So when you actually scale this framework, you wanna put agents that are looking at everything holistically. So the idea of having domain agents is one thing, but having those ones for governance, for security, for FinOps, you know, looking at these agents, looking at the flows, looking at everything that's happening, and making sure that it is the most effective use of token count and coming back and giving recommendations and giving tuning suggestions and iterating on the agents as you move forward so that the same response or the similar response next time only costs half as much because you've tuned the way that it's looking for requests. You've tuned the actual token counts as part of that. But same for governance. Governance, all the agent to agent interactions, we need to have a governance supervisor that is looking at those interactions and ensuring compliance across them, ensuring that when you are going cross domain, even like I mentioned earlier, when you're going cross organization, did we hold our governance and regulatory compliance controls while we went cross domain or while we went cross organization because those controls are very, very important for organizations to recognize. Yep. So thank you for that question. Yep. So, Paula, I think we're almost at the top of the hour. So I just wanna take a moment to, thank you for, for answering, those, questions I threw at you. I wanna thank Isabelle and Isabelle for hosting us. And if there are any additional questions, we'll take them offline. We'll we're happy to reply them to reply to those. If there's a way to to get back to you guys, we'll happy, happily send you, our feedback, our thoughts on on your questions. Absolutely. Thank you guys so much. Thank you, Paul. Thank you, Guy, for joining us, and thank you to all of our attendees, who logged on this morning. And with that, we will conclude today's program. The recording will be will be available in one to two business days. And on behalf of CBA, thank you again to our speakers and everyone who joined today. Have a great afternoon, and you may now disconnect. Thanks, guys. Thank you everyone for giving us this opportunity.</p></div></div></wistia-player>