The generative leap: How AI is redefining Big Data

So my next guest on the great indoors at MWC twenty twenty four here in Barcelona. He's come all the way from Atlanta. He's the EVP for data at AT and T. Mister Matt Dugan, welcome. Thank you for having me. Yeah. Thanks for being here. So we're we're approaching the sunset almost of the show Yes. This year. And before we get into what you've seen at the show and and everything you do, just give our listeners a history of yourself and your role at AT and T currently. Okay. Well, like we said, my name is Matt. Matt. I've been at AT and T now for a little over fifteen years in aggregate. Yeah. This is actually my fourth time with the company, like with the family of companies. So over a twenty five plus year career now, I spent over half of it with AT and T, part of the ATT thing. Yeah. And I've done a lot of roles. I started in field engineering. So think poles and holes. Yeah. Hand holes and and poles for cables. So distribution. And did that for a while and decided that I wanted to make a shift more into technology and software, my areas of passions that I developed as a young man. And went and got a computer engineering degree and made that pivot. Got brought into AT and T as a developer. Had gone up since then. Yeah. And then gotten to do a lot of roles. I've worked in application development, middleware and services, back end systems integration, getting into data, data and AI, advertising, VARTECH, deploy build CDPs, manage data platforms. And now in in the current state, work on all of the underpinnings of everything, funding our oldest technologies, the mainframes, all the way up to our very latest stuff, which is entirely cloud native and hyperscale. Wow. Wow. And what do you think of the show this year? What have you seen? What's been the talking point for you? Well, is my first time at MWC. Oh, it's definitely been an experience. There's a lot of people, virtually nothing in the way of foot traffic control procedures. Yeah. It is, you know, almost if you've watched a sci fi movie where they have completely autonomous vehicles and they're all going up and they're all at once and they somehow make their way, that is what the experience on the floor of everything you said is for And, seeing it this year and seeing it, everything come together, it's really amazing displays, really amazing products and services that are on on display, a lot of really cool demos. Yeah. Some, I I think it's actually wrong to call these as booths. Some of them are like micro cities. Yeah. Yeah. Yeah. Yeah. Full of full of, you know, independent stages and experiences to go see. And I just walking around and seeing the audience that's gathered around, how many people are so excited to see something loose. Yeah. You know, an animation or a presentation. Yeah. You could tell that there's a lot of energy here. Yeah. Yeah. Definitely. Definitely. And, you know, I've been coming to this show for many, many, many years. And, you know, you always come away from the show with like one talking point, maybe two big things that you see everybody double down on. Is there something that you see in here that you think, oh my gosh, that everybody is, you know, this is big. I mean, going from halls one through eight, we just take one look at the signage. Right. And you see everyone is talking about some level of integration of AI. Most of them are talking about an integration of generative AI. Yeah. But you balance that with what I've seen in the sessions. And when the talk turns into sort of just the general fluff of generative AI, people are getting up and moving. Everyone is beyond that point now. That curve is so steep that people aren't willing to sit to be introduced to generative AI again. Wow. They're like, no. We we we got this. Like, I wanna hear about actual deployments. I wanna hear about edge cases and experience. Yeah. I want to understand how this is applicable in my company and not in general terms. Wow. That's really interesting. I haven't heard that at all from my guests. It Well, maybe not be a popular No. You know, opinion to go spreading around that people are getting up and leaving sessions, but, you know, you said Yeah. It's it's totally right because there's a lot of monotony. There's a lot of repetition. There's a lot of buzzwords and slogans being associated with it that are repeating themselves over and over again. And like you said, you want to see a demonstrable example of clear value that, you know, and new stuff. Well, think there's a difference between the generation or the interest and the generation of hunger. Yeah. Interest is, I wanna hear anything that is out there because I I want to start making an understanding. I want to start learning. I want to I want to know. Yeah. And and this hunger is for growth. Yeah. If I'm not growing here, let me move to the one that can help me grow because that's what I'm trying to do. I already got it. I already know that I want it. I want to get something that's applicable in my space that I can take back with me. Yeah. That's fascinating. That's a really fascinating fascinating insight. And I noticed from your resume, Matt, and and your title of of head of data, you've been involved in the data space for many years. Right? A little while. I mean, in engineering disciplines and then information technology, you deal with data a long time. Yeah. My my job titles predate specialization. Right. So there was a time when we were just called developer, and then we were very happy to be started being called engineers. You know, some of us are actually engineers by trade. Yeah. And so, that was great, but now, there's like every kind of engineer. Yeah, yeah, yeah. Now that we're inventing new ones, now there's prompting. Yeah, Right, like this new disciplines that are popping up, and it's almost a micro segmentation of the overall discipline of problem solving. Yeah. So in that mode of problem solving with technology, we deal with data for since forever. Well, then even some of my first technological jobs were dealing with a database and dealing with information, how to cultivate it and present it back. So it's been a long time in data. It's only been though about ten years in what I would call big data. Yeah. Is petabyte scale data. Yes. Yeah. And and and the reason why I'm I think when my previous job in Amdocs before I came over to North America is I headed up the marketing for the big data division at Amdocs. At the time, this was nine, ten years ago. You took the Right on your time scales, but it was hyped. It was at the top of the garden, a hype spike that cycle. What did they used to call it? The peak of disillusionment before it went into the trough of The peak of interest before the trough of disillusionment. That was it. Yeah. Was it. Yeah. That's the model. That was the model. And I always remember being here in in in Barcelona, talking to analysts, talking to press. And and I think there was a hype about big data and just the term, but I don't think people really understood what the definition, what fell under that umbrella. And it was a number of different things. Obviously, the data management systems and, moving from a data warehouse to a data lake and things. And just moving from regular BI to actionable analytics everybody was getting excited about. And then there was the data scientists who were apparently gonna create all this magic with this, you know, swathes of swathes of of data in these clusters. And I remember always being asked, what's the killer use case? Right. What's the killer use case? What's the and and, you know, in in the world of telco, you know, you'd always say things like network optimization and next best offer and, you know, all all the things, improve customer care, all the all these things. But then it this was my own proclamation, by the way, of coming coming up with this. But then when generative AI reignited the hype about AI AI wasn't created on the thirtieth of November twenty twenty two with it existed what, nineteen forty seven in in in its but this reign this this this defibrillator, if you will, on on the big data hype came up again. And it occurred to me, and I'm interested to to get your thoughts on this. It occurred to me that maybe generative AI is the killer use case for big data that we were waiting for. What what I I was actually gonna say the same thing. Oh, really? So that if you remember the promise of big data Yeah. Was that there's all this data out there. Yeah. If you're not collecting it, you're dumb. You should. Yeah, yeah. And, well, we can't really tell you why, but we can tell you that if you have it, you're sitting on a, you know, precious metals, like gold mine. Yeah, yeah, yeah. Your ability to transform your business, if you only had it. Oh, but isn't that a lot of work to collect all the data? Yeah, but you gotta do it. Yeah. You don't have all the data, you're failing. Yeah, Okay, fine. So everybody collects all the data. And then it's, well, but don't I have to process my data? Like, don't I have to like cleanse it? Don't I have to look at structured data? No, no, no, no. You don't have to worry about any of that, because you could just data mine. Yeah. Right? Go in your unstructured data, your storage has completely changed, you know, you no longer, at the time, you wanted localized compute and storage Yeah. Which was a deviation from the prior iteration of the pendulum swing of I want a network attached storage and differentiated compute. Right now, turned into I want general compute on top of attached storage Yeah. Which is like interesting. And now we're moving back the other way again, because now we have hyper scale in the cloud, and even though storage is is sort of convinced technologically network attached, you don't get the latency delay. Yeah. The difference though is what became the big data, I would call it failed promise, which is if you have all the data, you will somehow be successful. Yeah. Step one, step two, step three question marks, set up a profit. Yeah. The that question mark is now, really, it's you're supposed to have had AI on top. Yes. Yeah. Yeah. The first thing we did in collection was then realized that what we needed to do was create structure. Now we create structure, but the volumes are so high that putting humans to task against it means that you can get some penetration, some models, and some insights, but you're not necessarily going to be able to create better models and create convergence of insights from multiple different data demands. And it's also still really hard to tell when your data is lying to you. Like, you pulled in all the data. Yeah. Some of it's garbage. Yeah. And or something. There was the Versus, wasn't there? Volume, Variety. That's it. Yeah, yeah, yeah, yeah, yeah. So yeah, sorry, I interrupted what you said. It just jumped straight into my hand. Yep. Yeah. Yeah. And so now on the generative AI side, it becomes, okay, well now I have the reason to actually go through my data and start to pick apart the pieces. Some assistance, right? But pick apart the pieces that really matter. Yeah. And so now the notion of the data races, now it's a data expectation. I'm expected to pull together the right data Yeah. To deliver the right business answers. And I have to enable the business to interrogate those answers. So generative is a great, you know, pattern to be able to support that. No. It is. And and and, you know, it's I I back on some of those other things in that big data expectation and some of the companies that are out there. I remember the Hadoop companies, Cloudera and Hortonworks. Yep. I remember going to AT and T in Dallas with Hortonworks and we were we were talking about big data. So whatever happened to Hadoop? I mean, Hadoop still exists. A lot of the patterns of Hadoop exist. Like, I would say less the technical implementations and more the concepts of a map, shuffle sort, reduce, right? Going through data derivation pipelines, a lot of them exist in data processing patterns today, even though it's not technically a Hadoop family operation. Yeah. But Hortonworks was acquired into Cloudera. Yeah. That's what happened. Yeah. Yeah. Yeah. Yeah. And Cloudera still has a robust stack of these data capabilities, but they're also branching out. Also saying, hey, Hadoop style technology, Hadoop family technologies are not the answer to everything. But there were some things that were solved really well in that ecosystem, like the ability to have a metastore of what we call today a catalog of all the data. Yeah. What metadata is attached to that data? And so, how can I describe this data? The fact that that was available in a centralized fashion, as something that could be produced, was an important concept then that didn't get enough attention for its importance. It was a technical function that was necessary to run the Hadoop lake. Yeah. Now, it's a business function that is necessary to understand what data do I have, what data should I choose for this use case, What data do I want to model against? What data do I want to bring into a data product context? Those are now all functions of a catalog. So the interrogation, that interrogative experience that a business user analyst or a data platform engineering group is gonna do, is now necessary to be done with the catalog. Yeah. Another thing that the Hadoop ecosystem got right was rights management on the data. So having a very mature RBAC who could access and use what data sets, this is something that has not yet been solved well in the generative AI space. We're gonna have to add the layers to solve it. But in doing so, you have to say, well, what kind of result am I expecting? So, as sort of an arbitrary example, if I was to ask you what's on this table, you would look down at the table and answer my question. Yeah. If I was to take another person, cover half the table in in a sheet, and say, what's on this table? They're gonna get a different answer. Yeah. Yeah. Well, imagine imagine in the generative AI view, you can't see that there's a sheet covering half the table. You're just you're only seeing the view the part of the table that you're being allowed to see, and you're and it's being presented to you as a complete table. So that presents an interesting problem, because it means that based on your your permissions, you could get not just a the right answer for you on the information that you're allowed to operate with based on a complete question, but you could actually get an entirely different answer, an entirely different reasoning structure Yeah. Because the generative AI couldn't actually look into the tables that are part of the ecosystem that you're not allowed to sleep. Yeah. Yeah. That's fascinating. And it's it's it's really, really interesting. And going back to generative AI as a whole, did you see it come in? Did you think that we would get there to the promised land with big data eventually? Was it something that that you because it surprised me when I I think generative took took everybody by surprise. Yeah. The the concept of generative has been around a long time. You and I are old enough to remember Eliza. Yeah. Yeah. Yeah. And being told that all our problems ultimately come back from, you know, how was the relationship with your mother? Yeah. Yeah. Yeah. In in a text based console experience. Yeah. And you you think about that being an early concept of of generative. Now, obviously, the engine wasn't using the transformer pattern and such, but it had the makings of the experience. It was a human like interaction that understood natural language and responded in natural language. Yeah. And I I think where generative popped up was all of these things have been possible, but they haven't been possible at scale. Scale and compute, scale of data awareness. Yeah, yeah. Yeah. And so now we're all everybody, if you look at the adoption curve, between one million to a hundred million users on generative AI colliders, it it is you have to zoom really far in. You have to get within twenty four hours Yeah. To see the adoption. Looks like a straight line. Yeah. Yeah. It's actually asymptotic over twenty five period. Yeah. But it's extremely rapid because suddenly everyone wanted to try it. Yeah. Yeah. Yeah. I know the statistic on on that. We had Laura before on the episode, and she said, oh, it took excellent to achieve this penetration. And AI was this. And you then you realize and you go, oh, my gosh. And I think the other interesting thing is, you know, whenever there's any big innovation that's gonna change people's lives, you know, it it starts in in in geographies fairly localized. So even if you go back as far as the industrial revolution in Great Britain with, you know, the mechanization of just building things. The industrial revolution then took two hundred years to reach India. Right? Right. I mean, there was a lot when the Internet first came into the world in the, you know, early nineties, it took a long time, you know, twenty, thirty years to reach Africa. It was only mobile Internet and the smartphones and and and that that allowed those people, even if, as the penetration increased, to access the internet. So those even the what what was the statistic I had from the creation of the telephone by Bell in in whatever it took a long time for for the rest of the world to start using these technologies. And and like Sattaya from Microsoft at Davos said, and within two weeks, people in India are using this, building on it, creating things. And and and that to him was as as significant as the rate of adoption where it was being adopted, which I thought was a pretty interesting insight. And my question is, in your history in technology, have you seen anything like this before? Anything comparable? I mean, in a this has happened over and over again. The time cycles are getting shorter because the platforms are getting more ubiquitous. Yeah. In in prior, you know, you had prior to the industrial revolution, you had a technological boundary that had to be overcome. And then it actually took time to, once crossing that technological boundary at one place, to then manufacture all the things that are necessary to unlock that boundary. Well, in many places. But if you think about the evolution that we had, say, in in Internet and connectivity, you know, prior to proper Internet, you know, we had electronic bulletin board systems. I operated one in the nineteen nineties. Right. And and we would network ourselves together to share messages between each other, so that users that would come in could read the message board and get messages that were actually written by a user of another BDS, right? This isn't the area where you still pay for long distance. Yeah. Yeah. Right. And, you know, the ham radio people have been figuring out how to do these relays and whatnot, so that they could they could get the same kind of patterns. But then, on the Internet side, the accessibility of knowledge, well, like, once you got connected. Yeah. You could get to, you know, depending on where you are in the world, you could get to virtually any knowledge that was available. But then the question became how to find it. So then we started getting a good search. Yeah. Yeah. So, you know, evolving from Lycos and Excite and Altavista, and now being able to go into Google or Big Yeah. Yahoo and search for something, like your results are worlds apart from what they from what they were. Prior to having a really good search, we were like, web breaks. Yeah. Right? So these platform level evolutionary changes, they become available, have allowed people to create step changes Yeah. And adoption curves and innovation curves of things that operate on that platform. Yeah. And the platform change that enabled generative AI is hyperscale architecture with integrated GPUs. It really didn't have to be GPUs per se, but it had to be extraordinarily dense computational layers, that it was possible to put it in something, you know, less than the size of a city. Yeah, same. And so now, it's like, now that platform is available, and someone said, well, why don't we take some of these models that we've been thinking about since the, you know, even from the fifties in Bell Labs with bigram and engram research for identifying clusters of character sets and how that implies into language, or all of the intellectual property that Microsoft bought up, natural language search and natural language processing, and language understanding, multi dialect language understanding that they've had since the nineties. And then someone thought, well, what about this transformers concept? And so you had the paper at the Goodwill and they said, okay, this is how we could interrelate them. Yeah. Yeah. And then someone thinks, what if we just ran this on a massive cluster and we just threw information at it? What would happen? Yeah. Yeah. And so then you get your first foundational model of an LLM that allows it to sort of, through repeated exposure, capture context of language. Yeah. And be able to appear quite lifelike. Sometimes very, very wrong, but still lifelike in being wrong. Yeah, yeah. And so it's interesting to see, well, now this is what LLM adoption has done, generative AI adoption, is to say that now this itself can be exposed as a service, and this itself could be a platform Yeah. For computing against other kinds of use cases. And so, I've seen sort of those platform Yeah. Degrees, logarithmic expansions several times. We've each seen it, but we were always starting from a smaller place. Yeah. Now we're starting from a much, much bigger place, much, much more connected world. And that connected world means that now there's a lot of minds thinking about what's next. Yeah. Yeah. So it's it's quite exciting. No. It's it's incredible. I like the way you articulated it there, Matt. It's it's quite unbelievable. But the other question, is there anything that scares you about generative? And I know that comes up here all the time and there's all these comparisons to sci fi. But I'll tell you what what I heard yesterday and I won't disclose my source on what whilst we're having this conversation, but he basically said, and I'd heard somebody say this in one of the keynotes, was it's only limited by the extent of our imagination. Right? You know, and and I'm I'm gonna ask you the same question in a moment, but I ask all our guests, what are they doing with it? And ninety percent of the answers I get back are unique. So the breadth of these use cases and when the mobile telephone came into fruition, what did you do with it? Well, I made a call, and everyone did the same. But with generative AI, the breadth of how people are using it with their own creativity and the things that they're doing are vastly different based on their imagination or which I thought was an interesting point. But then but then he said to me, I don't agree. I I put that quote to the gentleman and he came back. I don't I don't agree. He said it's limited by the machine's imagination, not people's imagination. And I said, well, what do you mean by that? He said, the machines are starting to create use cases without being pushed or asked to create use cases. And I was like, really? He said, yeah. I can see it. He wasn't worried about it, but I thought it was an interesting point that he put across. What are your thoughts on that and some of the, I wouldn't call them ethical issues, or maybe they are ethical issues, I don't know, but that side of the pace of where we're going. Yeah. It's I mean, I get the get this sort of apocalyptic nature of it. Right? Yeah. Humanity worships the sun. Yep. Humanity evolves. Humanity invents machines. Humanity invents AI. AI invents machines. AI perfects AI. You know, world blows up, humanity worships the sun. Right? Like, this is the loop. I've that cartoon. It's a very good one. So it's provoking in thought, in a sense. Now, unfortunately, we've developed to an extent that it may not be humanity looking at the sun and maybe the roaches. Yeah, yeah, yeah. But I would say I generally agree. What is frightening is not the sense that the machine can operate in and of itself and make some internal reasonings. I I think what what is potentially not well understood is to what extent the machine can make those reasonings that we can't unpack and explain. Yes. Yes. And so, in the LLM space, we worry about poisoning. And I'll give an example. I was using an enterprise chat client. Some bug occurred, and all of my text that I'm typing starts appearing perfectly backwards. As for every character, I then I then pre poned my cursor to the very beginning Right. And then added the next character. Right? So, backwards. And I thought, maybe, you know, I'm in the middle of my workflow. I'm trying to solve a problem. I'm working with people at the moment, I'm thinking maybe this is just on my screen. So I click set. And the response that comes back is, maybe we need to invent an LLM to decode whatever Matt's trying to tell us. I'm like, okay, great. So I said, well, that's interesting. Before I go reboot my computer and try to clear whatever is going on Yeah. Let me take what I wrote. I looked like a paragraph that started happening. And so then I went over to our GPT-four instance that we call inside of ATT, and I said that, and I meticulously backwards typed, translate this text, and I pasted in my paragraph a perfectly backwards input. And it responded as if it didn't understand why it was being requested to translate anything. It says, clearly the author is discussing blah blah blah blah blah, the language is in English. Right? Yeah, yeah. And I thought, well, that's odd. Yeah. So, but I guess, you know, in my thoughts, you know, sort of my naive understanding of what might be going on behind the scenes, it's like, okay, well, if I create all these Ngram clusters of adjacency and everything else, then if I put in perfectly backwards text or perfectly forward text, I'm likely to get the same graph, just inverted one way, you know, from left to right effectively. Right. And maybe there's some variation according to where spaces were and things like that, but, you know, close enough that the variation of just common misspellings and other things could still be satisfied, and it still looks like English text. Well, that's interesting. Yeah. So if that works, then there are probably quite, surreptitious ways of poisoning Yeah, yeah. LLM inputs and and being unable to explain why some result is coming back out. Yeah. And we see already That's really interesting. In the deployment of LLMs that are publicly accessible, that bias has been introduced into the LLMs on the part of the providers of those interfaces because that, you know, in the name of political correctness or certain sensitivities or other And it's like, okay, but do we fully understand this? Like, what what are we creating when we introduce those biases? And these are now explicit biases. Right? What about implicit biases? What about biases that are created because of structure in the data that just occurs, that we don't we don't see a pattern? Like, to us, as a as a human observer, as a human understander of the language, we don't see that this pattern exists, but it's in the subtext. And then, do these models pick up on subtext? Like, find that they pick up on all kinds of interesting things. Like, it's possible to get instructions of these in many different ways. Do we understand that problem really well? Or are we gonna end up with the thing that I you asked me, what what do I fear? I fear a misunderstood implementation being put in a decision making process that we can't decode. Yeah. Yeah. Even domestically in the US, we have the notions of credit scores. These algorithms are proprietary, but they're known. Right? They're known by the people that provide them. Yeah. The internal state of the LLM is not known, right? It's not knowable per se. And so, if you have something like this, and now this is deciding what level of medical treatment that you get, or does your driverless car turn left or turn right, or hit the brakes? You know, yeah. Like, things are reasoning decisions that we want to be able to, in very great detail, be able to explain and rationalize. So when you make a mistake, when something bad happens Who is liable? Correct it. You can't correct it, and who is liable? Yeah. Yeah. Exactly. Wow. That's really interesting, Matt. That's really interesting. Now, we're running out of time. I could carry on speaking about this all evening. It's fascinating that you've given me some incredible examples and and food for thought there. Even though my you know, at this stage of the show, I'm still being, you know, amazed at some of the insights that some of our guests have. It's incredible. Now, the way we finish each episode is just a fun we're gonna come down a little bit from the from what we're talking about. And we have a fun quick fire round called TGI to go. And it's just so we can finish on a bit of a laugh, a bit of fun, and it gives our listeners a different perspective of you, Matt. So should we do, TJ, to go? Let's go for it. Alright. It's very simple. I'm just gonna give you two choices, and you give me your answer based on your preference. Alright. Okay? Here we go. Hockey or basketball? Basketball. Which team? Detroit Pistons. Ah. Right. Is that because did you My favorite team as a child. Oh, really? Ah, interesting. This is an interesting one. I'll tell you why I think it's interesting afterwards. Chat GPT four or Gemini Advanced? GPT four. Have you used Gemini Advanced? Yes. What do you think? Too biased. Oh, really? I've asked that question to almost every guest this week, and none of them had used Gemini advanced. And the reason I put the question in there is because the New York Times were making such a big deal about it two weeks ago on on on one of their podcasters. This thing is even better than Chachipsy, but you believe it's super biased. It's overtly biased. Oh, really? It's like I I just don't know. But you were just talking about bias just a moment ago, but this is a good example of Yeah. A truly interesting. I made the mistake of asking that question as well to a guy from Amazon. He just looked at me and said, bedrock. You know, and we thought, okay, won't do that again. Singing or dancing? Neither. No. That's okay. I I would say I'd be more willing to observe singing than dancing. Okay. That's a good one. That's a good one. When you get back to the US, you've had a week of tapas, a week of paella, sabbatha, riocha. Would you rather go for a Japanese or a Thai dinner? Thai. Yeah. Thai. And what do you Is this your first time to Barcelona as a city? It is. Do you like it? Oh, it's amazing. Yeah. It is sort of everything is condensed together, like this, you know, what I was really struck by is, you know, we're standing in the corner and we go, okay, where should we eat? So let's load restaurants in the map, right? And they're literally everywhere. Yeah. Throw a stone and it hits a restaurant's window and the owner comes out and shouts at you in Spanish. Like, it's it's yeah. It's an amazing town for food and entertainment and all. I could see why MWC would be here. Yeah. Excellent. That's good to hear. That's good to hear. Now, and and I'll finish on this one. Star Trek or Star Wars? For me, Star Trek. Wow. Yeah. I'll explain if you like. Star Trek is still in the mode of discovery. And Star Wars, sort of everything has already been uncovered. Some things have been forgotten and have to be rediscovered, but everything else is already discovered. Like, everybody is everywhere. Yeah. Yeah. And I I think there's a a an attractiveness to the unknown in the Star Trek universe. That's a brilliant answer. That's a brilliant answer, Matt. So, we finished the episode today. Thank you very much for joining us. Thank you. I've really enjoyed the conversation. It's been great having you and and maybe we'll see you here again next year. We can pick off where we pick up where we left off and and and see how the world has changed. But Alright. I hope you enjoy the rest of your time here in Barcelona. Sounds good. Thanks.