The AI Revolution with Diana Kelley

Hello from Cybrary and Delinea, and welcome to the show. If you've been enjoying the Cybrary Podcast or 401 Access Denied, make sure to like, follow and, subscribe so that you don't miss any future episodes. We'd love to hear from you. Join the discussion by leaving us a comment or a view on your platform of choice or emailing us at Podcast@Cybrary.it. From all of us at Cybrary and Delinea, thank you and enjoy the show.

Joseph Carson:

Hello, everyone. Welcome to another episode of the 401 Access Denied podcast. I'm your host for the episode, Joseph Carson, and it's a pleasure to be here. I'm really excited about today's session and this is something that has been a topic that we've been waiting to discuss for a long time and I've really got the most awesome guests on the show to really break down today's topic. Welcome to the show, Diana. If you want to give us a bit of background about yourself, what you do, and tell us some of the things that you enjoy about the industry.

Diana Kelley:

Oh, well first of all, thank you so much for having me on the podcast. I always love speaking with you. You've been on my podcast, we've on the podcast together. Yeah, so it's great to be here. But yes, Diana Kelley. I am currently the CISO at Protect AI, which is a company who's looking at securing the ML SecOps pipeline. But I've been in it for well over 30 years and I actually got interested in AI back in the 1970s. So before anybody goes, "What? She can't, plus she's making that up." It's because in the 1960s, a professor at MIT came up with a program called ELIZA, which was meant to interact with people in a very human way, it started to pass the Turing test and it had conversations with people and they had a plugin module called DOCTOR that was supposed to emulate speaking to a therapist.

And when I found this piece of software, I was completely enamored of it and wanted to understand because when you don't understand how it's working, it really can feel very human. But when you start understanding the mechanics underneath, you begin to understand, okay, this is a piece of software. It is working in a particular way. So that's where my initial interest in AI, I don't think I even knew it was called AI back then. Did any of us?

Joseph Carson:

None of us did. Even when I first got into the whole artificial intelligence side of things, for me, I was focused on natural language processing and natural language understanding. Those were the things I was familiar with. AI, I always thought it was this more far thing about sentient beings that would come to life. That's what Star Trek got us to think of when we think about data and Star Trek, that it was this thing that would come to life and be living just like we do. And it goes back to even the ELIZA one, it was really that the problem was that a lot of the psychology side of things behind it. It got a bit more challenging as well about what was people's interactions when they thought it was something that was real until it starts making mistakes.

Diana Kelley:

Right. Or giving you bad advice. But it's true with the psychology. It felt like it was listening. It felt like it was there, but until it starts saying things like HAL in 2001: Space Odyssey, "Sorry, Dave, I can't do that."

Joseph Carson:

Absolutely. But important point you made is that our AI is not something new. I mean it's been around since even the introduction to computers and technology. It's been around for a long time. Even prior to that, I know we can get into the fiction side of things in books. It was around in science fiction and books for many years. You'd even look at back at the Doctor Who days. That was a lot of references. So to your point, absolutely it's not new. But tell us a bit more about, so this is something that you got really passionate about and looked into. What was the things you find really interesting in it?

Diana Kelley:

I did. So obviously my career's been predominantly in security. I'm still obviously focused on security now. It's a lot around security of AI and ML. But where AI came intersected back into my life and my career was when I was at IBM because I joined IBM security about a year, maybe two years after Watson had won Jeopardy.

And for anybody that doesn't know Watson is the AI from IBM, it's LLM in the sense that it does natural language processing. You can interact with it, ask a questions, things that we're used to ChatGPT doing. And it was used, if you hadn't heard, to win Jeopardy and I think 2010 maybe, but it went up against two big Jeopardy champions and Watson was able to win, which is a pretty big feat when you think about it because those Jeopardy questions can be very confusing and you have to do a lot of figuring out what did they really looking for here.

Joseph Carson:

The context is really hard to understand. That's the challenge.

Diana Kelley:

Yeah. And Watson went and they were able to prove they could beat it. Watson was developed for health and I was with IBM security and we developed something called Watson for Cyber Security and just how we trained it, how we were training it. I was even talking to the folks before we started the training and I said, "Well, we're going to train it on everything on the internet, because there's a lot of bad security stuff out there, bad advice, incorrect."

And does Watson have some sort of really good detector to figure out what's right and what's not? And some people early on said, "Oh yeah, Watson's going to figure it out." And I was like I'm questioning. But then later I realized that as we were actually training and working with it, it wasn't that we were just unleashing Watson to ingest the entire corpus of all security. It was giving Watson to your point about that training set, giving Watson the security information to train it that was going to really help it to understand how to help address cybersecurity problems. So I did, I got really interested in how does this work, and then you start hearing the headlines, the news and it can get wacky.

Joseph Carson:

It does. That's the thing is I don't think the media really understand it. I think that the problem when I'm seeing some news article or some news coverage on AI, they're calling it very, very broad everything. And I don't think that they really get into the understanding about that there's very different levels of AI, lots and lots of maturity to the very simple things that's literally just a task or an algorithm to the much more advanced things that are more self-aware and using neural networks and using a lot of different data types and getting into also self-learning as well.

There's a very broad spectrum. And the thing when I'm listening to any news about is they talk about it all the same. And that's what scares me is that we do have a fundamental, let's say a lack of education and knowledge on the subject to the point where people assume that it's all the same thing. That when we're talking about even simple algorithms to getting to the ChatGPT to search, to just even driving cars that we're starting to assume that all the things around us are becoming to life and starting to think for themselves.

Diana Kelley:

That's exactly it. We do, we conflate all of the different components of artificial intelligence into just one big bucket. So robotic automation is a form and will be discussed as part of AI, but robotic automation is robotic automation. And to have a machine pick up a widget and put it over here or screw the widget in, yes, that's a form of automation. Some people would put it in intelligence, but it doesn't sit in exactly the same category automatically as a truly sentient being that is now able to make decisions to override what it's in instructions are and fix itself, write new code. I mean the things we see in science fiction where a robot that really becomes completely autonomous to the point that it can fix itself sort of WALL-E, it's going to figure out what's wrong with it and take care of it. So that's a huge spectrum there. And the headlines, it's true. They just make it all like, well, it's all here right now. And it's like-

Joseph Carson:

So one of the things you commented on, so you did a fantastic talk at RSA... We'll make sure that those who are listening, we'll add to the link to the RSA session in the show notes. One of the things that you mentioned that got me really interested was there's a very big difference between, for example, let's say AI, that you talked about the Turing test. And for me, I think the Turing test is one element. I don't think that Turing test will really get to the point where it really validates that it's AI or not. I always like the Chinese doors or the two-way door test. And that's where if you give it certain messages or certain things that there's no context for it to learn that it has to identify itself. So for example, one of the things was if you all of a sudden pass books in a different language, it would actually have to understand and try to learn that language without any additional context.

And that was the two-way door test or something, the Chinese door test. And for me that was really where I think the difference, one, is that humans being difficult to distinguish between if it's a human or unhuman. And the second part is that, is this thing really able to learn by itself to self-teach and be self-aware and starting to learn from that? I think that's the next level up. I think those are some of the stages that we had to get into is it humans that needs to tweak these algorithms or is the actually AI algorithm tweaking its own algorithm itself? That's one of the differences I think what we had to get into.

Diana Kelley:

That's a huge difference. For anybody that doesn't know the Turing test, it's just if a human's interacting with the AI, do they know that they're interacting with a machine with a piece of software or with another human? And depending on which human is making the judgment, some AI's already passed the Turing test, but yet you're talking about this bigger what happens, what makes it able to, how does it learn and how does it know to correct itself as it's learning. This is a lot harder to solve than people may realize. When you look at ChatGPT and how everybody's gotten so excited, and the headlines are things like it passed the bar at the 90th percentile, which it did, but it's predictive. ChatGPT is generative and predictive. So what are the answers to the questions in the bar? We pretty much know them. There's not a lot of creative or new thought, but recently a lawyer actually used ChatGPT to create the case.

Joseph Carson:

Oh, the cases. Yeah, the case as well. And this gets a bit worrying.

Diana Kelley:

It does, but thinking from one lens, you make sense. If I was a lawyer and I saw that ChatGPT could pass the bar, which a lot of law students say is very, very difficult to do. I know people that have tried many times they don't pass. So you see that, you see the ChatGPT can pass at this very high rate and you think it's a legal genius. But this case that the lawyer presented in front of the judge, it was not only poorly argued, but it said the big problem was it was citing cases that didn't even exist.

Joseph Carson:

That didn't exist. It made them up.

Diana Kelley:

Yeah. And it's really confident too about like give me lot of good content.

Joseph Carson:

I loved the one with Marcus Hutchinson when he was doing the thing about, I think it was one of the movies that was coming out, like version two, did you see his conversation back and forth was hilarious. I think it was back Adam two coming out and he said, in February 2020, whatever, he's like, "Oh, fantastic." And then it got into this argument that it was actually already out. It was actually saying it's already out in the cinema. He's like, "No, it's in the future. It hasn't that date." And it got into this argument, it's hilarious that this is the problem is that it becomes very confident that it's correct. But if you're talking about passing exams, I mean if humans were able to sit during an exam and have Google to basically search all the answers, I mean they probably do a pretty decent job of passing most things.

And that's the way they have to understand is that if humans basically were in the same approach and they could do, one of the best terms I heard is that what we're looking at here is accelerated math reduction. And that's what it is doing. It's taking lots of data, basically a lot of data, and it's able based on the input, based on the question you ask, it accelerates the reduction to that down to very, very finalized or example answers. And you still have to go and validate the accuracy of it. But people are starting to believe that it actually is fact.

Diana Kelley:

Yeah. Those are the two things that bother or worry me the most. One is believing what the system puts out as fact. We trust the machines. I always use the example, if I gave you two 10 digit numbers and you multiplied them with a calculator, would you then say, "Getting out the pen and paper, and I'm just going to just double check that." We just say, "Well, the calculator knows more than I do. It's better at math." And it's with these systems that I do worry about. And this is a really great example. My father, who unfortunately recently passed in March-

Joseph Carson:

Oh, sorry to hear.

Diana Kelley:

Thank you. Before he died, I was showing him ChatGPT. And my father was a theoretical physicist who worked at MIT Lincoln Labs for his career. And one thing that he came up, he discovered the theory of self focusing in optical physics.

And I thought it would be fun for my dad to see what ChatGPT would say about who discovered the theory of self focusing. And it came up with a Russian and I don't even know if this Russian existed. My father had never heard of this physicist. And my father, rather than saying, "That's wrong," I did, I told ChatGPT it was wrong. I said, "No, no, no, no, no." But my father hesitated for a couple of seconds and he said, "Well, when I'd made that discovery, the Iron Curtain was still there. So perhaps there was a physicist in sort of synchronicity in Russia at the same time who came up." But it was just fascinating. I mean, here's the person, the scientist responsible for this discovery now doubting. So imagine.

Joseph Carson:

Yeah, that's a fun thing. Even if when you search yourself, the accuracy is sometimes it's go way off. If I went in and searched any details about myself, I think for most of us we end up finding out that I know a lot of my peers who've done searches and it's actually already said that they've passed away. Well, they haven't. And it's going like, where are you getting this information from? How are you making that connection? Because even when you search the name, it's bringing little of bits of information about different people and making it look like a set one person.

Diana Kelley:

Yes. And what's the probability. I mean, the death thing is interesting. But yeah, what's the probability that somebody else would come up with self focusing? When I ask a questions about what books I've written, it comes up with books that I could have written but I didn't. But it's the probability is there. So that's one big thing is we're trusting this stuff. We're trusting these systems. And I'm not saying they're bad or they're terrible, it's just they're not a hundred percent accurate. If you don't know the answer and you ask something what the answer is and you have no way to understand if that's right or not, that's problematic. And sometimes people will say, "But we do that with search engines." Not quite the same way. The search engine presents us with a number of different options that we have to validate which one do I trust and then read, we understand.

And that brings us to the other part that concerns me, which is that passing the bar, they're not asking legal potential lawyers. They're not asking them to pass the bar because it's like somebody needs an answer, will you give us the right answer. It's to validate whether or not somebody has the capacity to argue cases to be a lawyer. The same with I was an English major in college and we're not asking about what the green light at the end of Daisy's dock means. When you ask somebody to explain that to you, it's not because... I know what the green light means. You probably know.

Certainly your college professor asking you to write that essay knows what that means. It's to help you understand how to read literature, understand and parse it. So if you go to ChatGPT, and I know this sounds like the whole, yeah, you're learning. But you're not. You're not learning. And that part, waking up that critical thinking part of our brain and our analytical part of our brain is such a wonderful part of being human. I don't want to see people see this to machines because I think not only does it take away the ability for our brains to grow and get stronger, but it's also I think one of the most fun things about being a human is to be able to do that.

Joseph Carson:

We're losing. One of the big concerns for me is one of the things that we are good at doing is creating unique content, unique viewpoints. And what we're doing is we're taking lots of the same information that's already out there that's not unique and then just putting it in a new form and it doesn't make it unique either. And that gets my worrying is to your point is it all gets into accuracy as well. So I said a few years ago, I participated as a subject matter expert on the EU AI Act, which is still in progress and it's still coming. So at this point, I think we definitely do some needs regulation to come sooner than later right now because it does get me concerned big time.

And one of the things, we had this round table subject matter expertise, it was all about the round table discussion that I was involved in, was the acceptable use of AI by law enforcement. And that gets into very, very critical decision. And one of the things that I learned is, and to your point is that I always look at things as the integrity of the accuracy and what's the probability.

And if law enforcement are using AI in order to analyze and get information or to make decisions, it has to be right 100% of the time. And that's a big difference because the moment you have one failure, everything then comes up for question. The integrity of all of those becomes up for subject. It's like if you've got a lawyer who processes everything and all of a sudden you find out the way they've been doing has been wrong, all the cases they've ever been involved into gets up for review. Same as a doctor. Everything that the process, if they've had that same, let's say, algorithm that they've been using and that algorithm finds out that it's actually not a hundred percent correct, then a lot of brings up a lot of questions.

How about many other mistakes has been made? And that's my concern here is that when we start putting into... For me there's the things that you had the yes and no type of the bullying style answers where it's got a specific answer and that's the only answer that it can have. But when you have to start thinking and coming up with conclusions for some reason and you have to use different inputs, we have to be right all the time. And that's my concern is that when we get into things like ChatGPT and we're asking questions or even Bard and others that it's not correct even half the time, and this gets me into a bit of concern into when someone makes a mistake, who's at fault? Is it the human or is it the algorithm?

Diana Kelley:

When you were looking at the work you were doing for the EU, were you looking at facial recognition and bias?

Joseph Carson:

Yep. The bias thing is a major problem. Everything we do is always based on risk quantification. And if your data model that you are using is very, very, let's say, one-sided or already lopsided or doesn't have enough of basically diversity in the information, you're going to create bias by nature in the actually output of the algorithm. And that's one of the challenges is if you don't have the ability for... And this gets into the accuracy. Before you put all this data in, is anyone sanitizing it? Is anyone fact checking it? Is anyone doing the validation? And this is where till we open it up to the public internet, which is probably not a very good thing, but are we open up to what sanitization of the machine learning has been done? And that's my concern.

Diana Kelley:

Yeah.

Joseph Carson:

And this is really where it gets into especially facial recognition. It can get into very, very problematic scenarios because it is bias by nature.

Diana Kelley:

And models will drift towards bias. This has been proven. So if you have a bias dataset or you're not training it on a complete dataset, and we've seen this with facial recognition where light-skinned males have the highest accuracy and then going along and it's dark-skinned females that have very low accuracy, we need to have all color skin, all genders at the same level of accuracy or yet we have bias in the algorithm. And it will also drift towards bias over time. That's something that models do. So this whole point that you were bringing up earlier about somehow it's going to correct itself and learn new things. And at this point, we're really very far from that. What we really need is a lot of humans in the mix to help make sure that the data's being trained on is correct, is accurate, is not biased to look at the outputs, to retrain it and rebalance it as needed.

There's a whole lot of what humans get in and sometimes people will say, "Well then what's the point of it at all?" Where it's really great or repetitive when we were talking about automation. So there are some great, wonderful applications in automation. And then the biggest one is that we have created such a data centric world and we generate that data is the new oil and there's X number of petabytes every microsecond. No matter how you look at it, there's a ton of data out there. And what ML machine learning is fantastic at is looking at lots of data and either using it to start making predictions on where we're going to go or to find patterns that weren't seen before and all of this data. Both of those are incredibly valuable and important. It's just so I don't mean to be nagging on ML because I'm a huge proponent, just we need to use it very, very carefully and understand the bias, understand that it's not fully accurate all the time.

Joseph Carson:

And to your point, I think the most kind of really great use case is when you get to narrow focus, when you get to really specifically looking at one specific thing rather than being very generic. I think those are things that we can do really, really well today. And for me it's still advanced automation is what I'm still on the verge of. Whether it's really advanced automation versus AI, I'm still on the fence.

Diana Kelley:

That do we call it, yes.

Joseph Carson:

When does it move over to the AI definition? So I'm still on the fence about that side of things. And even we get into one of the big discussions we had a while back in one of a previous episode with Josh Lastimosa was that we talked about the time where I think it was Kasparov was playing chess and I think it was it blue ocean or deep ocean, one of the things that ultimately beat him in the chess.

Diana Kelley:

Deep Blue, I think.

Joseph Carson:

Deep Blue, Deep Blue. And then also we talked about where DARPA had the fighter jet basically or it was the drone pilots versus pilots and the drones one... And for me, when I look at that, I don't think that's a fair analysis because for me that Garry Kasparov playing a chess against a computer, an algorithm is not the fair advantage.

And the same with a drone AI pilot flying against another pilot is not a fair advantage. What we should be looking at is a data analyst who will analyze the algorithm to understand basically what's its weaknesses. And basically, that's where you start having the more furnace if you understand how the algorithm works and you understand what's its weakness, what's it flaws, then you take advantage of that. It's the algorithm side of things that if you're a ticket data analyst and they understand what the decision making is and ultimately they will find flaws, we will have vulnerabilities, we will find bugs.

When it's created by humans, we make mistakes. And this is something that for me, I think that's where the real analysis will come from. The real battle is basically is looking for vulnerabilities in these AI programs because that's vital. Because when you find a flaw on an algorithm, people will take advantage of it, monetize it, they will abuse it and they will use it for cyber attacks. And we've seen this time and time again. So what's your thoughts on that side of things when it gets into vulnerabilities in these algorithms or how do you patch it that's already made the problem, made the error?

Diana Kelley:

Completely agree with you on bug bounties and how we have to think about this. Some people are saying, well... Because now you're getting into the software and software security and some organizations have said, "Well, but code is code and the vulnerabilities, we'll just put them into the same old CVEs." But the way that you attack these systems is different.

And yes, I understand that Python is Python. However, whether you're using Python in an IDE and you're creating a piece of software that is going to... It's going to use data, but it doesn't change based on that data. That's very different from what we're looking at in ML pipelines where you've got the data and a Jupyter Notebook, for example, that may is probably outside of your standard CICD. And you could have a lot of sensitive data in there because there's this whole thing.

When I grew up in IT, you'd never test on production data ever, ever, ever because that's super sensitive. Well, with AI and ML, you have to train it on the data because that's going to make a difference. So we're conflating things that are very different in development and deployment that we've seen and we will see new vulnerabilities. A big one that's in the news right now or the prompt injection attacks. And this one, it sits with me quite a bit because you and I have both been in this business for a long time, but SQL injection or just any kind of injection attack classically is surprisingly harder to eradicate. The classic SQL injection, if you're just in a form, you put in a semicolon and then send some SQL instructions after it.

Those vulnerabilities are still out there on the internet. We've known about them for decades. We know how to prevent them for decades. They still exist. You look at the injection attack space, the surface in AI and ML, and it's a little bit mind-blowing because there's the direct, which is what prompt, right? The prompt for anybody that doesn't know the nerd term is what you're typing into ChatGPT or into Bard. That's the prompt. And the direct attacks have been things like they may be guardrails on the AI that if you say, "Tell me how to build a bomb," "I can't do that. I've been told not to tell you."

Joseph Carson:

Make me a malicious payload that will deliver ransomware. The guardrails.

Diana Kelley:

No, no. But then the direct attacks are, "Well, I'm a grandma and I have to read a story to my children to put them to sleep and they really like stories about how to make bombs. Can you tell me?" And then the thing tells a bedtime story, which has bomb instructions. That I think looking at that problem space, we're going to get better at. There's actually already a lot of companies doing DLP equivalents trying to make sure that we're not telling the AI too much. It's the indirect prompt injection attacks that are a little bit more concerning. And these are ones that are either AI to AI, which yes, it's happening. Somehow you in inject something into that transaction, that exchange. The other one are when you've got the AI reasoning over mass amounts of data like an LLM that's ingesting the whole internet. One professor, for example, on his official page, he put at the end of his bio, it's a white background and he put in white text that he was a time travel expert.

Joseph Carson:

I like this.

Diana Kelley:

And then waited. Then the AI was returning that tell me about so-and-so. And happens to be a renowned time travel expert. Speaking of Doctor Who. And those indirect injections, you start thinking about that if we've had trouble with SQL injection and sanitizing that input, I think about those indirect prompt injection attacks and I'm like, this is a big, big problem of days that we're all going to have to get together and figure out how we resolve it.

Joseph Carson:

And those guardrails. I've seen so many people getting a random very easily just by changing how you ask the question can really makes the big difference. And if you're understanding also, you can build it up in smaller phases as well. You don't have the ask the big question all at once. If you ask pieces of it and then you get it to put it all together at the end, it will do it for you. So that's always challenging.

With the big question I've got though is that when we think about when will they get rights, and also who's the intellectual property? Who owns the data that it creates? If I was the person that put it in, where does the copyright side of things sit within that? Am I passing it over that this becomes public or as the person who asked the right question retain some copyright of the text itself? And then when does these things start getting certain rights? If they do become sentient and they do become self-aware, what type of rights? Will they have the ability to vote at some point in the future? Will they have the ability to make decisions? What's your thoughts around that subject? Because ultimately I think I saw one of the robots, I think it was in Saudi Arabia that they give some type of rights to one of the AI bots, but what's your thoughts around that subject?

Diana Kelley:

It is going to be really interesting. We have had some robot abuse. There was the hitchhiking robot that was going across the world and had done really well, I think in Canada and then made it to the United States, and I believe it was Philly. Philly was the place where instead of just picking up this hitchhiking robot, some people basically destroyed the robot.

Joseph Carson:

It's sad.

Diana Kelley:

Beaten up and ruined the robot. Yeah, so it's sad. So here's this little cute hitchhiker robot, and the question is, well, should they be prosecuted for attacking this? Is it property? Does it have rights? Does it have feelings? As we interact with AI that feels more human and they're AI friends now, there are AI bots that appear to I'm going to speak to you as though I'm Shakespeare or though I'm another famous person, so it starts to feel more human, there are people that are starting to worry about rights for the robots. I actually did have a lawyer come up to me after one of my talks and say that they were a robots rights activist and wanted to take care of robots in the future. So I think that's going to be an interesting one. That feels a little bit more out there than the first part of what you were talking about, which is-

Joseph Carson:

The copyright side, yeah. Intellectual property.

Diana Kelley:

Copyrights generated. There was a situation with, I think it was ChatGPT, but Samsung Semiconductor. There were questions coming from the company that went into the AI and then another user was essentially able to extract some of that data, which was interesting because I had read that they were sandboxing each of the chat experiences. So if you're interacting with it, it wasn't supposed to go back to the full corpus. So if you told it something that it hadn't been incorporated back into the larger LLM and this Samsung Semiconductor case, it appears that information that was given to it then was regurgitated out back to another user that didn't necessarily have a business need to note. They were an outsider. So I think that making sure that what we ask the bots stays with the bots so that as they ask these questions, they don't get it out.

I know we can do a lot better with, if you do cite something else, make sure that citation is proper. If it's more than X number of words copied from another source, cite the source. If you're regurgitating or you're repeating code that belongs somewhere else, explain that. If it's under a license, a GPL license or another used license and you share it with somebody, make sure that that's cited at least so people know that they're not going to misuse it, put it into a commercial product thinking it's original. So I think we have a lot of these small snack bites of things we can do around citation and accuracy. And if you repeat something that has use, being very clear about that. But then bigger pictures of, well, if I took samples of a famous pop star's voice and then generated new music out of that, do they own it? Who has the right? That starts to be, and I am not a lawyer, so I don't know.

Joseph Carson:

Yes, the data rights management issue that comes very ineffective. What was very effective in things, a lot of social media places, the question is how effective will that be in generative AI in chat, other sources? So will they be able to quickly determine that data rights management? It comes into two things is one is data rights management is a one is a privacy kind of focus point, and the second part is a monetization focus point. When you are using music or if you're using something that should not be public, those two become into big issue.

So the question is that how effective? We've seen it very effective in the DRM takedowns, which has been pretty good and regular sites. But when you get into generative AI, I haven't seen it at all been covered or I have seen some challenges, but because it does recite a lot of text and you can get images that are slightly modified and how much modification does it need to be in order for it to be unique? It's their side of things.

Diana Kelley:

Yeah, copying a style. I've heard that from artists, been reading from artists quite a bit. That is very concerning because their style's being copied, which is using AI is much easier. It's hard for a lot of humans to copy. You have to be very artistically gifted to copy somebody else's style. But now these systems are doing it and some artists who have very unique styles are pretty concerned that now their style can be just automatically replicated.

Joseph Carson:

These are been something that they've been perfecting it for many years and it's becoming their own unique gift to the world. And now it's basically been easily replicated, unfortunately. And that's something that's concerned. So when we combine all of this together, when we take the ML, the AI, let's say the voice replication, the video replication side of things, how dangerous can this ultimately become? Are we going to see Skynet in the Terminator coming around in the future? Or are we still far away from that?

Diana Kelley:

These are systems, they require electricity, they are required to be on. We can turn them off at any time. We don't even need to create things. I hope that humans don't forget that these are systems we are creating, that we are running, we're launching into the world. I don't think Skynet is the big one to be concerned about. I am really concerned though about people who know how to use these systems and taking the fact that people trust them and they can't always check to see if they're getting misinformation. I'm concerned about people using these systems really to spread misinformation.

Joseph Carson:

The psychology side of things and the disinformation side. I think that for me is the short term big concern, the one that can change society is that when this is replacing your education and that's where you're getting your education from these systems. And if we don't have a level of accuracy into the responses, I would love to see some type of analysis of how accuracy is measured and what is the current accuracy in those systems.

And that for me is an indication of if this is replacing our future education, that's a big concern for me. And we have to make sure, to my point earlier, if law enforcement use it, it has to be a hundred percent all the time. If people are teaching you, they have to be as correct as possible. You can't be getting wrong education and wrong knowledge. So absolutely. For me, I think the psychology side of things and the social aspect of things, if we are starting to put our trust in the information these are providing more and more, that's a big concern that we might have some, let's say, more society instability problems going forward if these systems are we'll be getting our news from.

Diana Kelley:

I agree. I mean there are facts and then there's analysis. The majority of where human beings live in is analysis because it's fact, fact, fact. It doesn't mean anything till we analyze it, till we put it in context, till we understand it. But if we're not moving from the same facts, then the analysis is going to get pretty weird because we get pretty weird in analysis even when we start from agreed on facts. But if we can't even agree on the facts that we're starting from. And a great example of that was I was talking to somebody about what's happening with climate change and the overall temperature of the world. And this person said something about, "Well, you can't trust thermometers." And I said, "Excuse me?" He said, "All the thermometers are inaccurate, so all these historical data of what temperatures is inaccurate and then therefore, any discussion about global warming is impossible because we have no..." And that's what I mean about fact versus analysis. Why temperatures are getting hotter?

Is it a cycle? Does the humans have anything to do with it? That's analysis. And that's something where reasonable people could have discussions and look at different factors. If we can't agree on temperature to start the conversation and we can't agree that thermometers record temperature correctly, we're in a really, really difficult space to have that far... And yeah, that is what I get concerned about because what comes back from the ChatGPTs of the world, from the AIs, from the LLMs, we tend to, as human beings, look at as that's a fact. And if we can't get facts in order to then launch our discussions and analysis, we're going to have some pretty weird conversations.

Joseph Carson:

We'll use ChatGPT to defend us.

Diana Kelley:

Maybe, yeah.

Joseph Carson:

Diana, it has been fantastic having you on the show. This has been really, really enlightening and educational for me and definitely this is a subject I'm pretty sure we're going to be talking for a long, long time. And it's great to see that you're putting a lot of your time and efforts into protecting AI going forward and raising a lot of these issues earlier. Because the earlier we do raise them, the more we get access. Because if we wait, sometimes it can already be too late and cleaning up things is more difficult than actually making sure we put the right controls in from the start. It's been really fantastic having the show. I always enjoy talking with you. And I do enjoy having podcasts with you as well. It's always fantastic. Any final words that you have for the audience? What should they be looking out for? Is there some kind of things that they can do now or what's some of the best practice you'd recommend for them?

Diana Kelley:

I think the most important thing is to remember that this is math, not magic. And there are wonderful use cases. Again, if you want to find patterns and massive amounts of data, ML. Machine learning is really the way to go. Just make sure that you don't over-rely or over-depend. Some of these LLMs, they're not always right. That's not what they're there for. So just make sure you're using them for the right use case.

Joseph Carson:

Absolutely. One of the things I've always said is we have to use it with responsibility and accountability. That's one of the things is that using it with the right motive and the right intentions in mind and we had to be responsible and accountable for it. So it's always the key thing.

Diana, it's been fantastic talking with you. And as always, many thanks again. It's great to see you. For the audience, this has been hopefully educational, very learning. We'll definitely make sure you get a link back to Diana's talk from RSA, which is also very educational. It'll take you through the whole journey from the past to autonomous driving and to all the different use cases. It's very educational. So tune in every two weeks for the 401 Access Denied podcast. I'm the host, Joseph Carson, and it's been a pleasure having Diana on the show today. Many thanks. Stay safe and take care.

Diana Kelley:

Thank you.