End-to-end AI: Strategy, design and delivery

Saima Shafiq, Head of Applied AI Transformation, and Anuj Shah, Head of Intelligent Automation, of PNC Bank share with BAI CMO Holly Hughes what it’s like to think about AI as part of a market-tested technology strategy. “Starting small to scale later with agility can be truly transformative,” says Shafiq. Whether you’re looking to expand the reach of AI in select departments within your bank or credit union, or launch an enterprise strategy, three elements are important to keep in mind: Find a use case, define the scope and be ready to prove ROI.

Don’t miss the other insights, methods and tools that can help your financial institution shape and evolve its AI strategy in the BAI Deep Dive: A practical approach to AI.

Dive deeper into AI insights

You might also be interested in:

TRANSCRIPT:

Holly Hughes, CMO at BAI: Welcome to the BAI Deep Dive on AI. I’m Holly Hughes, Chief Marketing Officer at BAI, and pleased to be joined by two financial services AI thought leaders, Saima Shafiq and Anuj Shah, both from PNC. At the bank, Saima is Senior Vice President, Head of Applied AI Transformation, aligning with the businesses to implement AI solutions, and Anuj is Head of Intelligent Automation. He helps design PNC’s machine learning, generative AI, virtual assistant features and more.

As we’re about to learn, their roles are varied, yet complementary. They reflect the fact that AI certainly at PNC has matured beyond experimentation. It’s about scaling and maximizing value from people and products. Welcome, Saima and Anuj. It’s so great to have you with us today to talk about this popular topic. Now, certainly AI is not brand new to banking. It’s more about generative AI or gen AI that’s getting all the buzz and attention with the potential to rapidly change our industry in a number of ways. Anuj, I would love to start with you. Could you bring us up to speed on the differences between more traditional AI usage in financial services and what distinguishes it from generative AI that has gained so much attention

Anuj Shah, Head of Intelligent Automation at PNC: As you said, we’ve been using traditional AI for quite a number of years, and the distinction that we make, generative AI has the ability to generate net new content. Traditional AI is what we call deterministic. It has a known set of inputs, a known set of outputs. You can do testing to then determine whether something is working or not. You can send in a set of inputs and then you’ll get a set of outputs that you expect, and then you just check whether you’re getting the right set of tasks associated with that or not. Generative AI has a much more varied set of inputs and outputs, so generative AI can take the entire human language as input, so your number of inputs is significantly more, significantly higher, and it can generate a whole set of outputs depending on the context that’s provided as part of the prompt. That’s the big difference and that has a lot of downstream implications when you start to think through a lot of the model validation testing that we have to go through.

Holly: And we’ll definitely be talking more about gen AI, but just at a high level, how should banks be nurturing their investment in traditional AI? Anuj, another follow up question for you, does it have to be a trade-off?

Anuj: We’re not seeing it as a trade-off. We’re actually seeing them as complementary technology. We’re seeing a lot of the value that we’re generating as a technology organization come from traditional AI, so we’re still using a significant amount of traditional AI techniques to generate a lot of what we do from a project standpoint. That said, what we’re seeing is generative AI complements and fill in a lot of the gaps that we weren’t able to do using traditional AI.

We’re using combined traditional AI and generative AI projects now to deliver solutions, which I think are really compelling. One of the things that we’re doing is we’re doing a knowledge search use case and we’re using traditional AI to go and search for what is the relevant information based on the question that’s being asked. Now, what that would traditionally do is it would give you a set of search results you have to do and you have to click through potentially pages of links similar to a traditional search, and you have to go and hunt for your answer. What we’re using generative AI for is to take all of that information and summarize it into a response, so now it’s a little bit more of a natural language, you ask a question, you get a response, you get a set of links associated to it that you can do more research on, but it’s a little bit more of a Q&A interaction versus your traditional search.

Holly: Yeah, it really speeds up that process then to be able to analyze that data, certainly very valuable. Let’s talk about whether an organization must act on AI wholesale or if it can take more of an a la carte approach, maybe by department or solving a specific challenge or an opportunity. Anuj, I know in your role you start with strategy and product design, Saima, you have to deliver and implement related processes throughout the organization. So,  Saima first to you, should banks be thinking about AI wholesale or can AI be more focused and what are the cost considerations with either approach?

Saima Shafiq, Head of Applied AI Transformation at PNC: Sure, yeah, I mean that’s a great question and first of all I should say thanks for having us. I think this is an exciting topic. We love talking about the fun journey, so like you already mentioned, since my role is more around strategic technology implementations and delivery of the AI solutions using the technology that enterprise group for instance, Anuj’s team brings in, I find it instrumental to your point, first of all, to get before we decide wholesale or a la carte, we have to get the pulse of the organization goals as they get communicated across different business segments and the functional units.

If I had to say it in few words, I believe once again before the approach, I think AI can produce the most value simply by aligning with company’s larger strategic priorities. Now, as far as approach goes to your question, I think it truly depends on the current level of maturity of an organization and of course the available talent in AI space as well as the key leadership roles that have been identified or appointed or designated for the function as well as the technologies that we may have already brought in versus what we have to bring in for the newer space like generative AI and whatnot.

Now, for even the larger organizations who may not have previously invested heavily in the space of applications of AI, starting small to scale later on with agility can be really truly transformative. Now, there is no chance of a big bang or a wholesale until organization leaders, the data scientists, the experts as well as the process and technology specialists have been evaluating them on a very smaller contained manner. You have to have a defined scope. You have to have select use cases where you can define clear business outcomes. Until that is done, no approach can make you successful. You can get all hyper excited about it, but if you don’t have defined scope to prove the value, you’re not going to get on the right track right?

Now of course, after the initial assessments that these teams that are involved would do, they would have a lot of very valuable lessons learned. Applying AI for then major transformations can be tremendously rewarding, but establishing shorter milestones along the way on a multi-year roadmap possibly is the way to keep up with the track and keep everyone involved motivated as well. Now, adjustments of course will need to be made on the scope and expectations as warranted along the way. From the cost standpoint, the foundational technology cost is no questions asked. That’s absolutely needed. You want to bring in the best tools and technologies to make it happen. From an application standpoint then, you once again want to keep it very conditional. You would invest so much so you can see the value and then keep on extending based on the return that you’re getting.

Holly: And Saima, a follow-up question there. I love that point on getting the full organization buy-in, so important beyond the technology department. Could you speak a little bit more about that, the importance and who you’re engaging with? Who are some other key stakeholders that need to come along in that journey?

Saima: Yeah, this is a very good question. I mean, I think it’s critical for us to have the right level of engagement from different parts of the organization. While on one hand we believe that the AI or the technology department’s role is fundamental in enabling or making the transformation by application and implementation of AI, I see everybody, all the thought leaders in the AI space, they know the actual lines of businesses, the operations teams, the front office, middle office, back office teams. They all play essentially, I would say a vital role in sharing the areas of potential opportunities. Then you’ll have to work with them to understand the core business problems that they have to explain from their own stance, and then they have to also be willing to accept the cultural and process related changes that you would see evolve as you’re implementing, right? If they’re not willing or ready to adopt culturally, that can be a barrier, so you want to make sure that you have all parts of the organization engaged with you.

Now, as someone who has been in the field of AI for more than two decades, I truly feel that we cannot overemphasize the whole of an organization buy-in. I don’t think that’s something that we can underestimate. In my case when I’m explaining this, I have the thought process of, “Hey, top down, yes, leadership should be prioritizing the utilization of AI, making it an essential part of their strategy, socializing it in their key messages to their groups in their town halls, to their organizations and business segments.”

While on the ground upside, you should be considering where ultimate end users have to feel really that they need to have this. They should feel, “Yeah, this is the way for them to improve. They should feel empowered, they should feel engaged ground up,” right? You want to have the operations people working day in, day out enabled in these changes. Now, as I said earlier, if we are not doing ground up, the cultural barriers or the fear of change can really impede your progress or your success chances, right? So once again, to wrap it up, it is instrumental to have messages coming top down from the leadership as well as engagements working bottom up to gain the maximum value initiatives.

Holly: And Anuj, I’d love your thoughts here. Any learnings from traditional AI implementation or other technology upgrades perhaps that have conditioned the workplace for how it’s likely to use AI or are we really in new territory?

Anuj: I think there’s a lot of parallels between what we’ve learned in traditional AI and now. First and foremost, data is paramount, and as we’re quickly learning and going through this process and the testing process that we’re going through for generative AI use cases, if we don’t have good data going in, the results that were coming out of the system are not great. So we’re spending a lot of time looking at what our data quality is, what are good data sources, how do we get piping into those sources, making sure those sources have a level of completeness and accuracy that we’re comfortable with putting into the model, so we’re spending a lot of time doing that. I’d say the other pieces are engaging the right set of stakeholders across the organization. As Saima said, I think business and operations involvement is very critical in understanding the context associated to this.

A few of the other parts of the organization we started to involve are first and second line risk partners, specifically from a generative AI standpoint and security partners specifically from a generative AI standpoint to understand what our risk and control posture can remain and can be leveraged for generative AI and which ones we need to either enhance or build out net new specific to generative AI. There’s a mix. I mean, there’s some things that we can leverage and there’s some things that we’re going to have to build net new associated to this. We want to make sure that we’re putting the right set of use cases through this process. We’ve adopted a centralized approach where we’re funneling all of our generative AI use cases through the process, and there’s a set of filters that we’re starting to apply to make sure that things are going through that should be going through.

The first is really understanding, as Saima said, what the business case associated to this use case is and making sure it’s the right one, that there’s right ROI value associated to it, and we think from a technology standpoint it’s feasible. We think from a risk standpoint it’s within our risk posture as an organization, so there’s a whole set of filters as part of that whole business case process that we look at. The second thing is we want to make sure that we’re not using generative AI for the sake of using generative AI. It’s the shiniest of shiny objects as we like to say in the organization that we could use, but is it the right use for our end customer and our clients? So it’s really understanding why we need to go down the generative AI approach versus our traditional AI approach that we’ve spent quite a number of years and have done a lot of investment and have proven ROI associated to go down that path.

Once we’ve passed that filter, then it’s really understanding what generative AI capability we want to use. What we’re finding as we go through this process is generative AI is really good, has capabilities that are really good in specific use cases, and then it’s a stretch in other use cases, so we just want to make sure that we’re aligning it to the right set of capabilities. Summarization, for example, we’re finding is a pretty good capability out of the model and what were used… I mean, that’s one of the initial capabilities that we’re starting to build out.

And the last thing is we’re finding and say this is probably analogous to traditional AI as well, is getting a demo or a proof of concept spun up is pretty… you can do that really quickly. Getting something into production is where we’re spending probably the bulk of our calories and really going through and understanding the testing process, making sure that we understand a lot of the different variations. As I mentioned earlier, there’s a whole host of different ways to ask the question, and depending on the way you ask the question, you can get different responses out of the generative AI model.

Holly: So Anuj, another question for you. Our industry talks a lot about better AI-powered customer experience, fraud detection, revenue opportunities. You talked a lot about a lot of other really valuable use cases, but there could be a mixed message to staff when it comes to AI and Saima, you had some great points on that in terms of how it’s talked about within the organization. Anuj, we’d love to learn a little bit more from you in terms of what you found to be the best approach to just positioning AI overall with staff.

Anuj: Specifically in the generative AI space, there’s a lot of buzz that you see in the headlines around what the technology may do to employment and it varies from what we’re seeing internally. We’re really seeing it internally as a co-pilot and we’re really using it to help enhance or supplement the role that individuals play internally, and we find that we’re able to align individuals to take away some of their more… the mundane parts of their role and really have them focus on higher value activities, and I sort of see this as analogous to if you sort of line the clock back a few decades, when the ATM came out, the ATM came out and every headline that you were reading around that time was it was death of the teller roll and fast-forward to today, we still have tellers in every major branch.

The role of the teller has changed pretty significantly over that time and they’ve moved away from, if you look at this percentage of time that they spend, they’ve moved away from transactional activity that they were doing decades ago and they’re moving more towards a relationship management activity or relationship building activity with our customers, which I mean to be frank is our higher value activity and we see that play out in a lot of the different roles that we’re starting to think about deploying this technology in.

Holly: Yeah, those are great comparisons. I love that co-pilot analogy. I think that’s so strong and free them up to do other things, and we have been through these transformations before. I think gen AI is just so powerful and it feels like it’s come on so fast that maybe that’s what amped people or created that extra angst. Anuj, follow-up for you on that on this talent piece because I think it is so important, how important do you think it is this AI evolution in shaping how our industry hires and trains new talent ultimately? It could mean new roles related to AI project management. You talked about data governance for sure. It could also mean human soft skills are increasingly valuable and where you have that human interaction versus more of an AI interaction. Would love your thoughts on that.

Anuj: Yeah, I think talent is absolutely critical in generative AI and I think talent is important in general. I think the things that we’re finding are, there’s a little bit of a mix, so there’s a lot of cases where our company posture is to upskill the resources that we have. I mean, we have a whole data science community in the organization and the goal is to upskill a lot of that, our developer community, our business resources to upskill everyone to move towards this. That said, there are some net new roles that we are looking at and bringing in to help supplement a lot of our thinking. So prompt engineering, for example, is one that sort of comes up a lot and it’s interesting to see that we’re able to upskill some of our existing resources in our data science to help with this and then sort of supplement them with external talent that we’re bringing in as well. I think that’s sort of the mix that we’re trying to balance.

Holly: Very helpful, thank you. So as we wrap up, we’ve covered a lot of great ground. Saima, I want to give you the last word. I think your role is so key in terms of leading AI transformation and delivery really from end to end, thinking about AI, from identifying problems, strategizing a solution, designing and building the solution, and ultimately rolling it out. It’s not just one piece. It’s really that full end to end process and a great place, I think for us to leave folks with maybe some inspiration and thoughts. Could you talk about the importance of that and how the overall approach is with AI and that it’s holistic within an organization?

Saima: Sure. I would say we’ve all heard number of times on all different kinds of forums we hear, if we fail to plan, we’re planning to fail. So the first thing that I can say with the transformation mindset is the value or the importance of strategy leaders starting by identifying the biggest problems, the most impactful areas like you said, end to end, start by identifying the biggest areas of opportunity. Then of course, Anuj mentioned this as well, and you have to do it in collaboration. No one group can do it alone. Sometimes prioritizing by even return on investment for a given business segment may not be sufficient without cross collaborating, going multidisciplinary. You want to ask different groups that may be impacted by the opportunity that you are getting to invest in. So engaging more and more partners from your other organization, leadership, business folks, the risk and compliance folks, the enterprise technology folks, all of that really will help you put the two and two together in terms of calculating the return on investment.

Next, I think it’s also vital for us to think, hey, sometimes people when you’re starting, they would feel like, “Oh, I have to socialize so much, I just want to go get it done.” That’s the natural instinct of those who can actually hands-on get it done, which is true. I mean, I have to tell all my data scientists to slow down at times because yeah, they’re excited. They want to build POCs and show that, but then if you’re not doing the upfront ROI calculation and prioritization by value and the overall impact in the organization, we can end up spending and wasting a lot of time. And you all know time is money. We say that, but in this case, we are actually truly wasting both time and money, so it’s like double the loss.

The second point that I would mention, Anuj briefly hit on that one as well, the technical feasibility. We do not want to boil the ocean. We want to look at the technical feasibility, and as we are doing that, we want to make sure that you’re aligning it with the right set of tools and technologies right? You want to know… Now that everybody knows about gen AI because of ChatGPT, that’s not your only tool, right? There are so many other traditional AI techniques, algorithms, tools that you may have available that can do better of a job at much less cost. So you have to be reasonable in finding the design, really proposing something that’s best, really the biggest bang for the buck. Now, if you are not as an organization, as a bank, if that’s what you’re asking in particular, if you’re not investing upfront time in assessing, ranking, and then filtering further down based on technical feasibility, and one more, the change readiness, is this group ready to adopt this that we are doing, and you have to constantly work on making sure that is happening, you can end up wasting time.

So if you have done that, it’s not just done there. Okay, you identified the opportunity, you designed the best architecture, you’re using the best in class cutting edge AI tools that are at your disposal, or you’re bringing in something new, you deployed the solution and you’re working on evolving. At this point, you have to then consider what can I do as an AI leader to consider expanding the deployed applications for optimal use? You have to ensure effectiveness of what you have deployed, right? You want to tie it back to the scope of the application so you can show clear results that can be demonstrated for the business partners. And don’t forget, just like anything new, this new AI solution that, I mean, you may have hundreds of AI solutions and we do as well, more than hundreds, but you still need for every new launch, it needs love and care to scale and improve adoption, and then you have to continue to have this feedback mechanism from the systems and from the people, right?

You’re not just listening to the feedback from the monitoring tools that you have. Yes, that’s absolutely vital, but listening to the feedback from your leadership as well as from actual end users from your operations team, let’s say, who are using the tool or the customers who may be using the tool, getting that feedback incorporated into your refinements is essential, right? At the end, you want to make sure that you have a good governance mechanism on monitoring the performance of the models, the utilization of the tools that you’re deploying, how well are these being adopted, are there pain points or maybe missing things or gaps that you have not yet deployed, but others really would be curious to get them involved, included in your next releases and whatnot, and watching closely the metrics of containment, are people staying in here or they’re choosing to go to another channel instead?

So overall, I say any organization that has to look to adopting AI, they have to do it holistically, or there are changes of ad hoc success, but that cannot be mirrored in other parts of the organization if it is not considered carefully. And last but not the least, you heard Anuj mention that, I said that earlier as well, there are different flavors of AI. Don’t just get hung up on generative AI. We are excited about what gen AI can do, and we are seeing such positive results and opportunities, high value, tremendous value, but it also comes with a lot of additional risk assessment, additional governance processes that you have to set up, a lengthy, much lengthier validation process. Your testing framework for generative AI has really, I would say, made your SDLC haywire. Your SDLC used to be you spend a little time in requirements, a lot of time in… little more in design than in development and then some in testing and so on.

Now the development is done instantly. It’s the validation or the testing that’s taking the most time, so you want to choose the best tool in technology there. Document AI, we talked about that. I mean, I would say in a bank particularly, document AI probably is still… after machine learning, the traditional data-based ML models and statistical models and whatnot, AI ML models also that has been here for multiple decades. Document AI at this moment, still after almost 10 years of document AI presence in large organizations, we still have the highest potential in the doc AI space.

Now, yes, we could solve some of our doc AI problems with generative AI, but you would still see the prevalence of document intelligent processing, so you’ll have to be thoughtful. You have to learn your skills. You’ll have to learn the art of weaving your solution together, orchestrating to use the best strength of each of the tools that you have at hand, because not everything can be solved by one tool, and the more you learn and practice, you’ll get better at preparing your own decision tree, and your decision tree should tell you which tools and techniques to use for different kind of situations. But almost certainly you will always have some decent way of solving your problem using artificial intelligence. I mean, I would say really always. And if not, artificial intelligence, it’s like automation. Automation is possible for so many different levels. I can just close by saying that as excited as we are, as we are evolving and transforming using the applications of AI, I wish our listeners who are on the same road, they have a beautiful road to travel ahead and I can say have a nice journey. Thank you.

Holly: Well, great. Great advice for us to end on today. Saima, Anuj, thank you so much for sharing all the good work that you’re doing, how you’re doing it. I think you’ve left people with a lot to think about in terms of the opportunities that AI presents in different areas and how to overcome some of those challenges as they look at getting buy-in, implementing, really getting their staff excited about it. And also want to thank all of you who are watching. Be sure to check out all of the great content in this BAI Deep Dive series on AI and head to bai.org for more insightful thought leadership on the key issues financial services leaders are facing. Thank you so much.

Anuj: Thank you.

Saima: Thank you.