Fulfilling the Promise of GenAI

Fulfilling the Promise of Generative AI A Strategic Path to Rapid and Trusted Solution Delivery Mike Leone | Principal Analyst ENTERPRISE STRATEGY GROUP may 2024 © 2024 TechTarget, Inc. All Rights Reserved.

Fulfilling the Promise of Generative AI 2 Introduction It’s clear that generative AI (GenAI) is no longer the next new thing. It’s not around the corner, up and coming, imminent, or the object of technologists and think tanks. It is a here-and-now tool that is already transforming how organizations conduct their business activities and accomplish their most critical goals. More than that, it has unlocked a wealth of innovation shaped not only by technical geniuses but also by pragmatic business leaders. Anyone reading this eBook certainly has heard of GenAI, and chances are very high that most business and IT professionals have been at least tangentially involved in GenAI in their organizations. Whether it’s in conceiving, planning, developing, deploying, or managing a GenAI project, this groundswell of “first-generation” GenAI adoption has already far exceeded critical mass. That’s why it’s not simply the next new thing but rather an effective, highly leverageable tool to create new, almost unimaginable business value. © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

Fulfilling the Promise of Generative AI 3 Contents An Overview of GenAI and Why It Opportunities for GenAI Challenges, Roadblocks, and Speed Bumps: Matters So Much to Change … Everything Understanding the Pitfalls of GenAI PAGE 4 PAGE 5 PAGE 8 Balancing In-house Models With Proactively Prepping for A New GenAI Solutions Paradigm: Third-party Models GenAI Integration Motific, Outshift’s GenAI solution PAGE 11 PAGE 14 PAGE 16 © 2024 TechTarget, Inc. All Rights Reserved.

Fulfilling the Promise of Generative AI 4 An Overview of GenAI and Why It Matters So Much When TechTarget’s Enterprise Strategy Group took a deep look at GenAI and organizations’ plans for the technology, it uncovered a few truths. GenAI is being used in pragmatic, business-critical Cybersecurity Digital Cutting Automation Application (or even mission-critical) applications, such as:¹ resiliency transformation costs modernization Additionally, a major trend is the adoption of GenAI across a wide range of job functions and departments throughout the enterprise. Heading the list of departments that are planning to embrace GenAI are customer service (48%), marketing (45%), and software development (43%). A variety of other job functions are planning to adopt GenAI in the future, including research (39%), IT operations (38%), product development (37%), and cybersecurity (32%).² There is little debate that GenAI not only has gained traction in the enterprise, but that it has rapidly accelerated as well. GenAI pilots and sandbox projects have been deployed, results have been assessed, and organizations are putting together the financial, technological, and human resources necessary to make GenAI a central part of their long-term business strategy. That is not to say that every GenAI project will be an unfettered success, will run smoothly, will stay within budgetary guidelines, or will deliver the anticipated results. There still are many lessons to learn and obstacles to overcome. Figure 1. Top 10 Business Areas of Planned GenAI Usage Research 35% “Most organizations (58%) have Marketing 35% Software development 33% already deployed GenAI in production Product development 32% IT operations 28% systems, are experimenting with Customer service 26% Operations (non-IT) 22% GenAI, or are planning to adopt GenAI Cybersecurity 21% Sales 20% within the next 12-24 months.” Cloud infrastructure cost optimization 14% © 2024 TechTarget, Inc. All Rights Reserved. ¹Source: Enterprise Strategy Group Research Report, Beyond the GenAI Hype: Real-World Investments, Use Cases, and Concerns, August 2023. ²Ibid. Back to contents

Opportunities for GenAI to Change … Everything

Fulfilling the Promise of Generative AI 6 Opportunities for GenAI to Change … Everything GenAI’s ability to transform the way in which organizations use data for economic, operational, and even reputational gains is now widely embraced and established. If there is any lingering debate, it’s on how much, how fast, and how efÏciently GenAI can be leveraged to support more use cases. GenAI’s coming role in nearly every organization larger than a neighborhood lemonade stand is essential for a number of reasons, not the least of which is how to leverage the massive growth in data coming from more and more data sources. The evolution from the days of PC-based servers and legacy data centers is now complete: New, wider-ranging and more functional architectures, including edge systems, internet of things endpoints, and ubiquitous cloud computing, generate oceans of data— 66% a trend that will only accelerate. In addition to having more data—and more useful data—at its disposal, GenAI has benefitted from numerous other trends: of organizations say improving operational efÏciency is a AI infrastructure has made incredible New models are constantly GenAI and machine learning (ML) are gains in its ability to capture, process, emerging to speed and improve major shapers of organizations’ overall store, analyze, and share all this data— training and inference. drive to transform their data science primary business and actually make sense of it all in real initiatives. In fact, 66% of organizations time. Research indicates that high- say improving operational efÏciency is objective of their performance computing capabilities a primary business objective of their is the most commonly cited factor data science initiatives, followed by organizations consider important when improving product development and data science selecting AI infrastructure.³ innovation (60%).4 initiatives. © 2024 TechTarget, Inc. All Rights Reserved. ³Source: Enterprise Strategy Group Research Report, Navigating the Evolving AI Infrastructure Landscape, September 2023. 4Source: Enterprise Strategy Group Research Report, Decoding the Data Universe: The State of Data Science and Machine Learning, January 2024. Back to contents

Fulfilling the Promise of Generative AI 7 Improvements From GenAI Yield Use Cases Galore Many organizations already indicate that they are seeing numerous areas of substantial organizational improvements from GenAI, with primary benefits including improving processes and workflows (53%), supporting data analytics (52%), enhancing employee productivity (51%), improving operational efÏciency (48%), and improving the user/customer experience (46%).5 This has resulted in the emergence of more use cases, particularly ones with wide-ranging potential to improve efÏciency and effectiveness in achieving business goals. Certainly, content creation in all its forms—written, oral, artistic, visual, and more—is at the top of every organization’s GenAI use case portfolio, likely followed by everything from data analytics and software code creation to cybersecurity and customer experience. Corner ofÏce conversations, boardroom discussions, and everyday lunchroom conversations—every environment and scenario where employees gather to talk about the future of their organization—are all diving into the art of the possible when it comes to GenAI. And that could be one of the biggest benefits GenAI brings to organizations: the freedom and license to imagine, innovate, and invent like never before. The excitement and incentive for organizations to consider what feels like unlimited possibilities is being shaped by GenAI. Figure 2. Top 5 Primary Benefits of Using Generative AI 53% 52% 51% 48% 46% 3e+7d= 2e+8d= 1e+9d= 8d+2e= 6d+4e= Improve and/or automate Support data analytics Increase employee Improve operational Improve user/customer processes and workflows and business intelligence productivity efÏciency experience © 2024 TechTarget, Inc. All Rights Reserved. 5Source: Enterprise Strategy Group Research Report, Beyond the GenAI Hype: Real-World Investments, Use Cases, and Concerns, August 2023. Back to contents

Challenges, Roadblocks, and Speed Bumps: Understanding the Pitfalls of GenAI

Fulfilling the Promise of Generative AI 9 Figure 3. Top Challenges Faced When Implementing Generative AI Challenges, Roadblocks, and Speed Bumps: Understanding the Pitfalls of GenAI Employee expertise/skills 39% Ethical or legal considerations (bias and fairness) 32% Now that we’ve pumped up GenAI’s future and the opportunities for organizations to fully leverage it, it’s smart Data quality 31% to take a step back and acknowledge a few real-world truths. Algorithmic transparency/ understand limitations 30% Fulfilling all this potential won’t be easy; even when successful GenAI deployments have been made, there have Solutions are immature 28% been clear challenges. There are numerous issues that organizations have had to confront, and they will undoubtedly Regulatory compliance 25% continue to do so even as GenAI’s success stories build. These challenges (as shown in the research chart) are broad Technical complexity 25% in nature and highlight everything from people and ethics to Difficulty in integrating with existing systems/ 24% data and compliance to technical complexity and cost.6 tackling legacy systems Additionally, organizations need ways to manage and Cost 23% monitor AI, including models in development and production Monitoring for misuse 22% environments. MLOps is a new area where organizations need investment; after all, it’s hard to have trust in Coordination and oversight across the company 19% production-class GenAI if it’s not being regularly and closely monitored. Lack of use case 15% But MLOps does a lot more than monitor what GenAI is Data quantity 13% doing and how it’s doing it. Other areas, such as version control or performance optimization, are facets of MLOps Scalability 11% that are crucial for GenAI applications to function effectively. In addition, GenAI models require careful management like Impact on workforce 10% any other machine learning model. © 2024 TechTarget, Inc. All Rights Reserved. 6Source: Enterprise Strategy Group Research Report, Beyond the GenAI Hype: Real-World Investments, Use Cases, and Concerns, August 2023 Back to contents

Fulfilling the Promise of Generative AI 10 Moving From Development to Production “ Only 40% of organizations have a well-defined, formal process With MLOps for moving ML models into production environments.” Only 40% of organizations have a well-defined, formal process for moving Figure 4. Top Challenges When Managing the Deployment and Monitoring of Models ML models into production environments, highlighting the fact that organizations are not yet where they need to be when it comes to MLOps. And while another 48% have some formalized processes, they admit Difficulty managing multiple environments 35% improvements are needed. Either way, many organizations acknowledge Difficulty ensuring compliance with corporate that they face several significant challenges in managing deployment and governance policies 33% monitoring of models. At least one-third of organizations say they are confronted with issues relating to managing multiple environments (35%), Difficulty detecting and responding to data drift 33% ensuring compliance with governance policies (33%), and detecting and responding to data drift (33%).7 Even more important is the need for organizations to step up their Inconsistent model performance in production 29% progress toward improving the time to value of AI. For instance, over a Difficulty detecting and responding to model failures 29% three-year period, organizations said their ability to start seeing value from their AI initiatives immediately increased only slightly, from 7% to 11%. On a more positive note, the percentage of organizations saying they started Inefficient retraining processes 26% seeing value within one to three months jumped from 32% to 61% over that same three-year period.8 Difficulty managing dependencies 26% As important as these data points are in highlighting the challenges organizations must overcome in their GenAI journeys, many of the key challenges are quite visceral, prompting an emotional response about issues Using GenAI in a way to create Achieving the desired Ensuring the highest level of Achieving the anticipated that tend to keep executives, board a sense of responsibility, outcomes as rapidly and data security and protection level of financial return members, and stakeholders up at confidence, and trust. Doing efÏciently as possible. of sensitive, proprietary and business insights. night. These include: the right thing matters. Speed matters. information. Privacy matters. Results matter. © 2024 TechTarget, Inc. All Rights Reserved. 7Ibid. Back to contents 8Source: Enterprise Strategy Group Research Report, Navigating the Evolving AI Infrastructure Landscape, September 2023.

Balancing In-house Models With Third-party Models

Fulfilling the Promise of Generative AI 12 Balancing In-house Models With Third-party Models Organizations can approach supporting generative AI initiatives using large language models (LLMs) from different angles, whether that be leveraging third-party proprietary models, utilizing open source models as a starting point, or developing a net-new in-house proprietary model. And Enterprise Strategy Group research highlights that there is still a fair bit of uncertainty with which approach is best. While there is plenty of opportunity for third-party vendors and service providers in the GenAI market to provide proprietary pretrained models, it is clear that many organizations will rely on open source models to some extent. In fact, nearly one-third of organizations (30%) have plans to utilize an open source LLM as a starting point to develop their own GenAI solution in-house.9 This represents a set of organizations that might want more control, have their own data, and/or have in-house expertise. In addition, nearly 1 in 4 organizations (23%) plan to go the open source route but anticipate working with a third-party provider to help move development forward, while more than 1 in 4 (28%) will look to a third-party provider that offers access to a proprietary model via prompt or API.¹0 Figure 5. Approaches Organizations Are Actively Taking in Their Pursuit of GenAI 30% 28% 23% 9% 30+70+T 28+72+T 23+77+T 9+91+T We will utilize an open source We will work with a third- We will work with a third-party We will develop a net-new LLM and develop a generative party provider that offers provider that offers access to LLM entirely in house. AI solution in house. access to a proprietary an open source LLM and allows model via prompt and/or us flexibility to customize. API. © 2024 TechTarget, Inc. All Rights Reserved. 9Source: Enterprise Strategy Group Research Report, Beyond the GenAI Hype: Real-World Investments, Use Cases, and Concerns, August 2023. Back to contents 10Ibid.

Fulfilling the Promise of Generative AI 13 Balancing In-house Models With Third-party Models (continued) Leveraging Third-party Using an Open Developing a Net-new In-house Proprietary Models Source Model Proprietary Model Speed to market. Access to sophisticated AI capabilities Cost-effectiveness. Generally free to use, modify, and Complete customization. Built to spec, fulfilling the without the development time. distribute, reducing financial barriers to entry. unique requirements of the business. Pros: Less initial investment. No need for extensive research Flexibility and customization. Can be tailored to meet Competitive advantage. Offers the potential for unique and development resources upfront. the specific needs of an organization. capabilities not available to competitors. Proven solutions. Often backed by established Community support. Able to benefit from the knowledge Control over data. Data remains within the organization, companies with support and updates. and contributions of a global community of developers. mitigating privacy and security concerns. Cost over time. Licensing fees or per-use costs can add Resource intensity. Requires skilled personnel to High costs. Significant investment in terms of time, up, making it expensive in the long run. customize, maintain, and update. expertise, and financial resources for development and maintenance. Limited customization. Dependence on the provider’s Potential security risks. Open source projects might Cons: roadmap and priorities, which might not fully align with not always prioritize security updates, leading to Long development time. Can lead to slower market vulnerabilities. response compared to adopting existing solutions. specific business needs. Data privacy concerns. Sharing sensitive or proprietary Lack of formal support. Relying exclusively on Risk of failure. High investment with no guarantee of data with a third party can pose security risks. community support can be unpredictable and success; development might not result in a inconsistent. viable product. © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

Proactively Prepping for GenAI Integration

Fulfilling the Promise of Generative AI 15 Proactively Prepping for GenAI Integration Integrating GenAI with existing systems can be complex and challenging. Whether a retailer is concerned with using GenAI in concert with inventory management to reduce merchandise shrinkage or a media company is looking to optimize revenue opportunities in its subscriber management application, aligning GenAI tools with business-critical systems can be harder than it looks. Unquestionably, taking the right steps in GenAI integration with core systems requires the right talent, modern technology, and the proper use cases, as well as the appropriate level of financial investment. It also must be done as part of a thorough, open-minded analysis that starts with use cases and workloads. To make this integration process as efÏcient and successful as possible, consider it as part of a three-phase development and deployment process. In Phase 1, • Data preparation. organizations start with: • GenAI model selection. • Commitment to the right partner. • Integration approach and priorities, such as integrating to which systems, for which reasons, and in which order. • Model training. During Phase 2, • Methods for testing, analysis, and rearchitecting the solution. Organizations should be ready to repeat these to ensure there is no unexpected organizations settle on: variation in the results. • Decisions regarding APIs, such as if they are needed at all or which ones are necessary, and ensuring the proper documentation of those integrations. • Selection, installation, and testing of the AI infrastructure. • Installation and all relevant integrations. • Testing and analysis in real-world production environment. Finally, Phase 3 • Implementing adjustments, as necessary. is where much of the • Monitoring and maintenance. high-visibility activities • Conducting governance, risk, and compliance due diligence, such as audit trails, reports, and documentation for: ǝ Security. take place, including: ǝ Data protection. ǝ Privacy. ǝ Data governance. © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

A New GenAI Solutions Paradigm: Motific, Outshift’s GenAI solution

Fulfilling the Promise of Generative AI 17 A New GenAI Solutions Paradigm: Outshift Motific In order to embrace the vast potential of GenAI while also circumventing and overcoming its pitfalls, organizations need to think outside the box. While some organizations have tried—and some have succeeded— in building GenAI solutions from scratch, factors such as Capex costs, a lack of in-house skills, and unclear metrics for success have made it extremely challenging for organizations to move beyond pilots and sandboxes. This means organizations should consider new options for how they plan and implement their GenAI journey. One option organizations should consider is Motific, a SaaS-based solution designed to promote rapid, trusted delivery of GenAI apps built for organizations looking for faster time to GenAI value. Motific, which provides a single interface across an organization’s entire GenAI journey, comes out of the development labs of Outshift, Cisco’s incubation engine. Outshift focuses on creating and delivering leading innovations in emerging technologies such as Generative AI, Quantum, and Cloud Application Security. Motific provisions assistants and APIs to business and technical stakeholders, while providing integrated controls to protect sensitive, proprietary data. It also is engineered to conform to best practices for “responsible AI” in order to detect and mitigate the impact of risks that occur between user input and a model’s response. © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

Fulfilling the Promise of Generative AI 18 Motific’s goal is to align technical and business strategies in order to build and deliver GenAI-powered projects that speed time to innovation, reduce compliance risks, create a Motific’s three pillars for its GenAI SaaS environment are: sense of trust among stakeholders, and deliver keen business insights. 1. Accelerate innovation and delivery Motific’s approach helps organizations undertake their GenAI journey in several ways: Motific enables your teams to provision assistants and abstracted APIs • Motific’s architecture serves as a centralized hub that connects assistants, abstracted powered by your organization’s data sources with policy controls in minutes. APIs, knowledge bases, policies, and monitoring and intelligence tools in order to: Cut GenAI deployment times from months to days with built-in compliance controls ǝ  for overuse and overrun spending, as well as integrating with organizational data. 2. Reduce trust and safety risks Motific provides comprehensive policy controls for security, trust, access Allow developers to write code for their AI applications once and use it across control, and cost. ǝ  multiple service providers. Enable compliance in the face of a continually changing policy landscape with ǝ  3. Understand and optimize performance automated, built-in policy controls for sensitive data, including personally identifiable information, and security controls Motific delivers usage insights and intelligence for each user-model interaction, such as prompts for injections. enabling ROI analysis and selection of more suitable models. • Motific provides a deep business process and intelligence framework with ROI and cost analysis, including consolidated audit trail and key metrics tracking of all user requests. Finally, Motific provides enhanced business and operational value to organizations throughout their full spectrum of activities during their GenAI journey due to its access • Users can provision GenAI assistants and abstracted APIs with just a few clicks in to extensive technical and business resources through Outshift, Cisco’s incubation arm. Motific. These are customized with retrieval-augmented generation (RAG) on data sources for use out of the box or in the building of GenAI applications. Motific is working on the product continually for the benefit of customers. And, at the back end, organizations can get the help they need in the form of testing, validation, and performance-related metrics. • Motific offers the ability to prevent shadow IT practices by providing visibility into Motific’s goal is to become not just a superior technical solution but an end-to-end the use of uncertified third-party GenAI capabilities and by helping IT administrators business solution. provision certified and compliant alternatives. © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

Fulfilling the Promise of Generative AI 19 Conclusion Chances are good that, even before you read this eBook, you knew that GenAI is a big development in organizations’ efforts to use more and more data in new and astonishing ways. Yes, seeing instantaneous answers to our questions about how to cook a blueberry pie or to learn the lyrics to a foreign country’s national anthem can be pretty neat, but the real value in GenAI is turning data into insights we never even knew could be unearthed in order to achieve previously unreachable goals. Still, for all the exciting opportunities presented by GenAI, organizations also realize that there are significant challenges and obstacles that must be overcome in order to fulfill the technology’s full potential. Specifically, organizations need to address issues such as managing substantial Capex spending on AI infrastructure, securing massive amounts of data to protect privacy and other proprietary information, and using GenAI-created data in responsible, appropriate ways. The great news is that new solutions are coming to the forefront all the time—solutions that help organizations achieve trustful, rapid delivery of GenAI apps and, with them, stunning insights to help organizations make smarter, faster, more impactful decisions. Normally, doing that requires big commitments to on-staff expertise, big technology investments, and extensive knowledge in AI responsibility, safety, and security. Motific, an interesting and innovative SaaS solution from Outshift, Cisco’s incubation arm, helps organizations take their AI journeys from concept to sandbox to production deployment. This kind of end-to-end solution enables organizations to streamline their GenAI initiatives, with the right guardrails for responsible, safe, and secure practices, and with proper monitoring of costs, processes, and outcomes. LEARN MORE © 2024 TechTarget, Inc. All Rights Reserved. Back to contents

All product names, logos, brands, and trademarks are the property of their respective owners. Information contained in this publication has been obtained by sources TechTarget, Inc. considers to be reliable but is not warranted by TechTarget, Inc. This publication may contain opinions of TechTarget, Inc., which are subject to change. This publication may include forecasts, projections, and other predictive statements that represent TechTarget, Inc.’s assumptions and expectations in light of currently available information. These forecasts are based on industry trends and involve variables and uncertainties. Consequently, TechTarget, Inc. makes no warranty as to the accuracy of specific forecasts, projections or predictive statements contained herein. This publication is copyrighted by TechTarget, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of TechTarget, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact Client Relations at [email protected]. Enterprise Strategy Group is an integrated technology analysis, research, and strategy firm providing market intelligence, actionable insight, and go-to-market content services to the global technology community. © 2024 TechTarget, Inc. All Rights Reserved.