Shadow AI has emerged as a silent disruptor, posing both challenges and opportunities for organizations across the globe

Shadow AI, Navigating the Shadows in GenAI

AI Firewall,Data Protection,Access Management,Shadow AI,Top 5 LLM Security Risks

ByAditya Soni
March 22, 2024

Summary
 

In the rapidly evolving digital landscape, Shadow AI has emerged as a silent disruptor, posing both challenges and opportunities for organizations across the globe. As departments outside of traditional IT channels increasingly deploy AI solutions to enhance efficiency and decision-making, the risks associated with these unsanctioned initiatives become more pronounced. This comprehensive guide explores the concept of Shadow AI, uncovers the multifaceted risks it presents, and lays out a detailed blueprint for organizations seeking to harness the benefits of AI while mitigating its inherent risks.

What is Shadow AI?
 

Over the last twenty years, businesses have faced the hurdles of employees bringing their own devices and using their personal technology at work, a phenomenon known as shadow IT. Now, companies are dealing with a new trend in artificial intelligence. This involves employees using AI tools meant for general consumers in professional settings, a practice we’re referring to as Shadow AI.
Shadow AI refers to the development and utilization of artificial intelligence applications within an organization without explicit oversight or approval from central IT.

What is driving Shadow AI?
 

Several factors contribute to the rise of Shadow AI within organizations:

Rapid Technological Advancement: The pace of technological innovation encourages departments to quickly adopt new AI tools to gain a competitive edge.
IT Bottlenecks: When IT departments are overwhelmed or slow to respond, other departments might take matters into their own hands to avoid delays.
Lack of Awareness: There is often a gap in understanding the importance of compliance and security standards outside the IT department.

What are the risks and challenges of Shadow AI?
 

The unchecked growth of Shadow AI carries significant risks that can undermine the very benefits it seeks to provide:

Security and Privacy Concerns: Shadow AI applications may not be subject to rigorous security checks, increasing the risk of data breaches and privacy violations.
Regulatory Non-Compliance: Operating outside the oversight of IT governance, Shadow AI initiatives may fail to comply with industry regulations, exposing the organization to legal penalties.
Resource Fragmentation and Inefficiency: Duplicate efforts and incompatible systems can lead to resource wastage and operational inefficiencies.
Ethical Dilemmas: Without proper oversight, AI applications might be developed without considering ethical implications, leading to biased or discriminatory outcomes.

The challenges posed by Shadow AI are not only theoretical but have also manifested in significant real-world issues, as seen in the case of Samsung. The company was forced to ban the use of generative AI tools like ChatGPT.

Some Wall Street banks, including JPMorgan Chase & Co, Bank of America Corp, and Citigroup Inc, either banned or restricted the use of ChatGPT, these banks recognized the potential security risks associated with the use of generative AI platforms and took proactive measures to prevent data leaks and protect their intellectual property.

What are the strategies to Manage Shadow AI?
 

Leveraging Technology to Centralize AI Management: AI management platforms and tools can provide a centralized overview of all AI applications within the organization, allowing for better control and management. These tools can help in:

Monitoring AI Applications: Identify and assess all existing AI tools and projects across the organization.
Assessing Risks: Evaluate the security, compliance, and ethical implications of AI applications.

Building an AI Governance Framework: An AI governance framework establishes the rules of engagement for AI projects, detailing the processes for approval, development, deployment, and monitoring. This framework should:

Define AI Ethics and Principles: Set clear ethical guidelines for AI development and use within the organization.
Establish Approval Processes: Implement a streamlined process for departments to propose and gain approval for AI projects.
Set Security and Compliance Standards: Outline mandatory security protocols and compliance checks for all AI applications.

Cultivating a Culture of Transparency and Collaboration: A culture that promotes open dialogue and collaboration between IT and other departments can significantly reduce the appeal of pursuing Shadow AI initiatives. Encouraging departments to share their technological needs and challenges can foster a more cooperative approach to AI development, ensuring that projects are aligned with organizational standards and goals.

Educating Stakeholders on the Importance of Governance: Ongoing education and training for all stakeholders involved in AI development are crucial. Workshops, seminars, and resources on the importance of security, compliance, and ethical considerations in AI can raise awareness and foster a more responsible approach to AI projects.
Implementing Continuous Monitoring and Evaluation: Regular audits and reviews of AI projects can ensure they remain compliant with organizational policies and regulations. This continuous monitoring process helps identify potential issues early, allowing for timely interventions to mitigate risks.

Conclusion
 

Shadow AI shows us both sides of tech innovation – it brings great benefits but also new problems. By getting to grips with Shadow AI and having a solid plan to handle it, companies can use AI to spark new ideas and work smarter, without the downsides.

Moving from hidden risks to clear benefits doesn’t mean stopping innovation. It means guiding it with good management, teamwork, and doing the right thing. This way, companies can use AI as a strong force for moving forward, making sure it’s safe, follows the rules, and is fair to everyone.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.
Unlock Zero Trust Security for
GenAI and Data Access

Request a Demo

The combination of GenAI with the multiple components in the software supply chain introduces risks which need to be addressed to avoid losses

Software Supply Chain Risks in Generative AI

Software Supply Chain,AI Firewall,GenAI,Zero Trust Security,Top 5 LLM Security Risks

ByMatt
March 4, 2024

Summary
 

Gartner report forecasts that by 2025, about half of all organizations will experience a software supply chain attack, highlighted by OpenAI’s temporary shutdown of ChatGPT due to a ‘Redis’ library vulnerability, risking user data exposure.
The software supply chain poses risks to Generative AI (GenAI) due to inherited vulnerabilities, third-party dependencies, an expanded attack surface, the potential for compromised data and code, and regulatory compliance challenges, all of which can significantly impact the security and integrity of GenAI projects..
CISOs See Software Supply Chain Security as Bigger Blind Spot Than GenAI

Why OpenAI disabled ChatGPT?
 

Today’s software development heavily relies on third-party codes, libraries, and increasingly, Generative AI (GenAI), making it possible to build up to 90% of an app without starting from scratch. This method streamlines app creation and saves time but also raises security risks. A Gartner report predicts that by 2025, nearly half of all organizations will face an attack on their software supply chain, a threat growing in frequency and complexity.

On March 20th, OpenAI briefly disabled ChatGPT due to a vulnerability in the ‘Redis’ open-source library, affecting the software supply chain. This issue led to a breach exposing user data like chat history titles, names, email and payment addresses, credit card types, and the last four digits of card numbers.

How is Software supply chain a risk for GenAI?
 

Threat to Software Supply Chain poses a risk to GenAI due to:

Inherited Vulnerabilities: Generative AI (GenAI) systems are prone to the same security weaknesses as traditional software, due to shared supply chain components and dependencies.
Third-Party Dependencies: GenAI systems’ reliance on external software components introduces vulnerabilities from the broader software supply chain into GenAI projects.
Expanded Attack Surface: The incorporation of numerous third-party components in GenAI systems enlarges the attack surface, making it more challenging to secure against breaches originating from the supply chain.
Compromised Data and Code: If any part of the supply chain is compromised, whether through malicious code in libraries or tainted datasets, it directly affects the integrity and functionality of GenAI applications.
Regulatory Compliance: The software supply chain’s complexity, including compliance with legal standards for data protection, impacts GenAI projects that use these components, making them susceptible to regulatory risks.

What are some common supply chain attacks?
 

Cycode’s inaugural 2024 State of ASPM report reveals significant AppSec challenges: 78% of CISOs find current attack surfaces unmanageable, 90% see a need for better security-development team collaboration, and 77% view software supply chain security as a more critical blind spot than Gen AI or open source issues.

Browser-based Attacks: Run harmful code in users’ browsers, targeting JavaScript libraries or extensions, and can steal sensitive information stored in the browser.
Software Attacks: Disguise malware within software updates, as seen in the SolarWinds incident, enabling automatic download and infection of devices.
Open-source Attacks: Exploit vulnerabilities in open-source packages, potentially allowing attackers to modify code or embed malware to gain access to systems.
JavaScript Attacks: Take advantage of vulnerabilities in JavaScript, or insert malicious scripts into webpages that execute upon loading.
Magecart Attacks: Use malicious JavaScript to steal credit card details from online checkout forms through “formjacking.”
Watering Hole Attacks: Target websites frequented by many users to exploit vulnerabilities and distribute malware to visitors.
Cryptojacking: Hijack computational resources for cryptocurrency mining through malicious website code, open-source scripts, or phishing links.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

ShadowAI is becoming a growing concern similar to ShadowIT as it can lead to loss of sensitive data ranging from IP, Private data
Shadow AI and its threat on GenAI and data protection

AI Firewall,Data Protection,Access Management,Shadow AI,Top 5 LLM Security Risks

ByMatt
March 4, 2024

Summary
 

Surveys reveal that while 76% of IT leaders predict a transformative role for Generative AI in their companies, with 65% expecting benefits within a year, there’s a growing trend of employees using GenAI tools without official authorization.
Samsung banned generative AI tools like ChatGPT due to leaks of confidential information, highlighting ongoing concerns about AI security and privacy risks despite the evolving landscape.
Shadow AI poses greater risks than Shadow IT by exposing every level of an organization to potential data breaches, AI-generated inaccuracies, unauthorized access issues, and regulatory non-compliance.
Banning generative AI in the workplace can inadvertently increase risks by driving its use underground, bypassing security measures and highlighting the need for more nuanced management strategies.

What research from Dell, Salesforce and Forbes reveals about Shadow AI?
 

According to a recent Dell survey, 76% of IT leaders believe GenAI will play a crucial and potentially transformative role in their companies.
The same survey reveals 65% of IT leaders expect to see tangible benefits from GenAI within the next year.
A global Salesforce survey of over 14,000 employees in 14 nations found a significant number of generative AI (GenAI) users in the workplace are using these tools without official training, guidance, or authorization from their employers.
Forbes reports an increasing trend in the unauthorized use of generative AI within companies.

The rapid adoption of GenAI poses challenges, especially when employees use GenAI tools not officially sanctioned by the company. The trend of “shadow AI” usage heightens organizational risks, raising concerns around data security, regulatory compliance, and privacy.

Samsung’s Data Alarm: Shadow AI Emerges from the Shadows
The challenges of Shadow AI are significant, as evidenced by real-world incidents. Take for example the Samsung case :

Samsung banned use of generative AI tools like ChatGPT after they found that ChatGPT possessed confidential information.
The first incident was involving an engineer who pasted buggy source code from a semiconductor database into ChatGPT, with a prompt to the chatbot to fix the errors.
The second instance, an employee wanting to optimize code for identifying defects in certain Samsung equipment pasted that code into ChatGPT.
The third leak resulted when an employee asked ChatGPT to generate the minutes of an internal meeting at Samsung.

Gartner, as early as 2019, pinpointed security as a critical strategic trend in AI. Although the AI landscape has evolved since then, privacy risks remain a paramount concern, especially given the fast-paced changes in the AI field.

What is the impact and challenge of Shadow AI on your organization?
 

The impact of Shadow AI is expected to be greater than Shadow IT, as highlighted by cio.com. Unlike Shadow IT, where risks were mostly limited to developers, generative AI exposes every user in an organization, from admins to executives, to potential errors.

From discussions with enterprise clients, several emerging challenges associated with shadow AI have been identified:

Data Protection: This includes the possibility of users inadvertently sharing confidential data with GenAI, leading to unintentional disclosure of sensitive information.
AI “Hallucinations”: This refers to instances where the AI chatbot generates inaccurate or misleading information, which can lead to incorrect decisions or misinterpretations.
Access Management: There’s a risk of the GenAI service provider gaining unauthorized access by employees. This could happen through human review of the customer’s data inputs to the AI system.
Non-compliance with Regulations: The use of unapproved GenAI tools can lead to breaches in cybersecurity and data privacy standards, failing to meet legal and regulatory compliance.

Will banning GenAI help organizations?
 

Banning Gen AI in workplaces might reduce visible risks but can drive its use underground, making it harder to control and  potentially increasing risks.
Covert use of Gen AI bypasses security controls and oversight, potentially escalating risks rather than mitigating them.
Outright bans can be counterproductive; more nuanced strategies are needed to manage Gen AI use effectively.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

Intellectual property such as sensitive company information stands at a risk of loss when GenAI is adopted within organizations.

Intellectual Property Meets GenAI in the Battle for
Ownership

IP Protection,AI Firewall,GenAI,Source Code,Top 5 LLM Security Risks

ByAditya Soni
March 1, 2024

Summary
 

GitHub faced a lawsuit for training Copilot on open-source code without respecting licensing terms, and Samsung banned AI tools after employees’ use led to potential data breaches, underscoring the pressing IP and privacy issues with generative AI use.
Even with Intellectual property laws in place, IP and security emerge as top concerns for Fortune 500 companies using generative AI.
Companies are losing sight of IP protection as they prioritize rapid innovation and market expansion over the meticulous management of their intellectual assets, risking the dilution of their most valuable assets in the process.

What do we learn from Google, GitHub and Samsung about Intellectual Property rights?
 

The GitHub case isn’t the first-time technology and IP have crashed into each other. Google successfully defended itself against a lawsuit by arguing that transformative use allowed for the scraping of text from books to create its search engine.

The first Open Source Copyright Lawsuit challenged GitHub copilot

According to the pursuers, by training their AI systems on public repositories the defendants have violated the rights of many developers who posted code under different open-source licenses that require attribution, including the MIT, GPL, and Apache licenses.

Samsung had banned usage of AI tools

Samsung employees, in an effort to address work-related challenges, inputted sensitive company data into ChatGPT, including proprietary source codes and confidential meeting transcripts. Such actions were taken to leverage the AI’s capabilities for troubleshooting and optimizing work processes but inadvertently risked significant data exposure and security breaches.

These events have underscored the unique risks posed by generative AI technologies for Intellectual property. This misuse of generative AI highlights the critical need for stringent data protection measures and adherence to privacy protocols.

What do Fortune 500 companies think about GenAI and Intellectual Property?
 

A recent Acrolinx survey of Fortune 500 companies found Intellectual property concerns to be prominent for enterprise leaders considering generative AI technology.

Asked what their primary concern about the use of generative AI is, 25 of the 86 respondents selected intellectual property as their biggest concern.
This was followed closely by customer security compliance concerns, which received 23 votes. Comparatively, the other three answers (bias and inaccuracy, privacy risk, and public data availability and quality) received between nine to 16 votes each.
A total of 86 companies participating in a survey may not seem to make for a statistically significant study, but that number represents 17% of the Fortune 500 universe, which is a significant representation (statistics show a viable sample size at a minimum of 5%)

Why are companies losing sight of IP protection? 5 steps to mitigate risks of IP loss
 

In today’s competitive market, the race to generate revenue often leads companies into a precarious situation, particularly when it involves the intricate world of intellectual property (IP) and its intersection with generative artificial intelligence (GenAI). As businesses strive to innovate and capitalize on their unique IP amidst stressful market conditions, a new challenge emerges: safeguarding the ownership and value of their intellectual creations. This urgency to push products to market, license IP to third parties, or develop cutting-edge software-as-a-service (SaaS) offerings can inadvertently result in the dilution of a company’s most valuable assets.

Businesses can adopt several strategies to avoid the loss of intellectual property (IP) when using Generative AI (GenAI) technologies. Given the complex and evolving nature of IP law in relation to AI, these measures are essential to mitigate risks and ensure compliance:

Limited Access: To minimize the risk of data breaches or unauthorized access, only limited access of project files and sensitive information should be confined to those directly engaged in the project’s development.
Clear Ownership Agreement: Before any work commences, clear contracts should be established, outlining the specific ownership rights of project intellectual property (IP), ensuring clarity and protection for all parties involved.
Secure Infrastructure: To protect against potential cyber threats, the maintenance of robust security measures, such as firewalls, encryption protocols, secure servers, and regular software updates, should be prioritized.
Non-Disclosure Agreements (NDAs): To safeguard confidential information and trade secrets, NDAs should be signed, creating a legally binding commitment to protect sensitive data.
Version Control Systems: Utilizing version control systems, like Git or SVN, should be a standard practice to monitor development changes, ensuring that all previous versions of the work are securely archived and retrievable when necessary.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

Personal and sensitive data, such as PII, PHI, and PCI, necessitating the implementation of AI guardrails for privacy protection

Privacy Protection: Can the Digital Generation Maintain Confidentiality

AI Firewall,Privacy Protection,Data Protection,GenAI,Top 5 LLM Security Risks

ByMatt
February 28, 2024

Summary
 

Generative AI’s potential to boost the global economy by up to $4.4 trillion is driven by its broad industry adoption and massive data use, but challenges like data leakage and privacy, highlighted by Google and OWASP, underline the importance of protecting data for secure AI applications.
Italy’s ban on ChatGPT, driven by data privacy issues, showcases how GenAI introduces complex privacy risks harder to address than those from past technologies.
How to protect Privacy while using GenAI

What Mckinsey, Gartner and Google have to say about Gen AI and Privacy Protection?
 

Generative AI could add $2.6 to $4.4 trillion annually to the global economy, according to McKinsey. Its impact is due to widespread adoption across various industries and the use of large data sets.
Data is a key factor, with organizations collecting huge amounts in the cloud for AI applications.
Gartner predicts that by 2025, Generative AI will generate 10% of all data, up from less than 1%.
Google surveys at Modern Security event reveals “data leakage” and “privacy” as top AI risks.
OWASP lists these concerns in the top 10 risks for foundational applications.

Why did Italy ban ChatGPT? Is GenAI a risk to privacy protection?
 

Italy’s decision to ban ChatGPT was primarily due to concerns over data privacy violations as indicated by the Italian Data Protection Authority, Garante. The ban highlighted serious issues related to the handling of personal data by AI systems like ChatGPT. Garante’s investigation pointed to potential breaches of the European Union’s General Data Protection Regulation (GDPR), focusing on the improper collection and processing of personal data for training AI algorithms, among other concerns.

Generative AI poses a unique and significant risk to privacy, making it a more concerning issue compared to traditional technologies because of the following reasons:

Complexity and Opacity (Black Box Issue): Many generative AI models, especially those based on deep learning, are complex and not easily interpretable even by their creators. This “black box” nature makes it difficult to understand how data is being processed and used, complicating efforts to ensure privacy and comply with regulations.
Permanence and Indelibility: Content generated by AI, especially false or sensitive information, can spread quickly and be difficult to remove or correct once disseminated across the internet, leading to long-term privacy issues.
Scalability of Attacks: Generative AI can automate and scale social engineering attacks, phishing, and other malicious activities that compromise privacy. The efficiency and effectiveness of these AI-driven attacks pose a significant risk to personal and organizational privacy.
Inference and re-identification attacks: AI’s ability to connect seemingly unrelated data can expose sensitive information, necessitating strong security measures.
Data breaches and security risks: As AI integrates deeper into systems, it becomes a bigger target for cyberattacks, leading to severe consequences like identity theft and privacy violations.
Potential for Misuse: GenAI can create convincing fake content, posing risks of fraud and unauthorized use of identities, leading to privacy breaches.

In summary, the convergence of generative AI’s extensive data needs, its ability to create convincing fake content, and its complex, opaque decision-making processes create a potent mix of privacy risks that are more challenging to manage than those posed by previous technologies.

How can you protect privacy while using GenAI?
 

Managing and protecting personal and business privacy becomes crucial. As GenAI tools like Google Bard, Microsoft’s AI integrations, and ChatGPT become more ingrained in our daily digital interactions, understanding and utilizing available data protection measures is key.

Access management: By managing user access, organizations can prevent identity theft, data breaches, and unauthorized entry to sensitive business information. Identity and Access Management (IAM) systems are effective in halting the spread of compromised login details, blocking unauthorized network access, and safeguarding against various cyber threats like hacking, ransomware, and phishing.
Encrypted Communication Channels: Secure data using encryption during transmission and while stored.
Regular Privacy Setting Updates: Frequently update privacy settings in GenAI tools to match updates and new features.
Anonymization Techniques: Use data anonymization before feeding information to GenAI to preserve privacy.
Staying Informed About Data Policies: Keep up-to-date with GenAI services’ data policies to make informed privacy decisions.
Selection of Privacy-Focused Tools: Choose GenAI tools with robust privacy and data protection features.
Google Bard’s Data Management: Utilize Google Bard’s options for auto-erasing, manual deletion, or indefinite data retention to manage AI data while preserving privacy.
Microsoft’s AI Data Management: Leverage the integrated security and privacy settings in Microsoft’s products, including GenAI tools, to control and secure data.
ChatGPT Web Privacy Controls: Access ChatGPT’s web privacy controls to restrict data usage for model training and manage chat history.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

In Q2 FY2024 earnings call, Satya Nadella offered a compelling proxy for IT-Security leaders around GenAI. Likening it to a comprehensive database of all enterprise documents and communications, but with queries being made in natural language. This analogy is a powerful one, inviting us to consider how GenAI security should be thought about, maybe not very different from traditional data & infrastructure security.
Build a Robust AI Security Strategy with a Simple GenAI Maturity Framework

AI Firewall,Zero Trust Security,GenAI Security Maturity Framework,Top 5 LLM Security Risks

BySatish Veerapuneni
February 20, 2024

In Q2 FY2024 earnings call, Satya Nadella offered a compelling proxy for IT-Security leaders around GenAI. Likening it to a comprehensive database of all enterprise documents and communications, but with queries being made in natural language. This analogy is a powerful one, inviting us to consider how GenAI security should be thought about, maybe not very different from traditional data & infrastructure security.
Let’s take a moment to step back and envision what GenAI’s impact can be for businesses, both big and small. Let’s look into some examples from major software companies that illustrate how GenAI has influenced their performance. Consider for e.g. Microsoft, where Azure AI boasts over 53,000 customers. Or GitHub, with more than 50,000 organizations using GitHub Copilot. And, ServiceNow, which has seen a staggering 50%+ improvement in developer productivity by deploying internal tools powered by GenAI. They’ve even introduced a Pro Plus SKU, monetizing it at a 60% premium over their base SKU. These examples underscore a critical point: the importance of leveraging GenAI for all companies aiming to enhance their bottom line and drive top-line growth is undeniable.

Let’s come back to GenAI adoption concerns from CXOs, a recent Gartner study, surveying across ten industries, highlights the primary adoption challenges for GenAI as shown below. The concerns range from privacy issues to intellectual property loss. Over the past 6–9 months, my discussions with CISOs and CIOs across diverse sectors, from Financial Services to Retail, have revealed that their adoption maturity regarding GenAI typically fall in three distinct phases.
Foundational: This phase is all about the foundations — how can employees and contractors use “Public GenAI” safely and responsibly? Public GenAI refers to tools like ChatGPT, Perplexity, Claude, GitHub Co-Pilot, Bard, and many others.
Basic: The focus here shifts to organizational efficiency, or in simpler terms, reducing costs. In this phase, companies begin deploying internal GenAI-powered tools for customer support, developers, product etc., aiming to boost efficiency.
Advanced: The question here is about value addition — how can GenAI be utilized to increase revenue for my business? A prime example is ServiceNow’s Pro-Plus SKU, thanks to the added value of integrated GenAI.
Implementing GenAI Security should mirror a similar adoption maturity model — we call this GenAI Security Maturity Model. It offers an approach for Infrastructure, Security, and Privacy leaders to securely adopt GenAI in a manner that aligns with their organization’s GenAI readiness and risk profile.

This GenAI Security maturity model has been a cornerstone of my discussions with CIOs and CISOs. Most have found it helpful to conceptualize Securing GenAI. While it’s not exhaustive, it certainly sparks thoughts among CXOs about how to navigate the GenAI landscape securely.
Let’s delve into some of the top risks discussed in the maturity model and some questions IT-Security leaders should consider as you adopt GenAI.
Shadow AI involves AI applications and systems used without formal IT department approval, akin to Shadow IT, posing significant security and compliance risks.

Key IT-Security Questions:

Which GenAI applications are being utilized by our end-users?
Are there enterprise applications integrated with GenAI?
How are we managing and governing sandboxed GenAI usage?

IP Protection is about safeguarding proprietary assets like sensitive financials, customer conversations, source code, and product documents. The incident with Samsung employees inadvertently sharing code with ChatGPT, leading to restricted GenAI access, is a case in point. This concern is echoed by many IP-conscious firms, including JPMC.
Cyberheaven’s analysis of 1.6 million users across various companies revealed about 1,000 IP violations in a week, involving internal data, client data, source code, and project files.

 Key IT-Security Questions:

What are our top 3–5 Intellectual Property categories (e.g., Code, Customer Data, Financial Projections)?
How are we safeguarding our source code from accidental GenAI uploads?
What additional security controls are in place for GitHub Co-Pilot usage?

Privacy Protection entails strategies and policies to protect personal information (PII, PCI, PHI) from unauthorized use, access, or disclosure.

Can we address this through existing educational and process controls
Are our current DLP solutions adequate, or will they trigger excessive alerts?
How are we managing DLP risks in User <> GenAI and App <> GenAI interactions?

Prompt Injection represents a sophisticated cybersecurity threat in GenAI, where direct attacks manipulate AI responses through malicious prompts, and indirect attacks subtly bias the output from GenAI.

Do we have external LLM-Integrated Applications in production?
What safeguards are in place against Prompt Injection Attacks?
How are we assessing risks related to Public AI Agents or Plug-Ins?

 
Supply Chain Risks arise from vulnerabilities in the LLM application lifecycle, including third-party datasets, pre-trained models, and plugins.

Do we use any internal or external LLM-Integrated Applications?
What models and datasets have we employed?
Are there any GenAI Plugins we interact with?

Summary
Satya Nadella provides a simple framework for internalizing GenAI for IT-Security teams across enterprises. In today’s world, becoming AI-enabled isn’t just an option; it’s a necessity. The success stories of forward-thinking companies using GenAI are not just inspiring but also illustrative of the potential that awaits those ready to embark on this journey.
When it comes to securing GenAI, it’s crucial to adhere to a structured approach: begin with the foundational elements, progress to basics, and then advance to more complex strategies.
If you’re interested in a deeper discussion or even in contributing to refining this perspective, I’d be delighted to connect. Feel free to send me a direct message on LinkedIn here. Let’s explore the possibilities of Securing GenAI with a Risk Maturity Framework that’s simple to operationalize.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo