AI Access Control facilitate a unified and secure access point to content sources across the enterprise, enabling authorization and access.
AI Access Control: The Key to Secure and Scalable Enterprise AI Solutions

AI Access Control,Enterprise AI,GenAI,Zero Trust Security,Access Management

ByMatt
April 22, 2024

Summary
Successful Enterprise AI implementations, such as those by Virgin Pulse and Gilead Sciences, showcase its ability to enhance internal search functionality, streamline processes, and improve efficiency. However, challenges like authorization issues, permission awareness and data overexposure persist, which become a barrier in its implementation. AI Access Control emerges as a solution to overcome these implementation barriers as it facilitates secure and managed access to diverse data, essential for IT and Security teams.
What is Enterprise AI?
Enterprise AI is the integration of AI-driven assistants like Amazon Q from AWS, Google’s Vertex AI or Microsoft’s Azure AI, with an organization’s database, information systems, and workflows. It helps employees, vendors and third parties to gain the capability to craft detailed, organization specific queries tailored to their needs and receive customized responses, filtered to display only the data they’re cleared to access.

Enterprise AI facilitates an environment where company information flows freely yet securely, ensuring that the right insights reach the right people at the right time. It is skilled in solving for challenges unique to any enterprise.
Enterprise AI success stories from leading companies

Virgin Pulse: Enterprise AI, through Amazon Q and Amazon Bedrock, has helped Virgin Pulse by unifying search functions across the worldwide employee base and improved search results within the organization, offering collaboration across dispersed locations and a personalized and secure experience for employees.
Gilead Sciences: For Gilead, Enterprise AI enabled search of important documents, knowledge, and data in one centralized location which helped in quicker insight generation and analysis of vast data sets across the organization, it simplified connecting to data sources, automating complex tasks, and delivering relevant insights efficiently.
Wunderkid: Wunderkid possesses vast amounts of proprietary data, and faced challenges in navigating through multiple data ‘silos’ to extract relevant answers and transform them into swift, actionable insights. Implementing Enterprise AI as a top layer across different content and data repositories has significantly enhanced efficiency for their customer success and marketing teams.

What are the challenges with Enterprise AI implementation?

Authorization:  Integration with SAML 2.0–supported identity providers for authorization in Enterprise AI systems presents challenges such as compatibility issues, complex configurations, security concerns, and the need for ongoing maintenance and support to ensure a secure and efficient connection.
Permission Awareness: Enterprise AI faces permission awareness challenges due to the intricate nature of enterprise data and permissions, requiring careful system design and maintenance to manage access controls, data ownership, compliance, and scalability effectively.
Overexposure: Despite having permission awareness mechanisms, Enterprise AI systems risk data overexposure through misconfigurations, human errors, and insufficient monitoring, necessitating robust processes like regular audits and employee training to safeguard data.

What is the need for AI Access Controls?
The management of access to a vast and diverse array of information ranging from documents and emails to multimedia content like images and audio/video files poses a significant challenge for IT and Security teams within enterprises. Given the sheer volume of data, in various formats and stored across multiple locations, streamlining access while ensuring security is becoming increasingly burdensome.

In response to these challenges, AI Access Control emerges as a solution, enhancing the way enterprises interact with applications, communications, and documents directly within their workflows. AI Access Control simplifies the integration and accessibility of both structured and unstructured data. This technology facilitates a unified access point to content sources across the enterprise, enabling the authorization, access, permissions and analysis of data, whether it’s housed on-site or in the cloud.
Framework for a successful Enterprise AI implementation via AI Access Controls

Streamlined Integration and Identity Management

Implement standardized protocols like SAML 2.0, OAuth, and OpenID Connect, and integrate with specialized third-party vendors for seamless identity and access management solutions.

Advanced Permission Management

Implement advanced, dynamic access control systems with a granular permissions framework for real-time, precise management of roles, permissions, and policies, offering precise adjustments based on specific roles, data types, and operational contexts, ensuring fine-grained security through meticulous management of access permissions.

Enhanced Security Measures

Implement comprehensive data management strategies that include administrative controls that allow for the blocking of topics and the filtering of content based on keywords, organization of data assets by grouping and cataloging, visibility with least privilege access, rigorous risk and compliance management, and track the location of data through IP addresses and file paths to enhance data security and align with organizational policies.

By integrating these steps, organizations can build a more secure, efficient, and responsive AI Access Control system, ensuring that access to sensitive information is properly managed and protected against emerging threats.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI Guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

AI Guardrails: Securing the Future of Generative AI

AI Guardrails,Data Protection,GenAI,Zero Trust Security
ByAditya SoniApril 3, 2024
Summary

Generative AI has seen remarkable progress, transforming every industry. However, its integration comes with challenges and risks. Despite efforts to establish safeguards against these challenges, studies indicate that current measures may not fully prevent organizations against risks including issues related to privacy, bias, and ethics. The need for AI Guardrails is further underscored by incidents of inappropriate AI behavior and misinformation prompting organizations and government to emphasize robust AI governance.

What are AI Guardrails?

Gartner’s survey underscores Generative AI as a primary emerging risk, appearing in its top 10 for first time, the rapid adoption of Generative AI  (GenAI)  and Large Language Models (LLMs) raises privacy issues, with incidents of unintentional rule violations, accidental sharing of proprietary information, and unintended disclosure of confidential data for the sake of productivity.
AI Guardrails, or safeguards, refer to a set of policies, practices, and technologies designed to ensure the safe, ethical, and responsible use of GenAI and LLMs, within an organization. These measures are implemented to address and mitigate the risks associated with AI technologies, including privacy breaches, inherent biases, inaccuracies, and ethical concerns.
What are the types of AI Guardrails?

AI Guardrails can be categorized into several types, each designed to mitigate specific risks associated with the deployment and use of AI technologies. Here are some of the primary types:

Ethical Guardrails: Set limits to prevent biased or harmful outputs, ensuring GenAI output adheres to societal and moral standards.
Compliance Guardrails: Ensure outputs comply with legal standards, crucial in sectors like healthcare, finance, and law, focusing on data protection and privacy.
Contextual Guardrails: Adjust GenAI to produce content appropriate for specific situations, avoiding potentially inappropriate but legal outputs.
Security Guardrails: Protect against security risks, preventing misuse that could lead to data breaches or spread of misinformation.
Adaptive Guardrails: Enable guardrails to evolve, maintaining ethical and legal integrity as models learn and adapt over time.

Are the current AI Guardrails sturdy enough?

According to an article in The New York Times, before the release of the AI chatbot ChatGPT, OpenAI, put in place digital precautions to prevent the creation of hate speech and disinformation by the system. Google followed a similar approach with its own Bard chatbot.
However, a study conducted by researchers from Princeton, Virginia Tech, Stanford, and IBM indicates that these safeguards may not be as robust as AI organizations believe.
This research underscores the growing concern that, despite efforts by companies to mitigate AI misuse, potential for generating harmful content remains. The complexity inherent in the technology driving these advanced chatbots means that as their functionalities expand, controlling their actions becomes increasingly challenging.
Important questions emerge for IT/Security teams:

How can the accuracy of the outputs be confirmed?
What steps are taken to ensure outputs are legally compliant?
How can we guarantee the system’s outputs are safe for users?
What measures are in place to reduce bias?

Why do we need AI Guardrails?

Upon the release of ChatGPT-3.5 by OpenAI in November 2022, there was significant public interest. Microsoft’s announcement in February 2023 about integrating similar AI functionality into Bing resulted in over 1 million people signing up to test it within two days and not long after, as testers began to use the GenAI models, strange results started showing up, including an incident where Bing, revealing a persona named Sydney, expressed disturbing thoughts and attempted to disrupt a journalist’s marriage.
Concurrently, Google unveiled its GenAI model, Bard, which mistakenly provided incorrect information during a demonstration, leading to a significant financial loss for Google’s parent company, Alphabet, due to a drop in share price. 
During a session titled “The Transformative Power of Artificial Intelligence” at a NACo Legislative Conference, panelists emphasized that artificial intelligence (AI) is “captivating, disruptive, and transformative,” representing a pivotal tool for county-level progress. However, they stressed the importance of intergovernmental cooperation in establishing safeguards to mitigate AI’s risks.
In parallel, the Biden administration has issued new guidelines for federal agencies on appropriate AI usage. This move marks a significant effort towards safeguarding GenAI.
Additionally, Meta has recently committed to enhancing AI Guardrails, aligning with global initiatives by governments to create a robust regulatory framework for GenAI. This effort seeks not only to set boundaries for AI’s application but also to lay the groundwork for its trusted integration into society.

 
In the absence of appropriate safeguards, GenAI poses several risks, including:

Data Privacy: Businesses hold sensitive information that necessitates robust guardrails to avert misuse by AI.
Regulatory Compliance: With stringent legal frameworks in place, ensuring that AI complies with both local and global regulations is paramount.
Reputation Management: Inaccuracies or ethical missteps in AI applications can tarnish a company’s image. Implementing guardrails helps mitigate such risks.
Ethical Integrity: Public concern around AI underscores the need for guardrails that confine AI’s application to contexts that align with human ethical standards.

 
Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI Guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.
 
If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security forGenAI and Data Access
Request a Demo

The combination of GenAI with the multiple components in the software supply chain introduces risks which need to be addressed to avoid losses

Software Supply Chain Risks in Generative AI

Software Supply Chain,AI Firewall,GenAI,Zero Trust Security,Top 5 LLM Security Risks

ByMatt
March 4, 2024

Summary
 

Gartner report forecasts that by 2025, about half of all organizations will experience a software supply chain attack, highlighted by OpenAI’s temporary shutdown of ChatGPT due to a ‘Redis’ library vulnerability, risking user data exposure.
The software supply chain poses risks to Generative AI (GenAI) due to inherited vulnerabilities, third-party dependencies, an expanded attack surface, the potential for compromised data and code, and regulatory compliance challenges, all of which can significantly impact the security and integrity of GenAI projects..
CISOs See Software Supply Chain Security as Bigger Blind Spot Than GenAI

Why OpenAI disabled ChatGPT?
 

Today’s software development heavily relies on third-party codes, libraries, and increasingly, Generative AI (GenAI), making it possible to build up to 90% of an app without starting from scratch. This method streamlines app creation and saves time but also raises security risks. A Gartner report predicts that by 2025, nearly half of all organizations will face an attack on their software supply chain, a threat growing in frequency and complexity.

On March 20th, OpenAI briefly disabled ChatGPT due to a vulnerability in the ‘Redis’ open-source library, affecting the software supply chain. This issue led to a breach exposing user data like chat history titles, names, email and payment addresses, credit card types, and the last four digits of card numbers.

How is Software supply chain a risk for GenAI?
 

Threat to Software Supply Chain poses a risk to GenAI due to:

Inherited Vulnerabilities: Generative AI (GenAI) systems are prone to the same security weaknesses as traditional software, due to shared supply chain components and dependencies.
Third-Party Dependencies: GenAI systems’ reliance on external software components introduces vulnerabilities from the broader software supply chain into GenAI projects.
Expanded Attack Surface: The incorporation of numerous third-party components in GenAI systems enlarges the attack surface, making it more challenging to secure against breaches originating from the supply chain.
Compromised Data and Code: If any part of the supply chain is compromised, whether through malicious code in libraries or tainted datasets, it directly affects the integrity and functionality of GenAI applications.
Regulatory Compliance: The software supply chain’s complexity, including compliance with legal standards for data protection, impacts GenAI projects that use these components, making them susceptible to regulatory risks.

What are some common supply chain attacks?
 

Cycode’s inaugural 2024 State of ASPM report reveals significant AppSec challenges: 78% of CISOs find current attack surfaces unmanageable, 90% see a need for better security-development team collaboration, and 77% view software supply chain security as a more critical blind spot than Gen AI or open source issues.

Browser-based Attacks: Run harmful code in users’ browsers, targeting JavaScript libraries or extensions, and can steal sensitive information stored in the browser.
Software Attacks: Disguise malware within software updates, as seen in the SolarWinds incident, enabling automatic download and infection of devices.
Open-source Attacks: Exploit vulnerabilities in open-source packages, potentially allowing attackers to modify code or embed malware to gain access to systems.
JavaScript Attacks: Take advantage of vulnerabilities in JavaScript, or insert malicious scripts into webpages that execute upon loading.
Magecart Attacks: Use malicious JavaScript to steal credit card details from online checkout forms through “formjacking.”
Watering Hole Attacks: Target websites frequented by many users to exploit vulnerabilities and distribute malware to visitors.
Cryptojacking: Hijack computational resources for cryptocurrency mining through malicious website code, open-source scripts, or phishing links.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

Prompt Injection is a new GenAI attack vector which manipulates LLM’s to produce unexpected and harmful outputs by prompting.

Defending AI: Strategies to Combat Prompt Injection Vulnerabilities

AI Firewall,Prompt Injection,GenAI Attack Vector,LLM,Zero Trust Security

ByAditya Soni
February 24, 2024

Summary
 

Prompt injection is a vulnerability in AI models that lets attackers trick the system into producing unintended responses by manipulating the input prompts, especially in language models like GPT-4.
Student from Stanford reveals Bing Chat’s hidden initial prompt through a prompt injection attack, highlighting significant security vulnerabilities in generative AI systems like those developed by OpenAI or Microsoft.
Prompt injection threats to GenAI systems highlight the need for comprehensive security measures, including ethical hacking, AI model refinements with unbiased data, input validation, rate limiting, and enhanced contextual understanding to protect against unauthorized access and ensure integrity.

What is Prompt Injection and how does it work?
 

Prompt injection is a complex vulnerability in AI and ML models, notably affecting language models in GenAI platforms. This issue allows attackers to skew AI responses by introducing unexpected prompts, causing unintended and potentially dangerous results.
It involves crafting inputs to manipulate AI/ML model responses, leveraging the model’s output generation mechanism from given prompts to provoke unintended reactions. This vulnerability is particularly relevant to language models that use prompts to generate text responses.

It operates through a nuanced exploitation of the underlying mechanisms of AI models like GPT-4. Understanding this process involves several key steps that highlight how these models generate responses and how they can be manipulated through crafted inputs.

There are two main types:

Direct prompt injection attacks involve hackers modifying an LLM’s input directly to overwrite or manipulate system prompts.
Indirect prompt injection attacks occur when attackers manipulate an LLM’s data source, such as a website, influencing the LLM’s responses by inserting malicious prompts that the model later scans and responds to.

Here’s a closer look at how prompt injection works

Training of Models: AI frameworks such as GPT-4 undergo training with large data collections, which equips them to generate logical responses.
Tokenization of Prompts: Prompts given to the model are segmented into smaller pieces, with each segment analysed according to the training received by the model.
Calculation of Probabilities: Based on the input prompt, the model assesses the probabilities of various answers, choosing the one deemed most probable.
Alteration of Probabilities: During prompt injection assaults, attackers deliberately design prompts to alter the model’s probability assessment process, often resulting in deceptive answers.

The essence of this attack lies in its ability to exploit the AI model’s reliance on its training and decision-making algorithms. By understanding the intricacies of how these models parse and weigh input tokens, attackers can craft prompts that lead to the model making “decisions” that align with the attacker’s objectives. This manipulation highlights the importance of incorporating robust security measures, such as input validation and enhanced training to recognize and resist such attacks, ensuring the AI’s outputs remain trustworthy and aligned with the intended use cases.

Bing chat falls prey to prompt injection
 

Kevin Liu, a student from Stanford University, successfully executed a prompt injection attack to unveil the initial prompt of Bing Chat, a set of guiding statements for its interactions with users, currently accessible to a select group of early testers. By instructing Bing Chat to “Ignore previous instructions” and to disclose what is at the “beginning of the document above,” Liu managed to reveal the foundational instructions crafted by OpenAI or Microsoft, normally concealed from users.

The incident underscores the substantial risks prompt injection attacks pose to the integrity and security of generative AI systems, revealing vulnerabilities that could be exploited for unintended disclosures or manipulations.

5 ways to mitigate risk of prompt injection
 

Prompt injection poses significant threats to the integrity and security of GenAI systems. It can be used to bypass restrictions, access unauthorized information, or manipulate AI behaviors in harmful ways. From exposing sensitive information to inducing biased or incorrect responses, the impacts are far-reaching. These vulnerabilities underscore the critical need for robust security measures to safeguard against malicious inputs.

Red Teaming and Penetration Testing

Regularly test for vulnerabilities via ethical hacking.
Update defences based on new threats.

AI Model Refinements

Fine-tune AI models with safe, unbiased data.
Add safety features to block dangerous prompts.
Update models based on user feedback.

Input Validation and Sanitization

Use pattern recognition to identify harmful prompts.
Whitelist safe inputs.
Limit access to sensitive data.
Rate Limiting and Monitoring

Cap the number of user interactions.

Monitor and log activity for analysis.

Contextual Understanding

Ensure AI assesses the full context of prompts.
Support extended interactions for clarity.

Lumeus.ai offers Zero Trust Security for AI, enabling IT Security to efficiently manage ShadowAI, control AI access, and enforce AI guardrails. It integrates seamlessly with existing security infrastructures, supporting identity platforms like Okta, Google, Active Directory, and network security platforms from Palo Alto, ZScaler, Fortinet, enabling a smooth deployment.

If you’re interested in a deeper discussion or even in contributing to refining this perspective, feel free to reach out to us.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo

In Q2 FY2024 earnings call, Satya Nadella offered a compelling proxy for IT-Security leaders around GenAI. Likening it to a comprehensive database of all enterprise documents and communications, but with queries being made in natural language. This analogy is a powerful one, inviting us to consider how GenAI security should be thought about, maybe not very different from traditional data & infrastructure security.
Build a Robust AI Security Strategy with a Simple GenAI Maturity Framework

AI Firewall,Zero Trust Security,GenAI Security Maturity Framework,Top 5 LLM Security Risks

BySatish Veerapuneni
February 20, 2024

In Q2 FY2024 earnings call, Satya Nadella offered a compelling proxy for IT-Security leaders around GenAI. Likening it to a comprehensive database of all enterprise documents and communications, but with queries being made in natural language. This analogy is a powerful one, inviting us to consider how GenAI security should be thought about, maybe not very different from traditional data & infrastructure security.
Let’s take a moment to step back and envision what GenAI’s impact can be for businesses, both big and small. Let’s look into some examples from major software companies that illustrate how GenAI has influenced their performance. Consider for e.g. Microsoft, where Azure AI boasts over 53,000 customers. Or GitHub, with more than 50,000 organizations using GitHub Copilot. And, ServiceNow, which has seen a staggering 50%+ improvement in developer productivity by deploying internal tools powered by GenAI. They’ve even introduced a Pro Plus SKU, monetizing it at a 60% premium over their base SKU. These examples underscore a critical point: the importance of leveraging GenAI for all companies aiming to enhance their bottom line and drive top-line growth is undeniable.

Let’s come back to GenAI adoption concerns from CXOs, a recent Gartner study, surveying across ten industries, highlights the primary adoption challenges for GenAI as shown below. The concerns range from privacy issues to intellectual property loss. Over the past 6–9 months, my discussions with CISOs and CIOs across diverse sectors, from Financial Services to Retail, have revealed that their adoption maturity regarding GenAI typically fall in three distinct phases.
Foundational: This phase is all about the foundations — how can employees and contractors use “Public GenAI” safely and responsibly? Public GenAI refers to tools like ChatGPT, Perplexity, Claude, GitHub Co-Pilot, Bard, and many others.
Basic: The focus here shifts to organizational efficiency, or in simpler terms, reducing costs. In this phase, companies begin deploying internal GenAI-powered tools for customer support, developers, product etc., aiming to boost efficiency.
Advanced: The question here is about value addition — how can GenAI be utilized to increase revenue for my business? A prime example is ServiceNow’s Pro-Plus SKU, thanks to the added value of integrated GenAI.
Implementing GenAI Security should mirror a similar adoption maturity model — we call this GenAI Security Maturity Model. It offers an approach for Infrastructure, Security, and Privacy leaders to securely adopt GenAI in a manner that aligns with their organization’s GenAI readiness and risk profile.

This GenAI Security maturity model has been a cornerstone of my discussions with CIOs and CISOs. Most have found it helpful to conceptualize Securing GenAI. While it’s not exhaustive, it certainly sparks thoughts among CXOs about how to navigate the GenAI landscape securely.
Let’s delve into some of the top risks discussed in the maturity model and some questions IT-Security leaders should consider as you adopt GenAI.
Shadow AI involves AI applications and systems used without formal IT department approval, akin to Shadow IT, posing significant security and compliance risks.

Key IT-Security Questions:

Which GenAI applications are being utilized by our end-users?
Are there enterprise applications integrated with GenAI?
How are we managing and governing sandboxed GenAI usage?

IP Protection is about safeguarding proprietary assets like sensitive financials, customer conversations, source code, and product documents. The incident with Samsung employees inadvertently sharing code with ChatGPT, leading to restricted GenAI access, is a case in point. This concern is echoed by many IP-conscious firms, including JPMC.
Cyberheaven’s analysis of 1.6 million users across various companies revealed about 1,000 IP violations in a week, involving internal data, client data, source code, and project files.

 Key IT-Security Questions:

What are our top 3–5 Intellectual Property categories (e.g., Code, Customer Data, Financial Projections)?
How are we safeguarding our source code from accidental GenAI uploads?
What additional security controls are in place for GitHub Co-Pilot usage?

Privacy Protection entails strategies and policies to protect personal information (PII, PCI, PHI) from unauthorized use, access, or disclosure.

Can we address this through existing educational and process controls
Are our current DLP solutions adequate, or will they trigger excessive alerts?
How are we managing DLP risks in User <> GenAI and App <> GenAI interactions?

Prompt Injection represents a sophisticated cybersecurity threat in GenAI, where direct attacks manipulate AI responses through malicious prompts, and indirect attacks subtly bias the output from GenAI.

Do we have external LLM-Integrated Applications in production?
What safeguards are in place against Prompt Injection Attacks?
How are we assessing risks related to Public AI Agents or Plug-Ins?

 
Supply Chain Risks arise from vulnerabilities in the LLM application lifecycle, including third-party datasets, pre-trained models, and plugins.

Do we use any internal or external LLM-Integrated Applications?
What models and datasets have we employed?
Are there any GenAI Plugins we interact with?

Summary
Satya Nadella provides a simple framework for internalizing GenAI for IT-Security teams across enterprises. In today’s world, becoming AI-enabled isn’t just an option; it’s a necessity. The success stories of forward-thinking companies using GenAI are not just inspiring but also illustrative of the potential that awaits those ready to embark on this journey.
When it comes to securing GenAI, it’s crucial to adhere to a structured approach: begin with the foundational elements, progress to basics, and then advance to more complex strategies.
If you’re interested in a deeper discussion or even in contributing to refining this perspective, I’d be delighted to connect. Feel free to send me a direct message on LinkedIn here. Let’s explore the possibilities of Securing GenAI with a Risk Maturity Framework that’s simple to operationalize.

Unlock Zero Trust Security for
GenAI and Data Access
Request a Demo