Securing Sensitive Data for AI Agents

Introduction

AI agents are front and center right now and as more companies adopt AI agents for various workflows and tasks, an important question comes up - how do we ensure that AI agents aren't leaking sensitive data to model providers?

The Challenge

The problem is pretty straightforward. AI agents need data to be useful. The more context they have, the better they can help. But that creates an obvious tension - how much data is too much? And how do we ensure that sensitive data isn't accidentally sent to a model provider like OpenAI or Anthropic?

Think about a customer service workflow. An AI agent might need access to order history to help resolve issues. But if that agent accidentally includes credit card numbers or other PII in its prompts to the LLM, you've just leaked sensitive data that you can't get back.

What's at Risk?

There are a few key risks that engineering teams need to think about:

1. Prompt Injection - An AI agent could be tricked into revealing sensitive data through carefully crafted prompts
2. Data Persistence - Even if providers claim they don't store prompts, you have limited visibility into how they actually handle data
3. Access Sprawl - As agents get more capabilities, their data access often expands without proper controls
4. Audit Gaps - Most companies can't track exactly what data their AI agents are accessing and sharing
5. Compliance Issues - Sharing certain types of data with external models may violate HIPAA, GDPR or other regulations

Building Security Controls

Let's talk about how to actually solve this. You need three main components:

1. Data Classification

First, classify your data based on sensitivity:

• Public data that's safe to share
• Internal data that should be restricted
• Sensitive data that requires strict controls
• Regulated data that can't be shared externally

This is really the foundation and helps define the access controls.

2. Access Controls

Implement strict controls on what agents can access. For example, here's what a function in Python might look like to validate that an agent has the appropriate permissions to access data:

def validate_agent_access(agent_id, data_source):
    # Check if agent has permission
    permissions = get_agent_permissions(agent_id)
    if data_source not in permissions['allowed_sources']:
        raise AccessDeniedException()
 
    # Validate access hasn't expired
    if permissions['expiry'] < current_time():
        raise ExpiredAccessException()

3. Sensitive Data Detection and Anonymization

It's important to ensure that users and AI agents aren't accidently leaking any sensitive data to models through their prompts. Here is an example, using the Neosync Python SDK, of how you can detect and anonymization data in free-form text.

import grpc
import json
import os
 
from neosync.mgmt.v1alpha1.anonymization_pb2_grpc import AnonymizationServiceStub
from neosync.mgmt.v1alpha1.anonymization_pb2 import AnonymizeSingleRequest,TransformerMapping
from neosync.mgmt.v1alpha1.transformer_pb2 import TransformerConfig, TransformPiiText
 
def main():
 
    channel_credentials = grpc.ssl_channel_credentials()
 
    channel = grpc.secure_channel(os.getenv('NEOSYNC_API_URL'), channel_credentials)
    stub = AnonymizationServiceStub(channel)
 
    transformer_mapping = TransformerMapping(
        expression=".text",
        transformer=TransformerConfig(
            transform_pii_text_config=TransformPiiText(
                score_threshold=0.5,
            )
        )
    )
 
    data =  {"text": "Dear Mr. John Chang, your physical therapy for your rotator cuff injury is approved for 12 sessions. Your first appointment with therapist Jake is on 8/1/2024 at 11 AM. Please bring a photo ID. We have your SSN on file as 246-80-1357. Is this correct?"}
 
    input_data = json.dumps(data)
 
    access_token = '<neosync-api-token>'
    metadata = [('authorization', f'Bearer {access_token}')] if access_token else None
 
    try:
            # Make the RPC call
            response = stub.AnonymizeSingle(
                AnonymizeSingleRequest(
                    input_data=input_data,
                    transformer_mappings=[transformer_mapping],
                    account_id="<neosync-account-id>"
                ),
                metadata=metadata
            )
 
            # Parse and print response
            try:
                output_data = json.loads(response.output_data)
                print("Anonymized data:", output_data)
            except json.JSONDecodeError:
                print("Raw response:", response.output_data)
    except grpc.RpcError as e:
        if e.code() == grpc.StatusCode.UNAUTHENTICATED:
            print("Authentication failed - please check your access token")
        elif e.code() == grpc.StatusCode.UNAVAILABLE:
            print("Failed to connect to server. Please check the hostname and port.")
        else:
            print(f"RPC failed: {e.code()}, {e.details()}")
    finally:
        channel.close()
if __name__ == "__main__":
    main()

Looking Ahead

As AI agents become more deeply integrated into business processes, these security controls become even more critical. The companies that build these guardrails now will be able to adopt AI agents more confidently and effectively than those that treat security as an afterthought.

I expect we'll see more tools emerge specifically for securing AI agent data access. Remember that unless you're running your own AI agents, you're renting those frm someone else. And if you don't want that other party to see your sensitive data then you need to take the right steps to protect it.

Conclusion

Securing AI agent data access isn't optional - it's a critical requirement for safely deploying these systems in production.

The goal isn't to block AI agent usage entirely, but to enable it with appropriate safeguards. The companies that get this balance right will be much better positioned as AI agents become an even bigger part of how we work.