Private AI Systems, SSO, RBAC/ABAC & Data Residency Controls

Some data must never leave your control. Private AI runs models inside your own infrastructure so sensitive information stays within your perimeter. Access control ensures that even inside a private system, only the right people — or the right agents — can reach the right data. This chapter covers both: the architecture of private AI deployment and the security layer that makes it trustworthy.

Table of Contents

• What Is a Private AI System
• Public vs Private vs Hybrid
• Hosting Options
• Private Model Deployment
• Retrieval and Data Handling
• Model Size and Cost Trade-offs
• A Minimal Private Stack
• Why Access Control Is Different in AI
• Identity: Authentication and SSO
• Authorization: RBAC and ABAC
• Filtering the Context, Not Just the Prompt
• Row-level and Document-level Security
• Audit Logging and Accountability
• Access Control for RAG — the Full Loop
• Best Practices

What Is a Private AI System

A private AI system runs inference, prompt handling, and data plumbing entirely inside infrastructure you control — an on-prem cluster, your own cloud VPC, or an air-gapped network.

Public AI:   your data → provider's API → provider's servers
Private AI:  your data → your servers → your model → your output

The point is data residency: regulated, proprietary, or confidential data never transits a third party.

Public vs Private vs Hybrid

Start public for non-sensitive tasks; move to private when data ownership or compliance demands it.

Hosting Options

• On-premise — full control, but you run the GPUs and cooling.
• Private cloud VPC — cloud provider, your isolated network, no shared-tenancy risk.
• Managed private endpoints — a provider's dedicated model instance behind your VPC.
• Air-gapped — no network at all; for the most restricted environments.

private_system:
  location: "eu-central-1 vpc"
  network:  "no public egress allowed"
  model_host: "dedicated-gpu-instance"

Private Model Deployment

Deploying a model privately means serving it behind your own API.

# Conceptual: serve a local model
from transformers import pipeline

classifier = pipeline("text-classification", model="./private_model")
result = classifier("Approval request for vendor X")
print(result)   # runs entirely inside your host

Your own endpoint wraps the model; clients inside your network call it, and no request leaves the perimeter.

Retrieval and Data Handling

Private means retrieval too: your embeddings, vector store, and documents all stay inside the perimeter.

def query_local(question):
    context = vector_store_similarity_search(  # internal store
        query=embed(question, model=LOCAL_EMBEDDER),
        top_k=5,
    )
    return local_llm.complete(system, context, question)

Every step — embedding, index, generation — uses private infrastructure. Nothing falls out.

Model Size and Cost Trade-offs

Private models are usually smaller, and you pay for the GPU whether or not it's busy.

Full-size cloud model:   best quality, hosted, measured per token
Open-weight private:     good quality, predictable cost, you keep data
Smaller quantized model: cheap and private, but quality drops

Pick the smallest model that meets your accuracy bar; spend GPU budget on what matters.

A Minimal Private Stack

A workable private setup needs a few pieces:

private:
  inferencer:  vLLM or your favorite serving engine + open-weight model
  embedder:    local embedding model
  vector_db:   self-hosted vector database
  ingress:     only your VPN / fleet, no public route
  guardrails:  prompt/response filters, logging

Why Access Control Is Different in AI

Traditional APIs expose defined data. An AI system is open-ended: the user's question determines what gets retrieved, and you cannot predict every question in advance.

Classic app:  "GET /order/123" → you control the endpoint.
AI system:    "Summarize this customer order" → the model decides
              what to pull. You must gate retrieval.

The rule of thumb: filter at retrieval and at data access, not just the final answer.

Identity: Authentication and SSO

First you must know who is asking. Tie AI to your corporate identity provider.

• Use SSO (OAuth2 / OIDC / SAML) so users authenticate with the same account they use everywhere.
• Never issue the AI its own open credential that anyone can call.
• Enforce MFA on access to AI tools that see sensitive data.

user = authenticate(session_token, oidc_issuer=corp_idp)
assert user.roles, "must be an authenticated employee"

Authorization: RBAC and ABAC

Decide what each identity may do.

• RBAC (role-based): grant by job role — "Finance" vs "Sales."
• ABAC (attribute-based): grant by attributes — "region, business unit, clearance level."

Recommended: default to deny; allow explicitly.

policy:
  sales_rep:
    can_read: ["account_records_in_my_region"]
    regions: ["emea"]
  auditor:
    can_read: ["all", "audit_logs"]
    deny: ["delete"]
  intern:
    can_read: []   # nothing until explicitly granted

Filtering the Context, Not Just the Prompt

Prompting "only answer for your own account" is weak. Enforce access in the retrieval layer so the model never sees unauthorized text.

def authorized_context(question, user):
    permitted = access_filter(user)            # doc ids the user may see
    hits = similarity_search(question, top_k=10)
    return [h for h in hits if h.document_id in permitted]

Even a cleverly-phrased question cannot leak data the retrieval layer never returns.

Row-level and Document-level Security

Apply access rules at fine granularity, not the whole collection.

• Row-level: the database — a user only queries rows they own.
• Document-level: document in the vector store carries access metadata.
• Embed the access policy as metadata of each chunk so search can join it to the user.

chunk.metadata = {
    "document_id": "cur#3412",
    "allowed_roles": ["finance", "audit"],
    "allowed_regions": ["emea"],
}

Audit Logging and Accountability

"Who asked, what did they see, what was produced" must be logged.

def log_interaction(user, question, context_ids, answer, cost):
    audit_append(
        user_id=user.id,
        ts=utcnow(),
        retrieved=context_ids,
        answer_preview=truncate(answer, 200),
    )

Keep logs immutable and readable for compliance.

Access Control for RAG — the Full Loop

User → AuthN (who is this?) → AuthZ (may they use this tool?)
     → Retrieve (filtered by their permissions)
     → Generate → Log → Return

Best Practices

• Authenticate against the corporate identity, never a shared key.
• Filter at retrieval and storage, never trust the prompt.
• Start deny-by-default and open exceptions purposefully.
• Log every interaction; keep logs longer than you think you need.
• Keep a human review loop and plan a clean path to retire the system.