AI adoption inside enterprises rarely begins with a coordinated platform strategy. It usually starts with individuals and teams selecting tools that solve immediate problems.

A developer subscribes to an AI coding assistant. Another team builds an application using a foundation-model API. Employees begin using standalone cloud applications such as ChatGPT, Claude, Gemini, or Codex. A business unit connects an AI application to internal documents. Eventually, the organization considers an enterprise-wide knowledge assistant or retrieval-augmented generation system.

Each decision may be reasonable on its own. Together, however, they create a fragmented operating environment.

Users authenticate differently across tools. Applications depend on provider API keys. Usage is billed through unrelated subscriptions and cloud accounts. Sensitive data may pass through systems with different retention and training policies. Security teams have limited visibility into what is being used, while finance teams struggle to determine which teams, applications, or business outcomes are responsible for the expenditure.

The enterprise AI problem is therefore not simply about selecting the best model or application. It is about enabling several forms of AI consumption while maintaining consistent control over identity, credentials, data, cost, providers, and operations.

Fig: Different AI consumption paths require one consistent set of enterprise controls

Four Ways Enterprises Consume AI

Enterprise AI adoption can be organized into four broad consumption patterns. These patterns overlap, but their ownership, identity, billing, and data-flow characteristics differ enough to require separate consideration.

1. AI-assisted software development

The first pattern is the use of AI by software-development teams. It includes IDE assistants, command-line tools, coding agents, code-review assistants, and autonomous development applications.

Some products are purchased as per-user subscriptions. Others allow users to connect a model-provider account or consume models through API credentials. More autonomous tools may execute commands, inspect repositories, modify files, or interact with development infrastructure.

The enterprise must determine whether users authenticate through corporate identity, which repositories and environments the tool may access, how source code is handled, how expenditure is attributed, and whether agent actions are logged and auditable.

The risk is not limited to source-code disclosure. Coding agents can access credentials, execute commands, change infrastructure, or introduce dependencies. Their permissions and actions require greater scrutiny than conventional autocomplete.

2. Custom AI applications, agents, and workflows

The second pattern consists of applications built or operated by the enterprise. Examples include customer-service assistants, document-processing workflows, incident-response agents, internal copilots, and automated business processes.

These systems usually consume foundation models through APIs. They may retrieve enterprise data, invoke tools, call internal services, maintain state, and perform actions on behalf of users.

Unlike a standalone application, the enterprise owns much of the operating responsibility. It must manage application identities, API credentials, authorization, model selection, usage limits, monitoring, and failure handling.

Agents make this especially important. A traditional application normally follows a path defined in code. An agent can dynamically choose a model, call tools, retrieve information, and take several steps to complete a task. The architecture must govern both model access and access to data and tools.

3. Standalone hosted AI applications

The third pattern is the direct use of cloud-hosted AI applications by employees. This includes general-purpose assistants and specialized applications for coding, research, writing, design, analytics, and productivity.

The provider operates the application and normally controls the interface, model selection, data handling, and underlying infrastructure. The enterprise consumes the product through user accounts, team subscriptions, or an enterprise agreement.

These products may be introduced through centralized procurement, but they can also enter through individual subscriptions and expense claims. This creates shadow AI: employees using services that the organization has not reviewed or cannot administer.

The enterprise must evaluate SSO, automated provisioning, administrative controls, retention and training terms, audit logs, sharing, and connections to internal systems. A model gateway cannot normally intercept these interactions because the application communicates directly with its provider.

4. Enterprise knowledge search and RAG

The fourth pattern is enterprise knowledge search, often implemented using retrieval-augmented generation, or RAG.

These systems search enterprise data sources and provide relevant content as context for a model response. Sources may include document repositories, collaboration systems, source-code platforms, ticketing systems, knowledge bases, or business applications.

An organization can purchase enterprise search, use knowledge features within a hosted assistant, build a custom RAG platform, or combine managed services with internal components.

The difficult part is rarely retrieving a document and sending it to a model. The harder requirements are preserving source permissions, keeping indexes current, removing content when permissions change, recording citations, preventing unauthorized retrieval, and measuring retrieval and answer quality.

Why Fragmentation Becomes a Governance Problem

The four patterns do not share a common operating model.

Coding applications may be licensed per user. Custom applications consume APIs based on tokens or requests. Hosted applications may use individual, team, or enterprise subscriptions. Knowledge platforms may combine licenses, indexed-content charges, infrastructure costs, and model consumption.

Identity is similarly fragmented. Employees authenticate interactively, while applications and agents use non-human identities. Some products support enterprise federation and automated provisioning; others rely on independently managed accounts. One employee may also connect several applications to internal data using separate authorization grants.

Data may be entered into a hosted application, sent to an API by an internal workload, retrieved from enterprise repositories, recorded in traces, or stored in provider logs.

Managing every product independently leads to predictable problems:

• Former employees retain access to tools that were not centrally provisioned.
• Provider API keys are shared between developers or embedded in applications.
• Teams cannot identify which application caused an increase in expenditure.
• Sensitive information is sent to models that are not approved for that data class.
• Security teams cannot reconstruct which prompt, context, model, or tool produced an action.
• Applications become tightly coupled to provider-specific interfaces.
• Different teams build duplicate integrations and governance mechanisms.

A scalable AI program requires one set of enterprise requirements, even when those requirements are implemented differently across consumption patterns.

Seven Requirements for Enterprise AI Adoption

1. Identity and access governance

Human access should connect to the corporate identity system wherever the product permits it. This includes SSO, multi-factor authentication, role-based access, centralized provisioning, and timely deprovisioning.

Access decisions should reflect the sensitivity of the tool and the data it can reach. A general assistant with no internal connections does not present the same risk as an agent that can access source code, customer records, or production systems.

The organization also needs clear ownership. Someone must approve the product, determine who may use it, review access, and remove access when it is no longer required.

2. Non-human identity and credential governance

Applications and agents should not depend on personal API keys or shared provider credentials.

They require non-human identities with narrowly scoped permissions. Credentials should be issued through an approved process, stored in a secrets-management system, rotated, and revoked without disrupting unrelated applications.

Where possible, applications should receive virtual or intermediary credentials rather than direct provider keys. This allows application-specific limits and provider changes without redistributing credentials to every workload.

3. Cost attribution and financial governance

AI cost is variable and can grow quickly as applications gain users, agents perform more steps, or teams select more capable models.

Enterprises need visibility beyond the provider invoice. Usage should be attributable to a user, application, team, environment, or business unit. Budgets, quotas, alerts, and rate limits should operate at the same levels.

Small organizations may need only visibility and spending alerts. Larger organizations may require forecasting, showback or chargeback, unit-cost measurements, and formal FinOps ownership.

Where possible, expenditure should be connected to outcomes such as completed tasks, resolved cases, generated documents, or developer workflows. Token counts alone do not indicate whether an application is successful.

4. Security and data governance

The enterprise must define what data can be processed by which products, models, providers, and regions.

This requires consistent review of retention, provider-training terms, encryption, residency, subprocessors, auditability, and incident-response commitments. It must also cover information stored outside the primary request, including traces, prompts, retrieved context, cached responses, evaluation datasets, and application memory.

Controls should be proportional to data sensitivity. Public information, internal documents, source code, personal data, and regulated records should not automatically follow the same approval path.

5. Model and vendor flexibility

Model capabilities, prices, performance, and availability change rapidly. An application unnecessarily coupled to one provider becomes difficult to optimize or migrate.

Enterprises should use stable internal interfaces where the benefits justify the effort. Provider-specific features can still be used, but the dependency should be intentional and documented.

Vendor flexibility does not mean every request must switch dynamically between providers. It means the architecture preserves a practical path to compare, route, replace, or add models without redesigning every application.

6. Cost and performance optimization

Not every task requires the most capable or expensive model.

Applications should select models according to task complexity, latency, quality, privacy, and cost. Repeated requests may benefit from caching. High-volume and predictable workloads may eventually justify self-hosted open-weight models.

Optimization should follow measurement. A cheaper model is not cheaper if poor outputs cause repeated calls, human correction, or failed workflows. Cost, latency, and quality must be evaluated together.

7. Monitoring, tracing, and evaluation

Traditional infrastructure metrics are necessary but insufficient for AI systems.

Teams need to understand which model was used, how much was consumed, how long each step took, which context was retrieved, which tools were called, and where failures occurred. Agent traces may span multiple model calls and actions within one task.

Operational monitoring must be complemented by evaluation. An application can remain technically available while quality deteriorates because of a model change, prompt update, retrieval problem, or altered data source.

A Shared Control Point for Model Access

For custom applications, agents, and compatible coding workflows, many requirements can be enforced through a shared model-access layer.

This layer sits between enterprise workloads and model providers. Depending on the organization’s needs, it can authenticate applications, protect provider credentials, enforce model policies, route requests, apply quotas, attribute cost, record audit events, collect traces, and provide a consistent interface across providers.

It is often described as a model gateway or AI gateway.

The gateway is an important architectural control point, but it is not the entire governance solution. Standalone hosted applications generally operate outside it. Knowledge systems require source authorization and retrieval controls. Agents require tool and action governance. Coding tools require safeguards for source code, credentials, local execution, and development infrastructure.

Requirements Change with Enterprise Context

The requirements are broadly consistent, but implementation should reflect scale, risk, and operating maturity.

A small company may prioritize a limited set of approved tools, centralized billing, basic SSO, and managed model access. A mid-size organization may need a dedicated gateway, per-application credentials, team budgets, audit logs, and shared observability. A large enterprise may require multiple providers, regional deployments, formal data classifications, policy enforcement, business-unit isolation, chargeback, advanced evaluation, and self-hosted components.

Employee count alone is not enough. A smaller company handling regulated data may require stronger controls than a larger organization working mainly with public information. AI expenditure, data sensitivity, number of applications, regulation, and platform-team capability are often better indicators of architectural need.

Questions the Reference Architecture Must Answer

These requirements lead to concrete architectural questions:

• Where should model access be centralized?
• How should human users, applications, and agents authenticate?
• How should provider credentials be isolated?
• Which controls belong in the identity provider, gateway, application, or provider?
• How should standalone hosted applications be governed?
• Where should prompts, traces, retrieved context, and evaluation data be stored?
• How should costs be attributed and controlled?
• When is a cloud provider’s model platform sufficient?
• When does an organization need a dedicated gateway?
• When should enterprise knowledge search be purchased, built, or deferred?
• How should the architecture evolve as usage, risk, and maturity increase?

There is no single product that answers every question. The solution is an architecture that combines identity, model access, security, cost management, observability, and application-level controls.

The next article, A Reference Architecture for Governed Enterprise AI, develops that architecture and explains the available implementation choices.