Tag Archives: chatgpt

Prescribed Enterprise AI Architectures for Small, Mid-Size, and Large Organizations

July 11, 2026AIAI, chatgpt, Claude, llm, technologySreenivas Makam

The first article in this series, Enterprise AI Adoption: Requirements for Security, Governance, and Scale, defined the problem: enterprises consume AI through coding tools, standalone hosted applications, custom agents, and enterprise knowledge systems, but need consistent controls across identity, credentials, data, cost, providers, and operations.

The second article, A Reference Architecture for Governed Enterprise AI, translated those requirements into architectural building blocks and described the available implementation choices.

This article makes the opinionated decisions. It recommends what to implement, what to defer, and which representative products to consider for three enterprise profiles.

Product recommendations reflect the market as of July 2026 and should be revalidated before implementation.

Scope and Assumptions

Company size is the primary organizing variable because adding industry, geography, regulation, data sensitivity, cloud provider, and AI maturity as independent dimensions would make a simple prescription impossible.

The recommendations assume a software-enabled company with moderate data sensitivity, one primary public cloud, an existing corporate identity provider, both employee and custom AI usage, and a preference for managed services unless scale or control justifies ownership.

Size remains a proxy. Move to the next architecture tier when regulation, data sensitivity, AI expenditure, workload volume, geographic isolation, or operational complexity exceeds what is typical for the company profile.

Profile	Working definition	Operating reality
Small	5–50 engineers	No dedicated AI platform team; CTO or engineering lead owns AI
Mid-size	50–500 engineers	Emerging platform team; shared security and FinOps responsibilities
Large	500+ engineers	Dedicated platform, security, data, and FinOps teams; multiple business units

Engineering scale is more useful than total employee count because it better predicts the number of custom workloads and the organization’s ability to operate platform components.

Small Enterprise: Prefer Simplicity

The small-enterprise architecture deliberately accepts coarser attribution and fewer centralized controls in exchange for low operating overhead.

Prescription

Capability	Recommendation
Employee AI	ChatGPT Business
Coding assistant	Codex under the managed company workspace
Human identity	Existing Google Workspace or Microsoft Entra ID
Workload identity	Primary-cloud IAM and secrets services
Model inference	Existing cloud’s managed AI platform
Model gateway	None initially
Model selection	Static aliases in shared application configuration
Routing	No dedicated intelligent router
Guardrails	Provider-native controls plus application validation
Agent framework	Application-selected SDK; no enterprise standard
Agent runtime	Existing application infrastructure
Knowledge	Approved user connectors; no enterprise RAG platform
Observability	Cloud telemetry and application traces
FinOps	Provider budgets, cloud billing, and monthly review
Self-hosted inference	Do not implement

Hosted AI and coding

Use ChatGPT Business as the company-managed employee workspace and Codex as the standard coding assistant. Codex is included in ChatGPT Business, although usage limits and credit options depend on the plan. This reduces unmanaged subscriptions and keeps general AI and coding under one commercial and administrative relationship. OpenAI documents current Codex plan availability here.

Use SSO where available, central billing, approved data terms, and an explicit user-removal process. ChatGPT Business supports SSO but does not provide SCIM directory synchronization, so lifecycle management remains partly administrative. OpenAI documents the Business and Enterprise identity differences here.

Model access without a gateway

Applications call the primary cloud’s managed model platform directly using distinct workload identities. AWS-centric organizations use Amazon Bedrock, Google Cloud organizations use Vertex AI, and Microsoft-centric organizations use Azure AI Foundry. A company without meaningful cloud alignment may use a direct OpenAI or Anthropic API organization.

A shared client library may standardize model aliases, metadata, timeouts, retries, and trace identifiers. It is not a gateway and cannot guarantee centralized enforcement.

Attribution is therefore based on separate cloud projects or accounts, workload identities, billing tags, provider reports, and application logs. This is adequate while the workload portfolio is small, but it may not support reliable per-user or per-feature allocation.

Routing and guardrails

Use static model aliases such as general-fast, general-quality, and sensitive-approved. A centrally maintained configuration maps each alias to an approved cloud model deployment. Cloud IAM should restrict workloads to eligible deployments where supported.

Use provider-native guardrails and application controls. The application remains responsible for user authorization, tool permissions, structured output, agent limits, and approval of consequential actions.

Agents and knowledge

Do not standardize an agent framework yet. The first application can use an appropriate SDK and run within existing application infrastructure. Every agent still needs narrow tool permissions, least-privilege credentials, step and cost limits, and human approval for irreversible actions.

Do not build an enterprise knowledge platform. Employees can use administrator-approved connectors in the hosted workspace, subject to source permissions. Build retrieval only for a product workflow with a measurable requirement.

Graduation trigger

Introduce a managed gateway when the organization has several production AI applications, more than one provider, duplicated integrations, shared credential problems, or a need for centralized budgets, guardrails, routing, and detailed attribution.

Mid-Size Enterprise: Introduce a Shared Control Plane

The mid-size architecture requires all custom production model traffic to pass through a dedicated gateway. The platform remains primarily managed because the organization has an emerging platform team rather than a large AI infrastructure organization.

Prescription

Capability	Default recommendation	Alternative
Employee AI and coding	ChatGPT Enterprise standard seats, including Codex	Codex-only seats for restricted populations
Human identity	Entra ID or Okta with SSO and SCIM	Google Workspace
Workload identity	Cloud IAM plus gateway virtual credentials	Vault where already established
Model gateway	Managed Portkey	Self-hosted LiteLLM
Inference	Primary cloud model platform plus one direct provider	Two approved cloud platforms
Routing	Gateway aliases, conditional rules, and fallback	LiteLLM routing
Guardrails	Provider-native, Portkey, and application controls	Check Point Lakera for higher-risk workloads
Agent framework	OpenAI Agents SDK	LangGraph for durable workflows
Agent runtime	Existing container or serverless platform	Dedicated runtime when volume justifies it
Knowledge	Company knowledge in ChatGPT plus application RAG	Enterprise search when justified
Observability	Portkey plus Langfuse Cloud	LiteLLM plus self-hosted Langfuse
FinOps	Gateway budgets plus cloud billing	Finout or CloudZero
Self-hosted inference	Generally defer	Workload-specific exception

Enterprise workspace

Use standard ChatGPT Enterprise seats for employees and engineers. A standard seat includes ChatGPT and Codex and offers the greatest flexibility. Codex-only seats should be reserved for people who need coding access but should not receive the broader ChatGPT workspace. OpenAI’s current Enterprise documentation describes both seat types and their access boundaries.

Use enforced SSO, SCIM, groups, and role-based access. Separate groups should cover general AI users, Codex users, AI developers, production operators, platform administrators, and sensitive-data users.

Gateway and model portfolio

Use managed Portkey as the default gateway. It provides multi-provider access, virtual credentials, routing, budgets, rate limits, retries, circuit breaking, observability, and integrated guardrails without requiring the emerging platform team to operate the full gateway stack. Portkey documents these gateway capabilities and deployment options.

Use the primary cloud model platform as the main backend and one approved direct provider as a secondary path. All production applications authenticate to the gateway; provider credentials remain with the platform team.

Use logical model aliases, policy restrictions, availability fallbacks, and controlled model-version rollout. Do not add a dedicated intelligent router until expenditure is material and evaluation data can prove that dynamic routing preserves quality.

Guardrails, agents, and tools

Apply provider-native controls, gateway input/output guardrails, and application-specific rules. Portkey guardrails support checks and actions on gateway requests and responses. Add Check Point Lakera only for public-facing or higher-risk workloads that justify specialized prompt-injection and data-leakage protection.

Adopt the OpenAI Agents SDK as the default for straightforward tool-using agents. Allow LangGraph for durable, stateful, resumable, or human-in-the-loop workflows. Run agents on the existing container or serverless platform and provide shared standards for identity, tool registration, MCP servers, traces, execution limits, approvals, and action logs.

Knowledge, observability, and FinOps

Use company knowledge in ChatGPT Enterprise with administrator-approved apps for employee search. Build a shared managed-cloud retrieval foundation only when multiple production applications need common ingestion, permissions, citations, and evaluation.

Use Portkey for gateway-level telemetry and Langfuse Cloud for multi-step traces and evaluations where deeper analysis is required. Use gateway budgets for enforcement and the existing cloud-finance process for reporting. Add Finout or CloudZero when model and supporting cloud costs span providers and require formal allocation or forecasting.

Graduation trigger

Move to the large-enterprise pattern when the company needs regional isolation, multiple business-unit control, formal chargeback, private deployment, dedicated AI security, a shared agent runtime, or self-hosted inference.

Large Enterprise: Central Policy, Federated Delivery

The large-enterprise architecture supports multiple providers, regions, business units, and control teams. Policy and visibility are centralized, while application ownership remains federated.

Prescription

Capability	Default recommendation	Alternative
Employee AI and coding	ChatGPT Enterprise standard seats, including Codex	Codex-only seats for restricted populations
Identity	Entra ID or Okta with SSO, SCIM, groups, and RBAC	Existing enterprise IdP
Workload identity	Cloud workload federation plus gateway virtual keys	Vault where standardized
Model gateway	Regional self-hosted LiteLLM Enterprise	Managed or self-hosted Portkey Enterprise
Inference	Cloud AI platforms plus direct OpenAI and Anthropic	Additional approved providers
Routing	Gateway policy routing plus evaluated intelligent routing	Application-directed routing
Guardrails	Provider controls, gateway policy, and Check Point Lakera	NVIDIA NeMo Guardrails
Agent framework	OpenAI Agents SDK plus LangGraph where justified	Approved framework by exception
Agent runtime	Shared Kubernetes-based runtime	Managed cloud agent runtime
Knowledge	Company knowledge in ChatGPT plus shared product RAG	Glean for dedicated enterprise search
Observability	Self-hosted Langfuse Enterprise plus OpenTelemetry	Managed enterprise platform
FinOps	CloudZero	Finout
Self-hosted inference	vLLM for selected stable workloads	NVIDIA NIM

Regional gateway and inference platform

Deploy LiteLLM Enterprise in approved regions as the standard model-access plane. Each deployment should provide high availability, business-unit isolation, local credentials, virtual keys, budgets, quotas, model aliases, audit records, guardrail integration, and approved fallback policies. LiteLLM documents virtual-key, authentication, SSO, and enterprise gateway capabilities in its product overview.

Applications should not receive direct provider credentials except through documented exceptions. The model portfolio can include strategic cloud AI platforms, direct OpenAI and Anthropic access, selected inference providers, and self-hosted open-weight models for justified workloads.

Expose internal capability aliases such as enterprise-fast, enterprise-reasoning, enterprise-code, regulated-region, and open-weight-private rather than embedding provider model names throughout applications.

Routing and guardrails

Apply policy routing to every request. Region, data classification, business unit, provider approval, and budget determine the eligible model set before performance or cost optimization.

Use intelligent routing only for workloads with evaluation datasets and measurable quality thresholds. RouteLLM or another router may select among eligible models, but the gateway remains the enforcement point.

Use four guardrail layers: provider-native safety, gateway policy, Check Point Lakera for high-risk workloads, and application business controls. Measure false positives, latency, and bypass resistance rather than treating guardrails as absolute security boundaries.

Agent platform

Provide an approved framework portfolio and a shared runtime. Use the OpenAI Agents SDK for the default tool-using pattern and LangGraph for durable workflows. The runtime should provide workload and delegated identity, queues, resumability, isolated code execution, an approved tool and MCP registry, secrets injection, limits, approvals, traces, cancellation, and kill switches.

The gateway governs model inference. The agent runtime governs execution and tools. Deterministic services should verify authorization and execute consequential actions proposed by a model.

Enterprise knowledge

Use company knowledge in ChatGPT Enterprise with approved apps for employee search and synthesis. This is a ChatGPT feature, not a separate product and not a backend for custom applications.

Operate a shared permission-aware RAG platform for product applications. It should provide connectors, ingestion, permission synchronization, deletion, retrieval APIs, citations, evaluation, and regional isolation.

Consider Glean when dedicated cross-enterprise search is a strategic requirement that company knowledge in ChatGPT does not satisfy. Do not deploy both without defining their distinct user populations and responsibilities.

Observability and FinOps

Use self-hosted Langfuse Enterprise for traces, evaluations, prompt management, datasets, RBAC, retention policies, and audit logs. These self-hosted enterprise controls are described in Langfuse’s deployment and pricing documentation. Export platform telemetry through OpenTelemetry and security events to the enterprise SIEM.

Use gateway budgets for real-time enforcement and CloudZero for allocation, anomaly detection, forecasting, unit economics, and showback or chargeback across inference and supporting infrastructure.

Self-host models through vLLM only for workloads with stable demand, demonstrated economics, suitable model quality, and an operations team able to own serving and upgrades.

How the Three Architectures Progress

Capability	Small	Mid-size	Large
Employee workspace	ChatGPT Business	ChatGPT Enterprise	ChatGPT Enterprise with enterprise policy
Coding	Codex	Codex in standard seats	Codex with role and spend controls
Gateway	None	Managed Portkey	Regional LiteLLM Enterprise
Providers	One primary platform	Primary plus one secondary	Governed multi-provider portfolio
Routing	Static configuration	Gateway rules and fallback	Policy plus evaluated intelligent routing
Guardrails	Provider plus application	Provider, gateway, application	Provider, gateway, dedicated security, application
Agents	Application-selected SDK	Standard SDK and shared patterns	Framework portfolio and shared runtime
Knowledge	User connectors	Company knowledge plus application RAG	Company knowledge plus shared product RAG
Observability	Cloud and application logs	Portkey plus Langfuse Cloud	Self-hosted Langfuse Enterprise plus OTel
FinOps	Budgets and monthly review	Gateway budgets and optional FinOps platform	CloudZero with showback or chargeback
Self-hosted inference	No	Usually no	Selected workloads only

When Size Is Not Enough

Use a stronger architecture tier when any of the following applies:

Regulated or highly sensitive data
Mandatory data residency or private networking
High AI expenditure or customer-facing AI cost of revenue
Many autonomous agents with consequential tools
Multiple cloud providers or geographic regions
Formal business-unit isolation or chargeback
Strict availability requirements
A large portfolio of production AI applications

A small healthcare or financial company may need large-enterprise security controls. A large company with limited AI use may begin with the mid-size platform. Size determines the default, not the exception policy.

Final Recommendation

The prescribed progression is intentionally conservative:

Small: one managed employee workspace, Codex, one model platform, no gateway, no enterprise RAG, and no dedicated FinOps platform.
Mid-size: ChatGPT Enterprise and Codex, a managed gateway, two inference paths, shared guardrails and tracing, and application-focused RAG.
Large: regional gateways, a governed provider portfolio, layered AI security, a shared agent runtime, enterprise retrieval, formal observability, and chargeback.

The architecture should become more sophisticated only when usage, risk, cost, or organizational scale creates a concrete reason. The objective is not to deploy every available AI platform component. It is to provide the minimum architecture that enables adoption without losing control.

A Reference Architecture for Governed Enterprise AI

July 11, 2026AIAI, chatgpt, Claude, llm, technologySreenivas Makam

In the first article in this series, Enterprise AI Adoption: Requirements for Security, Governance, and Scale, we identified four ways enterprises consume AI and seven requirements that apply across them. Those requirements cover identity, non-human credentials, cost governance, security and data controls, model flexibility, optimization, and observability.

This article translates those requirements into a reference architecture. It defines the architectural building blocks, their responsibilities, and the implementation choices available to an enterprise. The third article in the series will use these building blocks to prescribe specific architectures for small, mid-size, and large organizations.

One Architecture, Two Control Paths

Enterprise AI cannot be governed through a single product because not all AI traffic follows the same path.

Custom applications, agents, retrieval systems, and compatible coding tools can access models through an enterprise-controlled model gateway. The gateway can authenticate workloads, protect provider credentials, enforce policies, route requests, attribute cost, apply common guardrails, and record telemetry.

Standalone hosted applications such as ChatGPT, Claude, Gemini, and specialized cloud tools normally communicate directly with their providers. Their requests do not pass through the enterprise model gateway. They must instead be governed through corporate identity, provisioning, product administration, contracts, connector controls, SaaS security, data-loss prevention, and audit integrations.

This distinction is fundamental. A model gateway is an important control point, but it is not a universal control plane for every form of enterprise AI consumption.

Fig: Enterprise AI requires separate control paths for hosted applications and enterprise-controlled model traffic.

Architecture Principles

The reference architecture follows five principles.

Centralize policy without centralizing every decision

The enterprise needs common boundaries for identity, credentials, approved models, data, cost, and auditability. Application teams should remain free to select models and implementation patterns within those boundaries. The central platform should provide a governed path rather than become a manual approval queue.

Separate human, workload, and delegated identities

Employees authenticate through the corporate identity provider. Applications and agents use workload identities. When an agent acts for a person, the trace should preserve both the agent identity and the initiating user identity.

Keep provider credentials away from applications

Applications should authenticate to an enterprise-controlled gateway. The gateway protects and rotates provider credentials, maps workload identities to policies, and issues virtual credentials where needed.

Make data policy part of routing

Not every model or provider is approved for every data class. The eligible destination should be constrained by data sensitivity, contractual terms, region, retention policy, and workload risk before cost or performance optimization is considered.

Allow complexity to grow with need

The logical architecture can remain consistent while its implementation grows from cloud-native services to managed gateways, regional control planes, dedicated security tools, and self-hosted inference.

The Technology Landscape

The following table identifies representative implementation choices. These are examples, not recommendations for every organization.

Building block	Implementation choices	Representative examples
Human identity	Enterprise identity provider	Microsoft Entra ID, Okta, Google Workspace
Workload identity and secrets	Cloud IAM, workload federation, secrets platform	AWS IAM, Azure Managed Identities, Google Cloud IAM, HashiCorp Vault
Model gateway	Cloud-native, managed, self-hosted	Amazon Bedrock, Vertex AI, Azure AI Foundry, Portkey, LiteLLM, Kong, Cloudflare AI Gateway
Routing	Gateway rules, intelligent router, application-directed routing	Gateway policies, RouteLLM, Not Diamond, Martian, OpenRouter
Guardrails and AI security	Provider-native, gateway-integrated, dedicated platform, open-source framework	Amazon Bedrock Guardrails, Azure AI Content Safety, Google Model Armor, Portkey, Kong, Cloudflare, Check Point Lakera, NVIDIA NeMo Guardrails
First-party inference	Direct model-provider API	OpenAI, Anthropic, Google, Cohere, Mistral
Cloud inference	Managed cloud model platform	Amazon Bedrock, Google Vertex AI, Azure AI Foundry
Hosted open-model inference	Managed inference provider	Together AI, Fireworks AI, GroqCloud, Cerebras
Self-hosted inference	Enterprise-operated model serving	vLLM, NVIDIA NIM, Hugging Face TGI
Agent framework	SDK, graph framework, managed agent service	OpenAI Agents SDK, LangGraph, cloud-native agent services
AI observability	Gateway telemetry, specialized tracing, open telemetry	Langfuse, LangSmith, Helicone, Arize Phoenix, OpenTelemetry
AI cost governance and FinOps	Gateway budgets, attribution, forecasting, showback and chargeback	LiteLLM, Portkey, CloudZero, Finout, Vantage
Enterprise knowledge	Hosted knowledge feature, enterprise search, managed or custom RAG	Company knowledge in ChatGPT, Glean, cloud search services, vector and search platforms

1. Enterprise Identity and User Lifecycle

The corporate identity provider is the starting point for human access. Hosted AI products, coding applications, gateway consoles, observability platforms, and internal applications should use SSO wherever supported. Automated provisioning should create, update, suspend, and remove accounts as employees change roles or leave.

Groups and roles should represent approved populations such as general AI users, coding users, sensitive-data users, platform administrators, and model approvers. Identity establishes accountability and lifecycle control, but it does not determine what data a user may enter into an AI product. Data policy and product-specific administration remain necessary.

2. Governance for Standalone Hosted AI Applications

Standalone hosted applications sit outside the model-gateway path. The enterprise should maintain an approved product portfolio and evaluate each product for SSO, provisioning, administrative controls, retention, training terms, audit logs, regional processing, sharing, and connectors.

Connectors require particular care. When a hosted application connects to email, files, source code, or collaboration systems, it gains a new authorization path into enterprise information. Administrators must control which connections are available, which OAuth permissions are granted, and how access is revoked.

This is an architectural concern, but not a separate deployable product. It is implemented through identity, SaaS administration, procurement, CASB or DLP controls, connector governance, and organizational policy.

3. Workload Identity and Secrets

Every production workload should have an identity independent of the developer who created it. Production and non-production environments should not share identities or unrestricted credentials.

Where possible, use short-lived cloud workload credentials. When secrets are necessary, keep them in an enterprise secrets platform and rotate them through an established process. Applications should receive gateway credentials or virtual keys instead of underlying provider keys.

This allows a compromised application to be revoked independently, enables per-workload budgets and model permissions, and permits provider credentials to change without application redeployment.

4. Model Gateway

The model gateway is the shared access layer between enterprise-controlled workloads and model providers. It should authenticate the caller and evaluate attributes such as application, team, environment, data classification, use case, cost center, region, and requested capability.

The gateway capability set may include:

Workload authentication and model authorization
Provider credential protection and virtual keys
Standard request and response interfaces
Model aliases and provider abstraction
Usage metering, budgets, quotas, and rate limits
Routing, fallback, retries, and circuit breaking
Audit events and trace propagation
Caching where authorization permits it
Input and output guardrails
Sensitive-data detection and redaction
Model and use-case allowlists
Request-size and context limits
Schema and structured-output validation

The gateway should not contain all business logic. End-user authorization, prompt construction, tool selection, and domain rules normally belong in the application or agent runtime.

5. Model Providers and Inference Platforms

Behind the gateway, the enterprise may use direct model-provider APIs, public-cloud AI platforms, specialized inference providers, or self-hosted open-weight models.

Cloud-native platforms are a practical starting point for companies already standardized on a cloud because they integrate with cloud identity, networking, billing, and regional controls. Direct providers may expose capabilities earlier but create an additional contract, credential, billing path, and data boundary. Self-hosted inference offers deployment control but transfers responsibility for scaling, security, availability, upgrades, and model serving to the enterprise.

The reference architecture permits these options to coexist behind a controlled access path.

6. Policy, Data Controls, and Guardrails

Policy connects enterprise requirements to runtime decisions. It may consider workload, user, environment, data class, model, provider, region, and budget before allowing or routing a request.

Guardrails should be layered:

Layer	Responsibility
Application or agent	User authorization, business rules, tool permissions, transaction limits, output validation
Gateway	Common input/output policies, data detection, model restrictions, schema checks, centralized enforcement
Model platform	Provider-native content and model safety controls
Dedicated AI security	Prompt-injection defense, data-leakage controls, adversarial detection for higher-risk workloads

Guardrail actions may block, redact, warn, log, retry, or reroute. No guardrail product can determine domain-specific correctness or eliminate the need for least-privilege application design.

7. Routing, Resilience, and Portability

Routing is a logical architecture block even when it is implemented inside the gateway.

Three patterns are available:

Gateway-integrated routing: The gateway selects a deployment using aliases, policy, cost, latency, availability, or region.
Dedicated intelligent routing: A router such as RouteLLM selects between eligible models based on predicted quality, complexity, or cost. The gateway remains the enforcement point.
Application-directed routing: The application selects a capability because it has the best task context, while the gateway verifies that the selection is allowed.

Fallbacks must satisfy the same data and compliance policy as the original destination. A technically available model is not automatically an approved fallback.

8. Agent Framework, Runtime, and Tool Governance

An agent framework defines agents, tools, handoffs, state, and execution flow. The runtime operates those workflows, including long-running execution, retries, queues, isolation, and resumability. The gateway controls model access but does not govern every action an agent can take.

The agent layer should provide:

An approved framework or supported framework portfolio
Workload and delegated user identity
Tool and MCP-server registry
Per-agent tool permissions
Step, time, cost, and concurrency limits
Isolated code execution
Action-level audit logs
Human approval for consequential actions
Retry, cancellation, and kill-switch behavior

For high-impact actions, the model should propose the action while deterministic code verifies authorization and performs the operation.

9. Cost Governance and FinOps

The gateway should record application, team, environment, provider, model, usage, and estimated cost. This enables real-time budgets, alerts, quotas, and rate limits.

Enterprise FinOps has a broader responsibility. It must reconcile provider invoices and include supporting costs such as retrieval, databases, queues, agent compute, and self-hosted inference. As maturity increases, the progression is:

Metering → budgets and alerts → attribution → forecasting → optimization → showback or chargeback.

Optimization may include smaller-model routing, caching, prompt reduction, agent-loop limits, batching, or self-hosted inference. Quality and cost must be measured together.

10. Enterprise Knowledge and Retrieval

Employee knowledge access and application RAG are separate architectural needs.

Company knowledge in ChatGPT can search approved connected apps for employee questions and return cited answers while respecting source permissions. It is a feature inside ChatGPT Business and Enterprise, not a separate product or a reusable backend for custom applications.

Product applications require a retrieval architecture with ingestion, change detection, classification, permission synchronization, authorized retrieval, citations, and evaluation. Permission checks should occur before unauthorized content is provided to the model or written to traces.

Organizations may use a dedicated enterprise-search product, managed cloud services, or a custom shared platform depending on scale and requirements.

11. Monitoring, Tracing, Audit, and Evaluation

The architecture produces four kinds of operational evidence:

Infrastructure monitoring for availability, latency, failures, and resource health
AI traces across retrieval, model calls, and tool actions
Security and compliance audit records
Quality evaluations using test datasets, production feedback, automated scoring, and human review

Telemetry requires its own data policy. Prompts, responses, retrieved documents, and tool output may contain sensitive information. The enterprise must define redaction, retention, encryption, regional storage, and operator access.

Cloud-Native Access or a Dedicated Gateway?

A cloud-native model platform may be sufficient when most workloads use one cloud, the model portfolio is limited, cloud identity and billing provide enough control, and cross-provider routing is unnecessary.

A dedicated gateway becomes valuable when applications use multiple providers, credentials must be isolated centrally, budgets and policies must apply across teams, or common routing, guardrails, and audit records are required.

The two approaches are complementary: a dedicated gateway can use cloud-native model platforms as backends.

Managed, Self-Hosted, or Built Internally?

Managed products reduce operating effort. Self-hosted products increase deployment and data control but require upgrades, security maintenance, scaling, and availability ownership. Internal development should be reserved for requirements that available products cannot satisfy.

A model proxy or RAG demonstration may be simple. A production platform with identity, policy, streaming, retries, accounting, permissions, deletion, tracing, and high availability is not.

From Reference Architecture to Prescription

The architecture is intentionally broader than any one organization should deploy immediately. A small company may need only an approved employee assistant, one cloud model platform, and basic budgets. A mid-size organization may need a managed gateway, shared tracing, and application-level RAG. A large enterprise may require regional gateways, multi-provider routing, dedicated guardrails, chargeback, a shared agent runtime, and self-hosted components.

The next article, Prescribed Enterprise AI Architectures for Small, Mid-Size, and Large Organizations, selects a coherent implementation and representative vendors for each profile.

Enterprise AI Adoption: Requirements for Security, Governance, and Scale

July 11, 2026AIAI, chatgpt, Claude, llm, technologySreenivas Makam

AI adoption inside enterprises rarely begins with a coordinated platform strategy. It usually starts with individuals and teams selecting tools that solve immediate problems.

A developer subscribes to an AI coding assistant. Another team builds an application using a foundation-model API. Employees begin using standalone cloud applications such as ChatGPT, Claude, Gemini, or Codex. A business unit connects an AI application to internal documents. Eventually, the organization considers an enterprise-wide knowledge assistant or retrieval-augmented generation system.

Each decision may be reasonable on its own. Together, however, they create a fragmented operating environment.

Users authenticate differently across tools. Applications depend on provider API keys. Usage is billed through unrelated subscriptions and cloud accounts. Sensitive data may pass through systems with different retention and training policies. Security teams have limited visibility into what is being used, while finance teams struggle to determine which teams, applications, or business outcomes are responsible for the expenditure.

The enterprise AI problem is therefore not simply about selecting the best model or application. It is about enabling several forms of AI consumption while maintaining consistent control over identity, credentials, data, cost, providers, and operations.

Fig: Different AI consumption paths require one consistent set of enterprise controls

Four Ways Enterprises Consume AI

Enterprise AI adoption can be organized into four broad consumption patterns. These patterns overlap, but their ownership, identity, billing, and data-flow characteristics differ enough to require separate consideration.

1. AI-assisted software development

The first pattern is the use of AI by software-development teams. It includes IDE assistants, command-line tools, coding agents, code-review assistants, and autonomous development applications.

Some products are purchased as per-user subscriptions. Others allow users to connect a model-provider account or consume models through API credentials. More autonomous tools may execute commands, inspect repositories, modify files, or interact with development infrastructure.

The enterprise must determine whether users authenticate through corporate identity, which repositories and environments the tool may access, how source code is handled, how expenditure is attributed, and whether agent actions are logged and auditable.

The risk is not limited to source-code disclosure. Coding agents can access credentials, execute commands, change infrastructure, or introduce dependencies. Their permissions and actions require greater scrutiny than conventional autocomplete.

2. Custom AI applications, agents, and workflows

The second pattern consists of applications built or operated by the enterprise. Examples include customer-service assistants, document-processing workflows, incident-response agents, internal copilots, and automated business processes.

These systems usually consume foundation models through APIs. They may retrieve enterprise data, invoke tools, call internal services, maintain state, and perform actions on behalf of users.

Unlike a standalone application, the enterprise owns much of the operating responsibility. It must manage application identities, API credentials, authorization, model selection, usage limits, monitoring, and failure handling.

Agents make this especially important. A traditional application normally follows a path defined in code. An agent can dynamically choose a model, call tools, retrieve information, and take several steps to complete a task. The architecture must govern both model access and access to data and tools.

3. Standalone hosted AI applications

The third pattern is the direct use of cloud-hosted AI applications by employees. This includes general-purpose assistants and specialized applications for coding, research, writing, design, analytics, and productivity.

The provider operates the application and normally controls the interface, model selection, data handling, and underlying infrastructure. The enterprise consumes the product through user accounts, team subscriptions, or an enterprise agreement.

These products may be introduced through centralized procurement, but they can also enter through individual subscriptions and expense claims. This creates shadow AI: employees using services that the organization has not reviewed or cannot administer.

The enterprise must evaluate SSO, automated provisioning, administrative controls, retention and training terms, audit logs, sharing, and connections to internal systems. A model gateway cannot normally intercept these interactions because the application communicates directly with its provider.

4. Enterprise knowledge search and RAG

The fourth pattern is enterprise knowledge search, often implemented using retrieval-augmented generation, or RAG.

These systems search enterprise data sources and provide relevant content as context for a model response. Sources may include document repositories, collaboration systems, source-code platforms, ticketing systems, knowledge bases, or business applications.

An organization can purchase enterprise search, use knowledge features within a hosted assistant, build a custom RAG platform, or combine managed services with internal components.

The difficult part is rarely retrieving a document and sending it to a model. The harder requirements are preserving source permissions, keeping indexes current, removing content when permissions change, recording citations, preventing unauthorized retrieval, and measuring retrieval and answer quality.

Why Fragmentation Becomes a Governance Problem

The four patterns do not share a common operating model.

Coding applications may be licensed per user. Custom applications consume APIs based on tokens or requests. Hosted applications may use individual, team, or enterprise subscriptions. Knowledge platforms may combine licenses, indexed-content charges, infrastructure costs, and model consumption.

Identity is similarly fragmented. Employees authenticate interactively, while applications and agents use non-human identities. Some products support enterprise federation and automated provisioning; others rely on independently managed accounts. One employee may also connect several applications to internal data using separate authorization grants.

Data may be entered into a hosted application, sent to an API by an internal workload, retrieved from enterprise repositories, recorded in traces, or stored in provider logs.

Managing every product independently leads to predictable problems:

Former employees retain access to tools that were not centrally provisioned.
Provider API keys are shared between developers or embedded in applications.
Teams cannot identify which application caused an increase in expenditure.
Sensitive information is sent to models that are not approved for that data class.
Security teams cannot reconstruct which prompt, context, model, or tool produced an action.
Applications become tightly coupled to provider-specific interfaces.
Different teams build duplicate integrations and governance mechanisms.

A scalable AI program requires one set of enterprise requirements, even when those requirements are implemented differently across consumption patterns.

Seven Requirements for Enterprise AI Adoption

1. Identity and access governance

Human access should connect to the corporate identity system wherever the product permits it. This includes SSO, multi-factor authentication, role-based access, centralized provisioning, and timely deprovisioning.

Access decisions should reflect the sensitivity of the tool and the data it can reach. A general assistant with no internal connections does not present the same risk as an agent that can access source code, customer records, or production systems.

The organization also needs clear ownership. Someone must approve the product, determine who may use it, review access, and remove access when it is no longer required.

2. Non-human identity and credential governance

Applications and agents should not depend on personal API keys or shared provider credentials.

They require non-human identities with narrowly scoped permissions. Credentials should be issued through an approved process, stored in a secrets-management system, rotated, and revoked without disrupting unrelated applications.

Where possible, applications should receive virtual or intermediary credentials rather than direct provider keys. This allows application-specific limits and provider changes without redistributing credentials to every workload.

3. Cost attribution and financial governance

AI cost is variable and can grow quickly as applications gain users, agents perform more steps, or teams select more capable models.

Enterprises need visibility beyond the provider invoice. Usage should be attributable to a user, application, team, environment, or business unit. Budgets, quotas, alerts, and rate limits should operate at the same levels.

Small organizations may need only visibility and spending alerts. Larger organizations may require forecasting, showback or chargeback, unit-cost measurements, and formal FinOps ownership.

Where possible, expenditure should be connected to outcomes such as completed tasks, resolved cases, generated documents, or developer workflows. Token counts alone do not indicate whether an application is successful.

4. Security and data governance

The enterprise must define what data can be processed by which products, models, providers, and regions.

This requires consistent review of retention, provider-training terms, encryption, residency, subprocessors, auditability, and incident-response commitments. It must also cover information stored outside the primary request, including traces, prompts, retrieved context, cached responses, evaluation datasets, and application memory.

Controls should be proportional to data sensitivity. Public information, internal documents, source code, personal data, and regulated records should not automatically follow the same approval path.

5. Model and vendor flexibility

Model capabilities, prices, performance, and availability change rapidly. An application unnecessarily coupled to one provider becomes difficult to optimize or migrate.

Enterprises should use stable internal interfaces where the benefits justify the effort. Provider-specific features can still be used, but the dependency should be intentional and documented.

Vendor flexibility does not mean every request must switch dynamically between providers. It means the architecture preserves a practical path to compare, route, replace, or add models without redesigning every application.

6. Cost and performance optimization

Not every task requires the most capable or expensive model.

Applications should select models according to task complexity, latency, quality, privacy, and cost. Repeated requests may benefit from caching. High-volume and predictable workloads may eventually justify self-hosted open-weight models.

Optimization should follow measurement. A cheaper model is not cheaper if poor outputs cause repeated calls, human correction, or failed workflows. Cost, latency, and quality must be evaluated together.

7. Monitoring, tracing, and evaluation

Traditional infrastructure metrics are necessary but insufficient for AI systems.

Teams need to understand which model was used, how much was consumed, how long each step took, which context was retrieved, which tools were called, and where failures occurred. Agent traces may span multiple model calls and actions within one task.

Operational monitoring must be complemented by evaluation. An application can remain technically available while quality deteriorates because of a model change, prompt update, retrieval problem, or altered data source.

A Shared Control Point for Model Access

For custom applications, agents, and compatible coding workflows, many requirements can be enforced through a shared model-access layer.

This layer sits between enterprise workloads and model providers. Depending on the organization’s needs, it can authenticate applications, protect provider credentials, enforce model policies, route requests, apply quotas, attribute cost, record audit events, collect traces, and provide a consistent interface across providers.

It is often described as a model gateway or AI gateway.

The gateway is an important architectural control point, but it is not the entire governance solution. Standalone hosted applications generally operate outside it. Knowledge systems require source authorization and retrieval controls. Agents require tool and action governance. Coding tools require safeguards for source code, credentials, local execution, and development infrastructure.

Requirements Change with Enterprise Context

The requirements are broadly consistent, but implementation should reflect scale, risk, and operating maturity.

A small company may prioritize a limited set of approved tools, centralized billing, basic SSO, and managed model access. A mid-size organization may need a dedicated gateway, per-application credentials, team budgets, audit logs, and shared observability. A large enterprise may require multiple providers, regional deployments, formal data classifications, policy enforcement, business-unit isolation, chargeback, advanced evaluation, and self-hosted components.

Employee count alone is not enough. A smaller company handling regulated data may require stronger controls than a larger organization working mainly with public information. AI expenditure, data sensitivity, number of applications, regulation, and platform-team capability are often better indicators of architectural need.

Questions the Reference Architecture Must Answer

These requirements lead to concrete architectural questions:

Where should model access be centralized?
How should human users, applications, and agents authenticate?
How should provider credentials be isolated?
Which controls belong in the identity provider, gateway, application, or provider?
How should standalone hosted applications be governed?
Where should prompts, traces, retrieved context, and evaluation data be stored?
How should costs be attributed and controlled?
When is a cloud provider’s model platform sufficient?
When does an organization need a dedicated gateway?
When should enterprise knowledge search be purchased, built, or deferred?
How should the architecture evolve as usage, risk, and maturity increase?

There is no single product that answers every question. The solution is an architecture that combines identity, model access, security, cost management, observability, and application-level controls.

The next article, A Reference Architecture for Governed Enterprise AI, develops that architecture and explains the available implementation choices.

AI Assistants Have Many Interfaces. Context Is the Real Product.

July 7, 2026AIAI, anthropic, artificial-intelligence, chatgpt, llm, openai, technologySreenivas Makam

AI assistants are no longer just chat windows. The same assistant now appears as a web app, desktop app, mobile app, browser extension, IDE extension, command-line tool, local agent, and cloud worker.

That is powerful, but it creates a new problem: deciding which interface to use, and keeping context alive when moving between them.

This post is based mostly on my experience with OpenAI and Anthropic products: ChatGPT, Codex, Claude, Claude Code, and their web, desktop, IDE, CLI, mobile, browser, and cloud interfaces. I also touch briefly on Gemini and browser-extension-style workflows, because they represent another way people are starting to interact with AI.

The question I am interested in is not just which model is better. It is: which interface should I use, when should I use it, and why does context still get lost when I move between them?

The OpenAI Interfaces

Interface	How I think about it
ChatGPT web	Best for general thinking, writing, research, analysis, and normal assistant workflows.
ChatGPT mobile	Useful when I am away from my laptop. Also useful as a controller for connected Codex hosts.
ChatGPT Voice Mode	Excellent for brainstorming. It feels like a real-time conversation, not just dictation.
Codex desktop app	My default for local agent work. Best when the task needs local files, terminal commands, browser sessions, or writing changes on my Mac.
Codex VS Code extension	Useful for bigger projects inside the IDE, especially when I want to work across multiple agents or keep the workflow inside one editor.
Codex CLI	Powerful for terminal-native workflows, but I do not use it much because I prefer seeing code and diffs visually.
Codex Web / Cloud	Useful when the repo is on GitHub and I want a small bounded change, PR-style task, or cloud execution without relying on my laptop.

The Codex desktop app is the OpenAI interface I use most for local work. For anything that needs access to local files, local folders, terminal commands, browser sessions, or writing changes on my Mac, the desktop app is my default.

It gives me a practical local agent environment where I can see what is happening, approve actions, inspect changes, and let the assistant work inside my machine.

The Codex VS Code extension is useful when I am already inside the IDE, especially for bigger projects where I want a single editor surface and may work across multiple agents or threads.

The Codex CLI is powerful, but I personally do not use it much. I prefer the feel of seeing the work visually while changes are being made.

The Codex Web / Cloud mode is different. It is useful when the work is already in GitHub and I want to make a small change, run a bounded task, or delegate something in a PR-style workflow.

In this mode I do not need a local workspace, and execution does not happen on my laptop. The assistant works in the cloud against the repository.

That has obvious advantages. If my laptop is not available, or if I want something long-running to continue without depending on my machine staying awake, cloud execution makes sense. It also works well when the task is self-contained and the repository can build and test cleanly in a cloud environment.

But cloud is not a replacement for local work in every case. If the task depends on unpushed local files, local credentials, desktop apps, browser sessions, local databases, or my Mac setup, the desktop app is still more convenient.

The Anthropic Interfaces

Interface	How I think about it
Claude web	Good for general Claude chat, writing, thinking, analysis, and projects.
Claude mobile	Useful for mobile access and remote workflows, but not a full replacement for desktop/project context.
Claude desktop app	Useful, but the experience feels split across Chat, Cowork, and Code.
Cowork	Useful for local desktop-style tasks, especially for non-technical users, but I do not fully understand why it needs to be separate from Chat.
Claude Code CLI	My main serious Claude coding workflow, especially inside VS Code.
Claude Code in VS Code	Useful when I want Claude close to the code editor.
Claude Code web/cloud	Good when I want execution to happen in the cloud rather than on my local machine.
Dispatch / Remote Control	Useful ideas, but they do not feel like one unified control layer yet.
Browser extensions / browser usage	Useful, but not the best long-term workflow yet because the integration does not feel smooth enough.

Claude has a similar spread of interfaces, but the experience feels more fragmented to me.

There is Claude web for general chat and projects. There is the Claude mobile app. There is the Claude desktop app, which separates the experience into areas like Chat, Cowork, and Code. There is Claude Code CLI, Claude Code in VS Code, and Claude Code on the web.

For serious coding, my main Claude workflow is Claude Code CLI inside VS Code. That combination feels powerful because I get the capabilities of the CLI while still keeping the editor open and visible.

Claude’s other modes are useful too. Cowork can help with local desktop-style tasks. Claude Code web provides a cloud coding mode when I want the execution to happen away from my machine. Dispatch and Remote Control are useful ideas for sending or steering work from another device.

But the product feels more split. The pieces are good, but I often feel the boundaries between them.

Voice Is Another Interface

One interface I do not want to ignore is voice.

ChatGPT Voice Mode is one of the most useful non-coding interfaces for me. It is especially good for brainstorming. Speaking to the assistant and getting a real-time spoken response feels very different from typing, or even from using a dictation tool.

Tools like Wispr Flow are useful because they let me speak instead of type. But that is still mostly a better input method for a text conversation. It is not the same as a real-time voice conversation.

ChatGPT Voice Mode feels closer to a true conversational interface. It feels less like “generate text, then read the text aloud,” and more like a direct voice interaction.

Claude also has voice capabilities, but in my usage it does not feel as natural as ChatGPT Voice Mode. It feels more like speech-to-text followed by a spoken response. That may not be the exact implementation, but from a user experience standpoint the difference is noticeable. The delay and response style make it less useful for live brainstorming.

I would also like to see this kind of voice experience inside the Codex app. If I am already working in a local agent workspace, being able to brainstorm with Codex by voice would be very useful. I may not want voice for every coding task, but for planning, debugging, architectural discussion, and reviewing tradeoffs, it would be a natural interface.

Mobile As A Controller Interface

Another interface that I find useful is ChatGPT mobile as a controller for Codex.

From the ChatGPT mobile app, I can connect to Codex running on my Mac or Windows machine and access the projects and threads available on that connected host. I can continue work, send follow-up instructions, approve actions, and review results from my phone.

That is a powerful pattern. The phone is not trying to become the full development environment. It is controlling the Codex environment already running on my machine.

As long as that host is awake, online, paired, and signed in, I can continue threads, approve actions, and inspect results. The permissions still belong to the host-side Codex session.

This is different from Codex Web, where the work happens in the cloud against GitHub. Both are useful, but they solve different problems.

Claude has related ideas through Dispatch and Remote Control, but it does not feel the same to me. Dispatch is more like sending work from mobile to desktop. Remote Control is useful for steering a running Claude Code session. But the experience still feels more split between Claude mobile, Claude desktop, Claude Code, Cowork, and Claude Code web.

What I would like is a more unified control layer: mobile, web, desktop, local machines, and cloud environments should feel like different surfaces over the same underlying work context.

Local vs Cloud

The way I think about local and cloud is simple.

Use local when the machine matters.

Use cloud when the shared repository or online workspace is the source of truth.

Local is better when I need my files, my terminal, my browser, my desktop apps, my local setup, or visual feedback. This is why I use the Codex desktop app so much.

Even though it runs with sandboxing and permissions, once I allow the right operations, it can work with my Mac and browser fairly smoothly. Compared with some other local agent experiences, this makes Codex feel more convenient for my daily workflow.

Cloud is better when the task is centered around a shared online source such as GitHub. If the code is pushed, the task is bounded, and the assistant can work in a clean environment, cloud agents are very useful.

They are especially good for small fixes, dependency updates, tests, review follow-ups, PR-style tasks, and background work that should not depend on my laptop staying awake.

The hybrid model is probably the most realistic. I may explore and develop locally, push a branch, then ask a cloud agent to do a bounded follow-up. Or I may use cloud for a small GitHub change while continuing deeper work locally.

The key is discipline: local and cloud workflows work best when the shared source of truth is clean and the assistant is given a clear task.

My Current Workflow

My personal workflow today is roughly this.

For OpenAI, I mostly use the Codex desktop app when the task needs local access or local file changes. It gives me the best balance of visibility, control, and convenience.

For bigger projects inside the editor, I use the Codex VS Code extension, especially when I want to work across multiple agents or keep the whole workflow inside one IDE.

I use Codex Web / Cloud selectively. If something is already on GitHub and I want a small change or a bounded task, it is a good fit. I do not use it as my main development environment, but I see the value clearly.

For general thinking, writing, research, and brainstorming, I use ChatGPT web, mobile, and Voice Mode. Those are still very useful interfaces. But they are separate from the local Codex app context, and that separation matters.

For Claude, I primarily use Claude Code CLI inside VS Code for bigger coding projects. That feels like the strongest Claude coding workflow for me right now.

I also use browser-based tools across Claude, Gemini, and Codex-style workflows, but I see room for improvement there. The browser is important, but the current extension-style experience does not yet feel like the final form.

What OpenAI Gets Right

What I like about Codex is that it feels like a unified local agent workspace.

In the Codex desktop app, I can ask questions about the project, inspect files, make changes, run commands, review diffs, manage threads, and use local browser/computer tools from one place. That reduces the number of decisions I have to make before starting work.

The local desktop app is especially useful because it works with my actual machine. It is sandboxed, and permissions still matter, but once I approve the right operations, it can interact with my Mac and browser smoothly enough for real work.

ChatGPT mobile controlling connected Codex projects is also a strong pattern. It shows what a good cross-device AI interface can look like. The mobile app becomes a control surface over the environment where the work is actually happening.

ChatGPT Voice Mode is another strong interface. For brainstorming, it is one of the best ways to interact with an AI assistant.

What Anthropic Gets Right

Claude Code CLI is very strong. Used inside VS Code, it gives me a powerful workflow while still letting me see the project in the editor. For bigger projects, this works well.

Claude also has powerful separate modes. Chat, Cowork, Code, CLI, web, mobile, Dispatch, Remote Control, and IDE integration all have a reason to exist. The pieces are good.

Claude Code web/cloud is useful when I want execution to happen in the cloud rather than on my machine. Dispatch and Remote Control are also interesting because they recognize that users want to start or steer work from different devices.

What I Would Like To See Improved In Claude

My issue with Claude is not capability. It is product shape.

As a user, I would prefer one universal Claude experience where chat, cowork, and code feel like modes of the same workspace rather than separate places.

I can understand Code being a specialized mode because coding has its own environment, tools, permissions, and workflows. But the separation between Chat and Cowork is less obvious to me.

Claude Cowork seems designed to make agentic desktop work easier for non-technical users. That makes sense. Not everyone wants to use a terminal or think in terms of repositories, branches, commands, and diffs.

But if I am chatting with Claude and the discussion turns into a task, why should I need to move into a different mode? Ideally, Cowork would feel like a capability inside the same Claude workspace rather than a separate place. The assistant should be able to move from discussion to action naturally, while still asking for the right permissions when it needs to touch files, apps, or the computer.

I would also like Claude’s voice experience to feel more natural for live brainstorming. In my usage, ChatGPT Voice Mode feels closer to a real-time conversation, while Claude voice feels more like speech-to-text followed by a spoken response.

Where The Interfaces Still Break Down

The issue is not that there are many interfaces. Different interfaces are useful for different jobs.

Voice is good for brainstorming. Desktop is good for local work. IDE is good for deep project work. Cloud is good for background execution. Mobile is good for steering work.

The problem is that the context does not always travel with me.

A few examples:

ChatGPT voice to Codex

If I brainstorm an idea in ChatGPT Voice Mode on mobile or web, that conversation does not naturally appear inside the Codex desktop app. If the brainstorming leads to an implementation task, I need to manually restate the context in Codex.

Codex desktop to ChatGPT mobile

This works better. If Codex is running on my Mac and the machine is awake, online, paired, and signed in, I can access those Codex projects and threads from ChatGPT mobile. This is one of the best examples of a useful cross-device AI interface.

Codex desktop to ChatGPT web

This is where the continuity feels incomplete. I can control connected Codex hosts from ChatGPT mobile, but I do not get the same connected-host control surface from ChatGPT web. Since I often work from a browser too, I would like the web interface to become another control surface for the same Codex host context.

ChatGPT web or mobile to Codex desktop

The reverse direction is also incomplete. General ChatGPT conversations, projects, and voice brainstorms do not automatically become available as working context inside Codex. That matters because many tasks start as thinking or planning before they become implementation.

Claude Chat, Cowork, and Code

In Claude, the fragmentation feels different. Claude has Chat, Cowork, Code, Claude Code CLI, Claude Code web, mobile, Dispatch, and Remote Control. Dispatch is useful because I can send work from mobile to desktop, and Remote Control is useful for steering a session. But it does not feel like one shared workspace where the same context naturally follows me across Claude web, desktop, mobile, and Code.

Claude Code local vs cloud

Claude Code has both local and cloud-style workflows. Local Claude Code is useful when I want the work to happen inside my own machine or IDE. Claude Code web/cloud is useful when I want the task to run away from my machine, usually against a GitHub-backed environment.

That separation makes sense technically. Local work and cloud work have different permissions, files, tools, and execution environments. But from a user experience standpoint, I still want the context to move more naturally between them. If I plan something in Claude chat, start work in Claude Code CLI, and later move to Claude Code web, I do not want to reconstruct the whole task manually.

The Real Problem Is Context Continuity

The missing piece is not one universal interface. I actually want multiple interfaces.

What I want is a shared context layer underneath them.

If I brainstorm in voice, I should be able to continue in desktop. If I start work in a local agent, I should be able to inspect it from mobile and web. If I delegate work to the cloud, the result should be easy to pull back into the local or conversational context.

If I switch from Claude web to Claude Code, or from ChatGPT to Codex, I should not have to reconstruct the entire task history manually.

The best current example of this working is ChatGPT mobile controlling Codex projects on a connected machine. That shows the direction I want: mobile is not replacing the desktop environment; it is becoming a control surface for it.

The next step is making that idea more universal across web, desktop, mobile, voice, IDE, local agents, and cloud agents.

At the same time, permissions should remain local to the right environment. I do not want every interface to have every permission. Local files, desktop apps, browser sessions, and computer control should stay tied to the machine where permissions were granted. Cloud work should stay in the cloud. Mobile should control what it is allowed to control.

But the reasoning context, project context, and user intent should travel better across these surfaces.

Where I Think This Is Going

The future of AI tools is not just better models. The models will keep improving, but the interface and context layer may matter just as much.

The winning product will be the one that lets me move between local, cloud, IDE, web, browser, desktop, mobile, and voice without constantly re-explaining what I am doing.

For me, that is the real product: not just the assistant, not just the model, and not just another interface.

The real product is context continuity.

My Multi-Agent Coding Setup to Lower LLM Cost

July 1, 2026AI, ProgrammingAI, artificial-intelligence, chatgpt, Claude, codex, llm, opencode, technologySreenivas Makam

AI coding tools are now good enough that the question is no longer “Should I use them?” For me, the more useful question is: how do I use them without letting LLM cost grow unnecessarily?

The obvious answer is to use cheaper models. That helps, but it is not the full answer. In practice, the bigger unlock is designing the coding workflow so that I can switch between tools and models without changing the way I work.

My setup is based on separating three things that often get blurred together:

Editor: where I view and manually edit code.
Harness: the coding agent layer that reads files, edits code, runs commands, applies patches, and manages repo context.
Model: the LLM doing the reasoning underneath the harness.

Once these are separated, I can optimize cost more intelligently. I can use premium models when reasoning quality matters, and use lower-cost models for exploration, boilerplate, summarization, first-pass refactors, or less risky changes.

The Architecture

At a high level, my setup looks like this:

The important part is that the editor, harness, and model are not the same thing.

VS Code is usually my editor. Codex, Claude Code, OpenCode, Copilot, and VS Code extensions are the harness layer or harness-like coding environments. Claude, GPT/Codex models, GLM, Qwen, DeepSeek, and other models are the reasoning engines underneath.

That distinction matters because cost optimization becomes easier when the harness is portable and the model is replaceable.

Shared Project Context

The foundation of the setup is shared project context.

I try to keep repo-specific guidance in files such as:

AGENTS.md
CLAUDE.md
architecture notes
test commands
coding conventions
reusable task prompts

This avoids re-explaining the project every time I switch tools. The goal is simple: if I move from Claude Code to Codex, or from Codex to OpenCode, the new harness should still understand the important repo conventions.

This also saves tokens. Instead of dumping long explanations repeatedly into every chat, I keep durable instructions close to the codebase.

Approach 1: VS Code as Editor, Harnesses Through Extensions

This is the setup I use most often for complex projects.

Here, VS Code is the editor. It is where I read code, navigate files, review diffs, and make manual edits.

The AI harness is usually provided through VS Code extensions or integrations.

Examples:

			
Editor:   VS Code
Harness:  Copilot extension / Codex extension or CLI integration / Claude extension / OpenCode extension
Model:    OpenAI / Claude / Copilot-supported models / OpenRouter-supported models
Context:  AGENTS.md / CLAUDE.md / repo docs / prompts

This setup works well for me because I still want to see the code clearly. I like reviewing changes in the editor, understanding the file structure, and staying close to the diff. Maybe that is old-fashioned, but for larger projects it helps me avoid blindly accepting agent output.

The cost benefit comes from keeping VS Code constant while changing the harness or model depending on the task.

For example:

I may use Copilot for quick inline completions.
I may use Claude through a VS Code extension for complex reasoning-heavy changes.
I may use Codex for agentic repo work.
I may use OpenCode when I want more model flexibility.

The key point is that VS Code is not the whole AI coding system. It is the editor. The harness and the model can still vary underneath.

Approach 2: Harness-First Workflow

The second setup is when I work directly inside the harness.

This applies to tools like:

Codex CLI
Claude Code
OpenCode app

In this workflow, there may not be a separate editor involved at every step. The harness becomes the main interface. It inspects files, proposes changes, applies patches, runs tests, and iterates.

			
Editor:   Optional / secondary
Harness:  Codex CLI / Claude Code / OpenCode app
Model:    Native model for that harness, or configurable provider if supported
Context:  AGENTS.md / CLAUDE.md / repo docs / prompts

I use this less for complex projects because I prefer to look at the code and review changes directly in the editor. But it works well for smaller modifications, scripts, quick utilities, and contained tasks where I can verify the output easily.

For small projects, the harness-first approach can be efficient because the agent can inspect files, make changes, run commands, and iterate without needing much manual navigation.

Cost-wise, this also helps because the harness can do local exploration. It can search files, inspect only what matters, and avoid loading unnecessary context.

Approach 3: Harness Plus OpenRouter for Model Flexibility

The third setup is where the harness/model separation becomes most explicit.

Some harnesses can be configured to talk to OpenRouter or other compatible model providers. That means I can keep the same coding workflow but change the model underneath.

			
Editor:   Optional
Harness:  Codex CLI / Claude Code / OpenCode
Model:    OpenRouter-routed models
Context:  AGENTS.md / CLAUDE.md / repo docs / prompts

For example:

			
Codex CLI harness
        |
        v
OpenRouter provider
        |
        v
z-ai/glm-5.2, Claude, Qwen, DeepSeek, etc.

		

This is useful for experimentation and lower-cost first passes. I do not need every task to use the strongest model. Some tasks are mostly repo search, summarization, simple code generation, or mechanical refactoring. Those are good candidates for cheaper models.

Then, when I need better reasoning, I can move back to a premium model.

That said, I do not personally rely on this approach much for daily coding. It may be stable enough for some workflows, but the interface between a coding harness and a third-party model router can change. Codex, Claude Code, or OpenRouter can update their API behavior, headers, tool-calling assumptions, or model compatibility. If that breaks, the workflow breaks.

So for me, Approach 3 is useful to know and useful for experiments, but it is not my default daily setup.

Using OpenRouter with Codex CLI

Codex can be configured with profiles. That lets the default codex command keep using the normal OpenAI/Codex setup, while a separate command uses OpenRouter.

Create:

~/.codex/openrouter.config.toml

Example config:

			
model_provider = "openrouter"
model = "z-ai/glm-5.2"
model_reasoning_effort = "medium"
[model_providers.openrouter]
name = "OpenRouter"
base_url = "https://openrouter.ai/api/v1"
env_key = "OPENROUTER_API_KEY"
wire_api = "responses"
http_headers = { "HTTP-Referer" = "https://chatgpt.com/codex", "X-Title" = "Codex" }

		

Add your OpenRouter key:

echo 'OPENROUTER_API_KEY=your_key_here' >> ~/.codex/.env

Add a shell alias:

alias codex-openrouter='codex --profile openrouter'

Now:

codex

uses the normal Codex/OpenAI setup, while:

codex-openrouter

starts Codex through OpenRouter using the configured model.

Inside Codex, verify the active configuration with:

/status

To use a different OpenRouter model for one run:

codex --profile openrouter -m anthropic/claude-sonnet-4.6

The important point is that I do not have to replace my main setup. I keep the default Codex workflow intact and add OpenRouter as a separate profile.

Claude Code with OpenRouter

At a high level, the idea for Claude Code is similar: configure the harness to talk to an OpenAI-compatible endpoint or route requests through OpenRouter, where supported.

The exact setup depends on how Claude Code exposes provider configuration at that time. This is one reason I treat this approach as experimental rather than my default daily workflow.

The pattern is:

			
Claude Code harness
        |
        v
OpenRouter or compatible endpoint
        |
        v
Selected model

		

If you use this path, I would treat it as a flexible experiment rather than a guaranteed stable interface. The harness, provider, or model compatibility can change.

How I Think About Model Routing

I do not try to use the cheapest model for everything. That can backfire. A cheap model that creates bad code, misses context, or causes extra cleanup may cost more in time than it saves in tokens.

Instead, I route tasks by risk and complexity.

For example:

Task	Model Strategy
Inline completions	Copilot or editor-native tools
Repo exploration	Lower-cost model or configurable harness
Boilerplate	Lower-cost model
Simple refactor	Start with lower-cost model
Complex debugging	Premium model
Architecture decisions	Premium model
Final review	Strongest available model

The pattern is:

Use cheaper models for reversible work. Use stronger models when mistakes are expensive.

That single rule handles most cases.

My Actual Usage Pattern

My own usage looks roughly like this.

Approach 1 is my default for complex projects.

I use VS Code as the editor and switch between harnesses through extensions or integrations. I prefer this because I can inspect code, review diffs, and stay oriented.

Approach 2 is useful for smaller changes.

I use harness-first tools like Codex CLI, Claude Code, or OpenCode for scripts, contained edits, and smaller projects where the blast radius is low.

Approach 3 is mostly experimental for me.

OpenRouter gives useful model flexibility, but I do not depend on it heavily for daily coding because the connection between the harness and the model provider can change.

Where the Savings Come From

The savings are not just from cheaper tokens.

They come from a few habits:

keeping project context reusable
avoiding repeated explanations
using harnesses that can inspect the repo directly
starting with cheaper models for low-risk work
reserving premium models for hard reasoning
avoiding huge chat histories when a fresh session with repo instructions is enough
choosing the right harness for the task

The most expensive workflow is often the one where I paste too much context into a chat, ask an unclear question, get a partial answer, then spend more tokens correcting it.

A good harness plus good project context reduces that waste.

What about smaller Models That can run locally?

One more workflow I am watching closely is smaller models that can run locally.

I have tried a few Qwen and DeepSeek models through Ollama. My Mac has 48 GB RAM, so it can run some reasonably capable local models. But for the coding tasks I care about, the quality has not yet been strong enough for me to use them as a primary workflow.

That said, I expect this to improve. Smaller models are getting better, and coding harnesses like OpenCode can make this workflow practical if they integrate cleanly with local runtimes such as Ollama.

If this becomes good enough, the stack could look like this:

			
Editor:   VS Code or optional
Harness:  OpenCode
Model:    Smaller coding-capable model running through Ollama
Cost:     No per-token API cost

That would be a meaningful cost-saving path. It would not be completely free in an absolute sense, because there is still hardware cost, electricity, latency, memory pressure, and quality tradeoff. But the marginal API cost per coding task could be zero.

For now, I see this as promising but not yet my default for serious coding work. I expect that to change as smaller coding models improve.

The Tradeoffs

This setup is not free.

There are tradeoffs:

Different harnesses support different features.
Some models are better at tool use than others.
OpenRouter compatibility can vary by model and API behavior.
Cheaper models may need tighter prompts and smaller tasks.
Switching tools too often can hurt flow.

So I do not treat this as a rigid system. It is a practical routing strategy.

If a task is important, ambiguous, or risky, I use a stronger model. If a task is routine, exploratory, or easy to verify, I am comfortable using a cheaper model.

Final Thought

My approach to lowering LLM coding cost is not just “use cheaper models.”

It is:

Make the harness portable, make the project context reusable, and make the model replaceable.

Once those layers are separated, I can choose the right level of spend for each coding task.

VS Code can remain my editor. Codex, Claude Code, OpenCode, or Copilot can act as the harness. OpenAI, Anthropic, OpenRouter, or other providers can supply the model.

Over time, smaller models that can run locally may become another important part of this setup. If a model running through Ollama can provide good enough coding quality through a harness like OpenCode, then some workflows could move from lower-cost API models to zero marginal API cost local inference.

That flexibility is what lowers cost without forcing me to give up the benefits of high-quality AI coding tools.

The Rise of CLI-Based AI Coding Agents: Claude code vs Gemini CLI

August 16, 2025AI, ProgrammingAI, artificial-intelligence, chatgpt, Claude, Gemini, llm, technology, Vibe codingSreenivas Makam

Introduction

I have been a Cursor user for vibe coding for 3 months. I was very skeptical about using Claude Code and Gemini CLI at first, since I wasn’t comfortable with the idea of using a terminal as an AI agent. But in the last 1–2 months, I’ve been trying them both — and it completely changed my opinion.

In this blog, I’ll share my experiences of using them, my favorite pick between the two, and a comparison of the three broad categories of AI-assisted coding approaches that exist today.

The Three Approaches to AI-Assisted Coding

I see broadly 3 kinds of AI-assisted coding approaches:

Chat interface with canvas → ChatGPT, Claude
IDE integrated AI tools → Cursor, Windsurf, Replit, Lovable
CLI-based AI agent tools → Claude Code, Gemini CLI, Warp

🧑‍💻 Category	💬 Chat Interface	🛠️ IDE Integrated Assist	⚡ CLI-based AI Agent
Where it operates	Browser	Standalone IDE or browser (Cursor uses IDE, Lovable uses browser)	Terminal or IDE
Use case	Prototyping, small functions, quick answers, “throwaway weekend projects.”	Augmenting the core coding loop: writing, refactoring, debugging.	Automating workflows: multi-step tasks, system commands, project-wide changes.
Vibe coding style	Pure “vibe coding” (conversational prompting).	Hybrid of “vibe coding” + “developer assist.”	Agentic + autonomous (give AI a goal and let it execute).

For a vibe coder like me, a CLI-based AI agent inside VS Code works perfectly — I get the best of both IDE and terminal with AI agent powers.

My Project: A 2-Way Translator App 🌍

To test these tools, I built a translation application.

When I visited Vietnam a few months back, I noticed cab drivers and restaurants using Google Translate effectively. But one problem stood out: only one device could be used for back-and-forth communication.

So, I decided to build a two-way translation application that solved this problem.

I drafted the following prompt (with ChatGPT’s help):

Global Translator App – MVP Requirements (Web Application)

Build a web app that lets two users communicate in real time via text translation.
Users connect via QR code or unique ID.
Support both text and speech.
Translate automatically for seamless conversation.

(Details moved to the appendix 👇)

Pre-requisites:

Google Translate API with GCP
Firebase backend

Claude Code vs Gemini CLI ⚡

🔎 Feature	🤖 Claude Code	🌐 Gemini CLI
📍 Location of use	Standalone terminal or inside VS Code. In VS Code, Claude Code has IDE context — lets you select code and ask about it.	Standalone terminal only. In VS Code, Gemini CLI has no IDE context (though Google offers Gemini Code Assist for IDE, without terminal capability).
💻 Terminal capability	Excellent — can view files, execute commands, analyze outputs.	Limited — shell commands can’t run in foreground, stateless (no persistent `cd`), no command completion.
⚙️ AI agent capability	Strong coding performance; required multiple iterations but reliable.	Decent, though not as strong as Claude Code.
🧪 Debugging & Testing	Superb. With terminal + MCP integration, I could run unit tests from both terminal and frontend.	Limited debugging/testing due to terminal restrictions and weaker MCP tool support.
🔌 MCP integration	Very good. I integrated Playwright (UI automation) + Firebase.	Okay. Playwright struggled (e.g., no 2-browser instance support). Firebase worked fine.
💸 Cost & model	$20/month plan (Sonnet). Didn’t use Opus ($200/month). Sometimes hit daily quota limits.	Free with generous limits (Gemini 2.5 Pro).

Verdict so far: Claude Code > Gemini CLI for most features, especially debugging and testing.
But Gemini CLI’s pricing (free) and generous usage limits are a big plus.

If Google can merge Gemini CLI with Code Assist and improve Playwright integration, it will become a fantastic package. On the other hand, Claude Code really needs a more flexible pricing tier between $20 and $200.

Project Output

Translation app built with Claude Code → [Demo link here]
Translation app built with Gemini CLI → [Demo link here]

Flow of the app:

User logs in with a username (no auth to keep simple).
Picks language + connects with another user via QR code or username.
Supports both text + voice translation in real time.
Built as a PWA → works on web + mobile.

Debugging & Testing with Claude Code 🔍

This is where Claude Code really shines:

Console errors are debugged + fixed automatically.
AI agent generates unit test cases, executes them, finds failures, and fixes them.
Even frontend integration testing works — thanks to MCP integration:
- It inspects browser console logs.
- Takes screenshots to analyze UI/UX issues (!).

I even asked Claude Code to:

Make a 90-second demo video of the app.
Simulate two users chatting with translations in the app. It worked beautifully.

Demo video created by Claude

Global Translation – User1

Global Translation – User2

Summary ✨

AI-assisted coding has matured tremendously in the last year and is now a top revenue driver among AI apps.

In my first blog on Vibe coding, I complained about limited debugging and testing with the AI coding tools. With these new coding agents, that problem feels largely solved.

Next, I’d love to see AI agents:

Do better system design.
Produce more modular code.
Integrate smoothly with existing codebases.

Between Claude Code and Gemini CLI → Claude Code wins hands down 🏆.
But I’m confident Gemini CLI will close the gap soon.

Appendix

Detailed prompt given for the translation application:

Tech Stack

Frontend Framework: React (or a similar modern JavaScript framework like Vue/Angular, but React aligns with future React Native plans)
Backend: Firebase (Firestore/Realtime Database for real-time chat, Authentication, Cloud Functions for server-side logic if needed)
Translation API: Google Cloud Translation API
QR Code: Open-source JavaScript libraries for QR code generation and scanning (e.g., qrcode.react, html5-qrcode)
Authentication: Anonymous sign-in (extendable to Gmail sign-in later)
Chat History: Local browser storage (e.g., LocalStorage, IndexedDB – no cloud sync for MVP)
Encryption: Not required for MVP
UI/UX: Simple, intuitive, and modern chat interface inspired by leading web messaging apps (e.g., WhatsApp Web, Telegram Web)
Dark Mode: Full support for dark mode from MVP

Core Features (MVP)

User Onboarding
- Anonymous sign-in (no registration required for MVP)
- Generate a unique user ID and QR code for each user upon entering the app
- Users can choose and save a unique username, which is validated against a central Firestore database to prevent conflicts.
Connection Mechanism
- QR Code Scanning: Allow users to scan another user’s QR code using their device’s webcam/camera (if available and permission granted).
- Manual ID Entry: Provide an option to manually enter another user’s unique ID to initiate a chat.
- Display your own QR code for others to scan.
- The application remembers the last 5 friends you’ve connected with, allowing for quick selection from a dropdown menu.
Progressive Web App (PWA):
- The application is designed to be installable on mobile and desktop

devices, offering an app-like experience with potential offline capabilities.

The layout is optimized to adapt and display correctly across various screen sizes, including iOS and Android mobile browsers.

Chat Interface
- Real-time text chat between two users.
- Each user selects their preferred language from a dropdown/selector. This language is the language to be used by the friend on the other side.
- Messages are automatically translated to the recipient’s language using Google Translate API.
- Show both original and translated text in the chat bubble.
- Support for dark mode.
- Friend Online Status (Basic): It displays whether a friend is currently “Online” or “Offline” (with a “Last seen” timestamp). Note: The “offline” status is not automatically updated on browser close in the current setup.
Session Management
- One-to-one chat sessions.
- Simple chat history stored locally in the browser.
Language Support
- Initial support for: Hindi, Telugu, Tamil, Kannada, English, and French.
Misc
- A version number is displayed on the screen, making it easy to identify the deployed application version.

Non-Functional Requirements

Responsive and intuitive UI/UX, adapting well to different screen sizes (desktop, tablet, mobile browsers).
Fast translation and message delivery.
Minimal data usage.
Accessibility support.
Dark mode support.
Cross-browser compatibility (Chrome, Firefox, Safari, Edge).

Future Extensions (Post-MVP)

Native mobile applications (Android & iOS) using React Native.
Gmail sign-in and user profiles.
Speech-to-text and text-to-speech for voice communication.
Discover nearby users (if feasible for the web, e.g., using WebRTC data channels or location APIs).
Group chats.
Persistent chat history with cloud sync.
End-to-end encryption.
Support for additional languages.

🚀 A Guide for B.Tech CS Students to kickstart your AI journey

June 29, 2025AI, ProgrammingAI, artificial-intelligence, chatgpt, coding assistants, education, machine-learning, Programming, technologySreenivas Makam

👋 Introduction

My daughter will be starting her B.Tech in Computer Science at MIT, Manipal this year. As a huge AI proponent, I often share the latest AI trends and tools with my family. When my daughter decided to pursue CS, she asked me several questions about AI, which inspired this blog. I hope this guide helps any student planning to specialize in CS and AI.

📚 Core Fundamentals for CSE Students

Before diving into AI, it’s crucial to master the basics. These are some of the building blocks for everything you’ll do in computer science. Following links will give you an overview of the basics before you deep-dive.

Data Structures & Algorithms: FreeCodeCamp
Operating Systems: Coursera Course, High-Level Overview
Programming Basics: Focus on logic and problem-solving, applicable to any language.
Systems Design Basics: Gaurav Sen’s YouTube Channel

📝 General Advice for Students

In addition to doing your coursework, following tips can help you to be more practically prepared for the industry .

Start with Fundamentals: Focus on math, programming, data structures, and algorithms.
Build a Portfolio: Work on projects, participate in Kaggle competitions and hackathons, and maintain GitHub repositories.
Network: Join AI clubs, attend meetups, and connect with peers and professionals on LinkedIn.
Stay Updated: Follow AI news, research, and trends.
Internships: Real-world experience is invaluable—seek internships early.

🛠️ Tools to Try Out

Following is just a sample collection at this point of time. The tools change so fast so it’s very important to keep yourself updated with the latest.

Chatbots: ChatGPT, Gemini (Try ChatLLM, an aggregator of chatbots and other AI tools collection, its very handy)
Vibe Coding: Cursor, Windsurf, Replit, Pythagora (see my earlier blog for more)
Image Generation: DALL-E(OpenAI), Midjourney
Video Generation: Google Veo
ML Platforms: Google AI Studio(Good to experiment with Google AI models), Kaggle(Kaggle competitions are good, good for datasets and notebooks), Hugging Face(Marketplace for models, datasets and easy to share the ML work with others)
Automation: Zapier (AI orchestration platform connecting different AI and non-AI tools and platforms)

Note: “Vibe coding” refers to using AI-powered coding environments that help you code faster and more intuitively.

🤖 Exploring AI Domains & Career Paths

Here’s a quick overview of different AI roles, what they do, prerequisites, and how to get started. AI industry is still at its nascent stage, these roles can change as the technology matures.

Role	What They Do	Prerequisites	How to Get In
AI Researcher	Develop new AI models/algorithms, advance the field, publish research	Strong math (linear algebra, stats), deep ML/DL, Python, PyTorch/TensorFlow, research skills, academic writing	Advanced courses (Master’s/PhD), join research labs, open-source, publish papers, attend conferences
ML Engineer	Build, optimize, and deploy ML models in production; manage ML systems	Programming (Python, C++/Java), ML frameworks, software engineering, cloud (AWS/GCP/Azure), MLOps basics	End-to-end ML projects, internships, open-source, learn CI/CD, Docker/Kubernetes, model deployment
Data Engineer/Scientist	Build data pipelines, clean/process data, extract insights, visualize findings	Python, SQL, data wrangling, statistics, data viz, ML basics, big data tools (Spark, Hadoop)	Data science/engineering courses, Kaggle, portfolio projects, internships, learn data tools and visualization
AI Application Engineer	Integrate AI models into real-world apps/products; focus on APIs and UX	Programming (Python, JS, etc.), API development, front/back-end, basic ML, UX/UI	Build AI apps, hackathons, internships, learn REST APIs, cloud deployment
AI Security & Safety	Ensure AI systems are secure/safe; address ethical, legal, and risk concerns	Security fundamentals, cryptography, adversarial ML, AI ethics, risk, regulations, ML basics	Cybersecurity/AI ethics courses, CTFs, follow AI safety research, join labs/organizations
AI Product Manager	Define vision/strategy for AI products; bridge tech and business teams	AI/ML concepts, product management, communication, business acumen, user research	Start as engineer/analyst, PM courses, AI projects, internships, develop leadership/communication
AI Hardware Specialist	Design/develop hardware/software (GPUs, TPUs, SDKs) for AI training/inference	ECE/CS, digital design, computer architecture, parallel computing, C/C++, CUDA, ML basics	ECE/CS courses, hardware internships, FPGA/GPU projects, hardware-software co-design, follow NVIDIA/AMD/Intel

🧑‍💻 AI Basics for Students

Following is just a sample to get started with AI basics.

Teachable Machine by Google: Learn about ML models (eg: by training one to distinguish between a cat and a dog)
Tensorflow playground
Intro to TensorFlow Code playground (YouTube)
Basics of Generative AI (Google Cloud Skills Boost)
Introduction to Large Language Models (Google Cloud Skills Boost)
AI for Everyone (Coursera): 5-hour course for all backgrounds.
Machine Learning Introduction (Coursera): Covers supervised/unsupervised learning, popular ML libraries, and neural networks.
Deepdive into LLMs – Andrej Karpathy
How I use LLMs – Andrej Karpathy

🤔 How Should College Students Use AI (and How Not To)?

Don’t: Use AI chatbots to solve class assignments directly—this can kill creativity and hinder learning.
Do: Use AI as a learning tool to explore new ideas, get feedback on completed assignments, and clarify concepts after self-study.
Tip: Treat AI as a personalized teacher—seek help only after you’ve tried solving problems yourself.

🔄 Staying Updated with AI

Curate Resources: Make a repository of your favorite podcasts, blogs, and YouTube channels.
Hands-On Practice: Try new AI tools and work on personal projects.
Mix Coding Styles: Combine “vibe coding” (AI-assisted) with traditional coding to strengthen your skills.

💡 Is AI Going to Take My Job?

A typical software engineer spends only 30–40% of their time coding; the rest involves architecture, design, spec reviews, cross-functional discussions, integration testing, and release processes. While AI can assist with coding, these other activities are equally critical and difficult to automate.

Even within coding, engineers must structure code, manage module interactions, choose technologies, debug, test, scale, and deploy—tasks that require human judgment. AI coding tools can boost productivity by 30–40% today, and possibly up to 70% in the next 1–2 years. However, over-reliance on these tools can erode core skills, and poorly organized AI-generated code can become hard to maintain.

There’s no substitute for strong design and coding fundamentals. Use AI tools as an assistant, not a replacement.

Jevons Paradox: If coding becomes much easier and cheaper, we’ll see more coding projects and more coders, not fewer. The demand for skilled engineers will grow as we automate more of the world.

For the next 5–10 years, CS engineers will remain essential. If AI ever surpasses humans in all aspects (AGI), it won’t just be engineers—every profession will be affected.

🌱 Final Thoughts

CS or CS with AI specialization are fields of endless possibility. Stay curious, keep building, and remember: the journey is as important as the destination. Embrace change, focus on fundamentals, and use AI as a tool to amplify your learning and creativity.

Wishing all new B.Tech CS students an exciting and rewarding journey ahead!

Picture with my lovely daughter!

Are Smart Glasses the Future of AI? My Hands-On Review of Meta AI Glasses

June 1, 2025AIAI, artificial-intelligence, chatgpt, llm, meta, technologySreenivas Makam

Honestly, I never believed smart glasses would become a mainstream AI form factor—until I bought the Meta Ray-Ban Smart Glasses two weeks ago! 😎 This gadget had been on my wishlist for a while, but it wasn’t available in India, and even if you managed to get one from abroad, the app didn’t work well here. Thankfully, Meta launched these glasses in India a month ago, and you can now buy them online or from certified optical dealers. In this blog, I’ll share my hands-on experience from the past two weeks.

Why Glasses? The Hands-Free Advantage 🙌

The first thing I realized: glasses are a fantastic form factor when you want to go hands-free and avoid constantly reaching for your phone or laptop. Google tried this a decade ago, but the tech just wasn’t ready. (More on Google’s new AI glasses later!)

I mostly use the glasses outdoors—while walking, running, or cycling. Indoors, I didn’t find much need for them.

Design & Comfort 🕶️

The design is sleek and modern, not clunky at all. They look like regular sunglasses, so you won’t stand out in a crowd (unless you want to!). However, after a few hours, they do feel a bit heavy, and I sometimes want to take them off for a break.

Use Cases: Where Smart Glasses Shine ✨

Photos & Videos 📸🎥
The 12MP ultra-wide camera delivers good quality photos and up to 3-minute videos. While it’s not quite smartphone-level, the hands-free capture is a game-changer—especially for impromptu moments or when you’re on the move. There’s even blur compensation to keep your shots clear. Selfies are a bit tricky, but you can always take them by holding the glasses like a phone.

Music, Podcasts & Calls 🎶📞
With 5 microphones and 2 speakers, the audio quality is impressive. The directed audio keeps you aware of your surroundings—crucial for outdoor activities. Personally, listening to music made my uphill cycling sessions much more enjoyable! 🚴‍♂️

The AI Edge: Meta AI in Your Glasses 🤖

The real magic is in the AI. Meta AI uses the latest Llama models, giving you robust speech-to-text and general chatbot capabilities. While Llama isn’t quite at OpenAI’s level, it works well for most queries. The best part? Multimodal capability! You can ask questions about what you’re seeing. For example, I spotted a tree with unique flowers, asked the glasses to identify it, and got an accurate answer. This feature will be super useful when traveling or reading foreign text.

Live Speech Translation 🌍🗣️

Currently, live translation supports French, Spanish, and Italian. It works best if both people have Meta glasses (for two-way translation), but even one-way translation is handy. I tested it with my daughter’s French and while watching a French video—worked well as long as the audio wasn’t too fast.

Cons & Limitations ⚠️

The glasses are a bit heavy and feel bulky after extended use.
Occasionally, they freeze and need a restart.
Battery life is about 3–4 hours—okay for most outings, but longer would be better.

Pro Tips for Buyers 📝

If you need prescription lenses, get the AI glasses fitted accordingly (external vendors can help).
If you don’t need a prescription, consider transition lenses for both indoor and outdoor use. I use reading glasses, so transition lenses are perfect for me.

Some pictures and videos that I took 📸🎥

Cycling clip

Final Thoughts & Google Glasses Comparison 🥽

After seeing Google’s latest demo at I/O, I’m excited for their upcoming glasses, especially with XR and virtual screen features. That could be a game-changer, but it’s likely a year away and pricing is still unknown.

For now, I absolutely love my Meta AI glasses. Priced between ₹29,000–₹35,000, they’re a solid investment for the features you get. I’m convinced glasses will be a major new form factor for AI—though not the only one.

Would I recommend them? Absolutely, if you love trying new tech and want a taste of the future—hands-free! 🚀

🤖 AI Customer Support using an Agentic Framework

May 21, 2025AIAI, chatgpt, langchain, langsmith, llmSreenivas Makam

In this blog, I’ll walk you through the design, development, and lessons learned while building a multi-agent AI customer support assistant using the LangChain framework and related AI tools. 🎮💬

🎯 Motivation: Why Build This?

At KGeN, a game aggregation platform connecting publishers and gamers, our primary users are gamers and clan chiefs (micro-community leaders).

These users often ask questions about:

Platform features
Game-specific achievements
Player and clan statistics

Some answers come from a static knowledge base, while others depend on dynamic user-specific data.

⚡ We wanted an intelligent, scalable AI assistant that could:

Understand natural language queries
Route them to the appropriate data sources
Continuously improve through feedback

Based on the poc feedback, I wanted to take this to production.

🧠 The use case generalizes to any industry with static documentation and dynamic user data—only the context changes.

🔗 Application & Code

The poc application is deployed in Render, you can try this out. The Github also contains instructions to run it locally or in cloud.

🌐 Live Demo (Needs pre-heating, if the query does not work, execute this curl:
curl -s https://support-assistant.onrender.com/ > /dev/null)
💻 GitHub Repository

📌 Business Goals

I wanted a system with following business goals:

🗣 Answer queries conversationally
⚙️ Route questions to the right agent (static/dynamic/hybrid)
🧾 Escalate unresolved issues via Jira tickets
⭐ Collect feedback for iterative improvements
📚 Learn from feedback to enhance performance

🧪 Prototyping Approach

I followed a “vibe coding” model:

Start fast with a working prototype
Use AI to assist (ChatGPT + Cursor editor)
Iterate with real feedback

💡 Tools Used:

ChatGPT to generate mock data (static + SQL)
Langchain/Langsmith as agentic framework
Cursor for AI-assisted coding
Render for cloud deployment

⚠️ Tip: Feed detailed requirements to AI code editors. Without clarity, they produce unreliable or messy code.

🧭 Agent Flow: How It Works

Each user query is first routed by a Main AI Agent, which classifies the query as:

📘 Static: Uses vector search on documentation (FAISS)
🗄 Dynamic: Converts to SQL query on structured data
🔁 Hybrid: Mixes both static + dynamic sources
📥 Follow-Up: Needs more user input
🚨 Escalation: Routed to a human via Jira

Each type has a specialized agent with its own system prompt.

🧠 LangChain powers the routing, agents, and execution logic.

📌 Architecture Diagram: Agent Flow

🧱 Architecture & Tech Stack

Component	Tool / Framework	Reasoning
Agent Framework	LangChain	Modular, battle-tested
Monitoring	LangSmith	Easy trace/debug for agents
Vector DB	FAISS	Simple to set up for POC
LLMs	OpenAI (pluggable)	Can switch to others like Claude
Backend API	FastAPI	Lightweight, async-friendly
Frontend (POC)	Streamlit	Quick prototyping
Deployment	Render	Easy cloud deployment
Ticketing	Jira API	For support escalations
DB (Local/Test)	SQLite	Lightweight
DB (Production)	Postgres	Scalable

🐞 Issues & Learnings

This whole application took me around 8-10 hours over a period of 2 weeks. I got time to spend only on weekends to do this.. Following are some issues I faced:

🧩 Dependency Hell: LangChain and LLM libs change fast. Cursor couldn’t resolve pip issues well. I had to request cursor to get latest details in internet to resolve it.
🧪 Streamlit Cloud Problems: Ended up moving to Render for better compatibility.
🌍 Env File Confusion: Environment-specific bugs were hard to debug in prod as Cursor does not integrate with Render deployment.

🔍 Debugging with LangSmith

Langsmith is great to understand if the agentic workflow is working as expected. I was able to fix the following issues with Langsmith:

🔎 Identified issue that search from vector database is giving the whole static knowledge base instead of giving the specific context. Adding semantic analysis to vector database match helped solve this.
🧩 Fix hybrid agent’s output merging logic
🔁 Debug why hybrid/support queries didn’t escalate to Jira

📂 Sample Queries along with Langsmith trace

Static query example: What are legendary items?

From the above trace, we can see that there are 2 LLM chain calls and 1 call to vector database. The first chain call is to identity the type of query and the second chain call is to summarize the response from vector database.

Hybrid query example: How many gold achievements has DragonSlayer99 earned and what rewards do they give?

From the above trace, we can see that there are 6 chain calls in the above query:

first to identity type of query
second to summarize results of vector database
third to check if there is username in the query
fourth to generate sql query and get results from postgres db
fifth to take the results from sql query and generate summarized response
sixth to combine the summarized static data and dynamic data to give response to the user

📋 Requirements Summary (Generated via ChatGPT)

I fed these requirements as initial prompt into cursor after few iterations of discussions with chatgpt.

💳 Business Requirements

Build an AI system that can:
- 📘 Answer static queries from documentation
- 🗄 Query live backend data
- 🔁 Combine static + dynamic sources
- 📥 Handle follow-up interactions
- 🚨 Escalate to Jira when needed
Serve two user roles:
- 👤 Gamers (general players)
- 👑 Clan Chiefs (advanced users)
Goals:
- Reduce manual tickets by 70%
- Improve first-response time
- Maintain conversational accuracy

📈 Functional Requirements

Static Question Answering
- Vectorize and index knowledge base with FAISS
- Use RAG (Retrieval-Augmented Generation) to answer
Dynamic Question Answering
- Use LangChain SQL Agent to convert natural language to SQL
- Query SQLite (for testing) and Postgres (in prod)
Hybrid Handling
- Mix RAG results with SQL data for composite answers
Follow-Up Logic
- Prompt for missing data (e.g., usernames)
Escalation
- Auto-create Jira ticket with conversation context if unresolved
Multi-Agent System
- Router → specialized agents (static, dynamic, hybrid, etc.)
API & UI
- FastAPI for backend
- Streamlit for POC frontend; React for future UI

🚀 Technical Requirements

LangChain (Python)
FAISS or ChromaDB for vector storage
OpenAI or Claude LLMs
SQLite (via LangChain SQL agent) as simulated backend
JIRA API for ticket creation
FastAPI (backend API layer)
Streamlit (prototyping UI)
React (future UI)

🔐 Security & Testing

Role-based access (gamers vs. clan chiefs)
Environment variable protection
Unit tests, evaluation prompts, and simulated load

✅ Success Criteria

90%+ accurate responses in test cases ✅
Sub-3-second latency ✅
Smooth Jira escalation pipeline ✅
API ready for frontend integrations ✅

🚀 Final Thoughts

This project demonstrates how AI agents, vector databases, LLMs, and good system design can solve real-world support problems.

There are many improvements needed to take this into production. Following are some of them:

Use Langchain conversation memory. This is maintained at streamlit level to stitch a conversation.
RBAC based on user login and queries based on the user.
Performance improvement using caching at different levels, database connection optimisation.
Agent self learnings from user feedback
Improve UI/UX

🔄 This customer support agent is a template that can be adapted across industries—from gaming to banking to e-commerce.