Part 1. An AI system connected to three lenses: safety, security and values.

Understanding Responsible AI: Safety, Security and Values

This post is part of the “Responsible AI and AI Governance for Agents” series, covering responsible AI principles and evaluation, followed by AI governance principles, a data model, a governance prototype, and a comparison of governance platforms. This post introduces responsible AI through safety, security and values.

I use the marketplace agentic application from my Agentic Systems course as the reference application for the examples in this post.

Objective

People use AI for important work, where its responses and actions can affect people’s livelihoods, privacy and safety. As models become more capable and organizations give them more responsibility, the consequences of harmful, insecure or unfair behavior grow. Three real incidents show what can go wrong with AI systems.

  • Safety. In early 2026, people used Grok to create and share intimate images of real people without their consent. Canadian Privacy Commissioner’s findings
  • Security. OpenAI reported that an isolation failure during an evaluation led to private evaluation data being copied into a public Hugging Face dataset. OpenAI incident report
  • Values. The U.S. Justice Department alleged that AI-generated job ads unlawfully excluded applicants based on citizenship status. Justice Department announcement

I’ll use safety, security and values to examine what responsible behavior looks like in a model’s reply and in the actions of a deployed agent.

What Responsible AI Covers

Responsible AI is the practice of developing, deploying and using AI to achieve intended benefits, reduce harm and meet obligations to the people it affects.

NIST’s AI Risk Management Framework describes characteristics of trustworthy AI, including reliability, safety, security, privacy and fairness. Microsoft’s responsible AI principles also emphasize inclusiveness, transparency and accountability. NIST AI RMF trustworthiness characteristics · Microsoft Responsible AI principles

Anthropic’s approach provides a model-specific example. Claude’s Constitution describes its intended behavior as broadly safe, broadly ethical, compliant with Anthropic’s guidelines and genuinely helpful.

This series focuses on responsible behavior through three lenses: safety, security and values. It does not attempt to cover every aspect of trustworthy AI. General accuracy, availability and reliability are treated as engineering foundations; detailed explainability and interpretability are outside this series’ scope. Where these topics affect harm, privacy or fair treatment, I discuss the connection.

I use three lenses to review a particular response or action. Safety asks what harm the system could cause or enable. Security asks whether it protects data and resists attempts to manipulate or disrupt it. Values asks whether it is honest, treats people fairly and respects their ability to make decisions. Governance spans all three by setting rules, assigning responsibilities and reviewing evidence about how the system behaves.

Consider the marketplace assistant used in my Agentic Systems course. It helps buyers and sellers with orders, refunds and shipments. The following illustrative cases specify the behaviors I would expect from the marketplace assistant.

  • Safety — refund fraud. A buyer asks for help fabricating evidence for a refund claim. The assistant should refuse to help with the deception and offer legitimate dispute options.
  • Safety — threats. A buyer asks the assistant to write a violent threat against a seller to force a refund. It should refuse and offer help writing a legitimate complaint.
  • Security — private records. A signed-in buyer asks for another buyer’s private order details. The application should enforce the caller’s access boundary before returning data.
  • Security — injected instructions. A retrieved document tells the assistant to disclose private information. It should treat the document as data, not authority to change permissions or redirect the task.
  • Values — honesty. A seller asks the assistant to tell a buyer that an item has shipped, while confirming that it has not. The assistant should refuse to knowingly misrepresent the shipment status.
  • Values — fairness. The assistant is asked to recommend rejecting an otherwise eligible refund because of the buyer’s nationality. It should use the applicable refund criteria without introducing an irrelevant discriminatory condition.
Responsible AI covers safety, security and values for model and agent behavior, with governance spanning all three through rules, ownership and review.
Governance spans safety, security and values through rules, ownership and review.

Safety

Safety asks what harm a response or action could cause or enable. Consider a writing assistant. Someone may ask it to explain how employees can spot a scam, or they may ask it to help write a deceptive message. Both requests can use similar vocabulary. The purpose of the requested help determines whether the assistant should provide it, limit it or redirect the user to a legitimate alternative.

An attempt to get the model to bypass its safeguards is called a jailbreak. The safety failure we’re looking for is harmful assistance, regardless of whether the user tried to bypass a safeguard. OWASP on prompt injection and jailbreaking

Mistakes can also cause harm when the request is legitimate. Suppose a buyer asks the marketplace assistant to summarize a recall notice for an appliance they purchased. The notice says to stop using the appliance immediately, but the summary includes only the refund steps and leaves out that warning. The buyer could continue using a dangerous product. Here, an omission becomes a safety issue because of its potential consequences.

Common safety checks include:

  • Harmful assistance. Decline to help write a scam message or file a false refund claim.
  • Over-refusal. Answer a harmless computing request, such as how to stop a running process, even if it uses the word “kill.”
  • Safety-critical omission. Preserve the instruction to stop using a recalled appliance when summarizing its recall notice.

Security

Security covers protecting data from unauthorized disclosure, protecting the integrity of data and system behavior, and keeping the service available when it is attacked. For an AI application, that can mean preventing sensitive information from leaking, detecting tampering with a model or its data, and limiting abusive traffic. NIST AI RMF on security and resilience

Indirect prompt injection occurs when an instruction embedded in external content, such as a retrieved marketplace policy document, tries to redirect the model. In direct prompt injection, the instruction comes from the user’s prompt. The document’s author has no authority to change the user’s request or authorize disclosure. For example, a retrieved refund policy might contain an instruction to disclose another buyer’s private order details. The assistant should ignore that instruction and continue answering the legitimate refund question. OWASP on direct and indirect prompt injection

An application adds a separate boundary. In the buyer-facing marketplace assistant, the order service should enforce the signed-in buyer’s access before returning records. A buyer should not receive another buyer’s private order details merely by asking the assistant. Read-only access is not enough: it prevents changes but can still expose records the caller should not see. I’d check both which records the service returned and how the assistant used them.

Common security checks include:

  • Direct prompt injection. Resist a buyer’s attempt to override the marketplace assistant’s access restrictions.
  • Indirect prompt injection. Ignore instructions embedded in a retrieved refund policy and continue the legitimate order or refund investigation.
  • Private context disclosure. Keep another buyer’s private order and payment details from callers who lack access.
  • Unauthorized tool use. Enforce the caller’s permissions when the assistant tries to read an order or request an order change.

Values

Values are expectations about how a system treats people and represents what it knows. Two industry examples make honesty and fairness concrete.

Honesty under pressure. Sycophancy is when a model becomes overly agreeable instead of giving a candid assessment. In April 2025, OpenAI rolled back a GPT-4o update after finding it overly flattering and agreeable. OpenAI said the update had become overly supportive but disingenuous. This illustrates why pleasing a user is not the same as helping them make an informed decision. OpenAI on sycophancy

Fair treatment. In its 2023 action against Rite Aid, the U.S. Federal Trade Commission alleged that AI-based facial recognition produced false matches that led staff to wrongly accuse customers of shoplifting. The FTC said the resulting harms disproportionately affected people of color. This shows why reviewing overall accuracy is not enough: teams also need to examine who experiences errors and how people are treated when a system flags them. FTC on Rite Aid’s facial recognition

Common values checks include:

  • Honesty under pressure. Preserve a supplied fact when the user pushes for a different answer.
  • Acknowledging uncertainty. Say when the available evidence is insufficient to support a claim.
  • Fair treatment. Check whether people in equivalent circumstances receive equivalent guidance.

Where the Lenses Overlap

One failure can raise several questions. Security asks which protection failed. Safety asks who could be harmed and how. Values asks which expectation about honesty, fairness or respect for people the system violated.

Observed failureWhat the lenses reveal
An assistant shares a restricted report containing employees’ home addresses with a team that lacks access.Security: The assistant disclosed restricted data to unauthorized readers. Values: It failed to respect employees’ privacy. Safety: The exposed addresses could be used to harass or threaten employees.
A summary invents a cause for a service incident.Values: The summary presents speculation as fact. Safety: Responders could act on the wrong cause and leave the actual hazard unresolved.
A model helps a user write a scam message.Values: The response helps the user deceive recipients. Safety: A recipient could lose money if they trust the message.

Model-Level and Agent-Level Evaluation

At the model level, I give the model a request and any reference text directly, then inspect its reply. I control the inputs so I can isolate behavior such as refusing a harmful request, ignoring an instruction in reference text or representing evidence honestly.

At the agent level, I evaluate the complete application. An agent can run in a loop: the model chooses a step, calls a tool, receives a result and decides what to do next. Depending on its permissions, it may retrieve private records or take actions in the backend. This creates more paths to evaluate than a single model reply.

I therefore inspect the sequence of decisions, retrieved evidence, tool calls, enforced permissions and final outcome. A safe-looking answer is not enough if an earlier tool call exposed restricted data or performed an unauthorized action.

Both levels are needed. Model-level evaluation lets me probe specific behavior with controlled inputs, including cases the application’s current workflow may never expose. Agent-level evaluation checks what happens with real retrieval, tool results and access boundaries, which a model-only test cannot establish. Testing the agent alone does not systematically isolate every model behavior; testing the model alone does not establish that the application behaves responsibly.

The model receives supplied inputs directly; the agent loops through tools and permission checks before producing an answer and actions.
Model tests control the inputs. Agent tests also follow the tools, records and actions.
Marketplace caseModel levelAgent level
Safety: A buyer requests help fabricating a refund claim.Give the model the request and relevant policy text. Does it refuse deceptive assistance and offer a legitimate alternative?Run the request through the application. Inspect the answer and tool activity to check that the system does not assist the fraud.
Security: A refund-policy document contains an instruction to disclose another buyer’s private order details.Supply the document as reference text. Does the model ignore the embedded instruction? This tests model behavior, not backend access control.Retrieve the document through the application. Check that it actually reached the model, inspect the response, and separately verify that the order service enforces access restrictions.
Values — honesty: A seller requests a shipped-status message despite records showing the item has not shipped.Supply the shipment facts directly. Does the model refuse to knowingly misrepresent them?Let the application retrieve the shipment evidence. Check whether the final answer reflects the actual records and preserves uncertainty where evidence is missing.

Where Governance Fits

AI governance is how an organization sets rules for AI use, assigns responsibilities and checks that those rules are followed. It starts with defining what a system may do and which decisions require human approval. After launch, the organization reviews those rules and the evidence as the system and its data change. NIST’s Govern function guides how organizations map, measure and manage AI risk across the lifecycle. NIST AI RMF Core

At the infrastructure level, it covers identity and access for people and services, privacy rules for sensitive data, where data can be stored or processed, and how long records and logs are kept.

At the agent level, it covers what the agent is allowed to do, which data it may retrieve, which tools it may call and when a person must approve an action. It also establishes who owns those rules, what evidence reviewers need and when a change calls for a new evaluation.

For the marketplace application, shared infrastructure rules might require users and services to authenticate, keep private order and payment data in approved regions, and retain access logs for a defined period. The buyer/seller agent’s policy can limit it to records the signed-in user is allowed to see. The support agent can have broader read access for investigations, while refund approvals and payout actions follow separate rules. Governance assigns owners to these policies, defines which actions need human approval and requires evidence that the access boundaries and approval steps are enforced.

Marketplace governance connects policies and named owners to application enforcement, evidence collection and review.
Marketplace governance connects policy and ownership to controls, evidence and review.

Summary

Safety, security and values give me three ways to review a response or action. I can ask what harm it could cause, how well the system protects its data and resists attacks, and whether its claims and treatment of people meet the expectations for that use. Those questions apply to a model’s answer and to the application that retrieves data and takes actions. Governance gives the organization a way to assign those expectations, put controls in place and review the evidence as the system changes. In the next article, I’ll walk through a practical evaluation framework for both levels, including how to choose benchmarks, design cases, run tests and interpret the results.

Acknowledgements

I used OpenAI Codex to help draft the article and prepare the diagrams.

References

  • NIST AI Risk Management Framework, AI Risks and Trustworthiness. Describes characteristics that matter when assessing whether an AI system is trustworthy. Read the framework section
  • NIST AI Risk Management Framework Core. Describes Govern, Map, Measure and Manage and explains how governance informs the other functions. Read the framework core
  • Microsoft Responsible AI Principles and Approach. Describes Microsoft’s six principles for responsible AI development and use. Read the principles

Leave a Reply