08 - AWS Healthcare RAG: Clinical Accuracy Architecture

This guide builds on the AWS Prescriptive Guidance for healthcare RAG. Healthcare RAG turns a generic LLM into a clinician-grade assistant grounded in patient data, clinical guidelines, and medical knowledge graphs, where accuracy is non-negotiable and HIPAA shapes every architectural choice.

Healthcare RAG: What You’ll Learn

Healthcare RAG inherits everything hard about RAG and adds the constraints of a regulated industry: PHI, audit trails, BAAs, and the reality that a confident wrong answer is worse than no answer at all. This guide maps the use cases AWS Prescriptive Guidance endorses three production use cases plus the LangChain-on-Bedrock orchestration that ties them together.

It is written for engineers who already understand basic RAG and want the healthcare-specific delta: which extra controls to add, which retrieval mistakes are unique to clinical data, and where a generic RAG tutorial will actively mislead you in a regulated setting. Healthcare RAG is the variant of RAG where these deltas are non-negotiable, and healthcare RAG is the subject of the rest of this guide.

Why Healthcare RAG Demands Higher Accuracy

Healthcare is the canonical high-stakes domain. A retail chatbot that hallucinates a price costs a refund; a clinical assistant that hallucinates a drug interaction costs a patient. The bar is “good enough to act on, with citations.”

Three forces make healthcare RAG harder: data fragmentation (structured, semi-structured, image data, no single retriever handles all); regulatory scope (PHI encrypted everywhere with audit trails); and clinical risk (a plausible hallucination is the worst failure mode). Healthcare RAG architectures respond to this constraint.

Clinical-accuracy validation cannot rely on generic LLM benchmarks. A model that scores well on trivia or coding tasks tells you nothing about whether it will hallucinate a dosage. Healthcare RAG teams instead build a held-out evaluation set of de-identified clinical questions with physician-adjudicated gold answers.

Every candidate response is scored on three axes: factual grounding (does every claim trace back to a retrieved passage), completeness (did the answer surface all clinically relevant findings, not just the first match), and appropriate refusal (did the system decline to answer when evidence was insufficient, rather than guessing).

A model that answers confidently one hundred percent of the time is not accurate. It is uncalibrated. The refusal rate, tracked over time like any other production metric, is often a better proxy for trustworthiness in healthcare RAG than raw accuracy, because it exposes exactly where retrieval coverage is thin.

How RAG Reduces Clinical Hallucinations

RAG grounds the model’s output in real-world knowledge. At query time, the question and context are embedded to retrieve top-k passages, injected into the context window with a system prompt instructing the model to answer only from provided context and cite passages. The model’s job shifts from recall to reading comprehension.

Healthcare RAG layers three retrieval backends:

• Vector database (Amazon OpenSearch Service): unstructured clinical text; semantic retrieval finds similar cases even when terminology differs.
• Knowledge graph (Amazon Neptune): structured, temporal patient data; relational retrieval.
• Foundation model (Amazon Bedrock): synthesizes a response with citations, calibrated uncertainty, and refusal when evidence is insufficient.

Vector-only misses structured timelines; graph-only cannot interpret free-text nuance. Together they give the model both evidence and relationships. Bedrock, OpenSearch, and Neptune are the canonical trio, all HIPAA-eligible, all integrating cleanly with LangChain.

Chunking strategy matters more in healthcare than in most RAG domains. A discharge summary chunked at a fixed token boundary can split a medication name from its dosage, or a lab result from the reference range that gives it meaning, and a retriever that returns the fragment without context reintroduces exactly the ambiguity RAG exists to remove.

Healthcare RAG pipelines typically chunk on clinical structure (per encounter, per note section, per lab panel) rather than fixed token windows, then attach structured metadata (patient ID, encounter date, note type) to every chunk so OpenSearch can filter before it ever runs a similarity search.

A second-stage re-ranker, often a smaller cross-encoder running inside the same VPC, reorders the top candidates by relevance to the specific clinical question before they reach the Bedrock context window. This matters when k is kept small (eight to ten passages) to protect the latency budget.

Use Case 1: Patient Data Augmentation

In a busy clinical environment, clinicians need a synthesized view of patient history at the point of care. The EHR has the data but scattered across notes, labs, and medications no human can read in 15 minutes. A medical intelligence engine retrieves history from Neptune, clinical knowledge from OpenSearch, and uses a Bedrock LLM to synthesize a summary scannable in seconds:

• Step 1, Discovering data: ingest MIMIC-IV plus EHR feeds.
• Step 2, Knowledge graph: extract entities via few-shot prompting; load temporal relationships into Neptune.
• Step 3, Retrieval agents: deploy a Bedrock or LangChain agent that queries the knowledge graph.
• Step 4, Knowledge base: index unstructured clinical text into OpenSearch.
• Step 5, Responses: LangChain coordinates retrieval across Neptune and OpenSearch, hands combined context to a Bedrock LLM.

The key decision is Bedrock agent vs LangChain agent for the graph retrieval layer. Most systems start with LangChain and migrate to Bedrock agents once the retrieval pattern stabilizes.

Entity extraction into Neptune is rarely a single pass. Clinical notes mix structured fields (vitals, orders) with free text that describes the same facts differently, “BP 140/90” in one note and “blood pressure elevated at follow-up” in another. Amazon Comprehend Medical is a common first-pass extractor for named clinical entities before a Bedrock LLM resolves relationships and disambiguates mentions across visits.

The resulting graph edges (patient PRESCRIBED medication, medication TREATS condition, lab_result FLAGGED_ABNORMAL_FOR condition) are what let Neptune answer temporal and relational questions no vector search can: which patients started a new medication in the last 30 days and had a subsequent abnormal lab.

Use Case 2: Predicting Hospital Re-Admission Risk

This use case predicts which patients are at elevated re-admission risk after discharge, combining predictive ML with generative AI for both a numerical risk score and a natural-language recommendation. Three retrieval backends: Neptune for structured chronological data (encounters, prior re-admissions, labs, adherence); OpenSearch for unstructured data (discharge summaries, missed-appointment notes); and a fine-tuned Bedrock LLM consuming both.

Four steps: (1) predict outcomes via Neptune time-series analysis plus a fine-tuned LLM classifying deterioration/improvement/stable; (2) predict behavior from adherence and missed-appointment patterns; (3) combine signals into a patient-level score; (4) aggregate into a facility-wide propensity score.

Scores are indexed in OpenSearch so care managers query via a conversational agent at patient, facility, or specialty level. Alerts fire on threshold crossings, encouraging proactive intervention.

The re-admission model benefits from combining a discriminative signal with a generative one instead of treating the problem as pure classification. Neptune supplies the time-series features (days since discharge, count of prior admissions in the trailing twelve months, medication-adherence gaps inferred from refill timing) that feed a fine-tuned model producing a calibrated probability.

Separately, a Bedrock LLM reads the unstructured discharge summary and missed-appointment notes to surface qualitative risk factors a numeric score misses entirely: a patient living alone, a language barrier, an unresolved transportation issue.

Neither signal alone is sufficient for healthcare RAG-driven care management. The numeric score without narrative context tells a care manager who is at risk but not why, and the narrative alone doesn’t scale to a facility-wide triage list.

Use Case 3: Healthcare Talent Management with AI

The third use case is internal: managing and upskilling the workforce. Healthcare RAG reads each employee’s profile, identifies skill gaps against the target role, and recommends personalized learning pathways. Three components:

• Intelligent resume parser: a fine-tuned Llama model in Bedrock that extracts skills, role, experience, and certifications.
• Talent knowledge graph: Neptune, modeling role and skill taxonomy.
• Learning recommendation engine: measures semantic similarity between candidate and target skills, identifies gaps, recommends content.

Three steps: build a skills profile; traverse Neptune for role-to-skill relevance; feed gaps into the recommendation engine. This is RAG retrieving skills rather than documents. The same Bedrock + Neptune + OpenSearch architecture applies.

The similarity scoring itself reuses the same Bedrock embedding model as the clinical-text pipeline. Skills and role descriptions are embedded once, cached, and compared with cosine similarity, so no separate matching infrastructure is needed for this internal, lower-stakes use case.

AWS Architecture for Healthcare RAG

The reference architecture centers on three pillars: Amazon Bedrock (foundation model), Amazon OpenSearch Service (vector database), and Amazon Neptune (knowledge graph).

Amazon Bedrock hosts the LLM (Claude, Llama, Titan, Mistral) plus fine-tuned variants. It provides managed Agents and Knowledge Bases, is HIPAA-eligible, supports customer-managed KMS keys, and never trains base models on your data.

Amazon OpenSearch Service stores embeddings of unstructured clinical text, serving k-NN queries with approximate (HNSW) and exact (script-score) search, hybrid search, and per-tenant isolation.

Amazon Neptune stores structured patient data in a property graph (Gremlin/openCypher), excelling at multi-hop queries like “all patients who took medication X and had lab Y outside range within 30 days.”

LangChain orchestrates Neptune + OpenSearch retrievers, merging context for Bedrock. Teams migrate to managed Bedrock Agents when the pattern stabilizes. Amazon SageMaker handles surrounding data-science: fine-tuning extraction, embedding pipelines, evaluation.

Integration between the three pillars follows a consistent pattern regardless of use case. OpenSearch and Neptune are never queried directly by the front end. A LangChain (or Bedrock Agent) orchestration layer sits between the user-facing application and both data stores, so retrieval logic, filtering, and access control live in one place instead of being duplicated across every client.

Embeddings written to OpenSearch are generated by a Bedrock embedding model (Titan Embeddings, typically) at ingest time, and that same model, not a substitute, must be used again at query time, or the vector space won’t align and similarity scores become meaningless.

Neptune’s property graph schema is usually versioned separately from the application code, because adding a new edge type (say, capturing a new class of adverse event) is a schema migration that needs its own clinical review, unlike an ordinary code deploy.

Data residency and per-tenant isolation shape the physical layout of both stores. A multi-facility healthcare RAG deployment usually gives each facility, or in stricter deployments each patient cohort, its own OpenSearch index and its own Neptune graph partition, encrypted with its own customer-managed KMS key, so a single compromised credential can’t expose PHI across facilities.

SageMaker sits alongside this pipeline rather than inside the request path. Its role is offline: fine-tuning extraction and summarization models on institution-specific vocabulary, running the embedding backfill when a new note type is onboarded, and executing the weekly regression evaluation suite.

LangChain Orchestration on Amazon Bedrock

LangChain composes cleanly with all three pillars: Neptune retriever, OpenSearch retriever, Bedrock LLM. Each retriever is a tool the agent calls based on the question. The canonical pattern runs retrievals in parallel rather than sequence. Sequential retrieval doubles latency with no value when retrievals are independent. LangChain’s RunnableParallel keeps response inside the 3-second window clinicians expect.

When the pattern stabilizes, teams migrate to managed Bedrock Agents, trading flexibility for operational simplicity. Security wraps every layer: customer-managed KMS keys encrypt data at rest; VPC endpoints keep traffic private; CloudTrail logs every API call; IAM roles enforce least privilege; a signed BAA covers every HIPAA-eligible service.

Compliance: HIPAA-Eligible AWS Services

Compliance shapes every service selection from day one. “HIPAA-eligible” means AWS has signed a BAA and supports the controls required to process PHI. Every service in the reference architecture is HIPAA-eligible: Bedrock, OpenSearch, Neptune, SageMaker, plus S3, KMS, IAM, CloudTrail, and VPC. Five controls deserve attention:

• Customer-managed KMS keys: never AWS-owned keys for PHI.
• Per-tenant isolation: partition PHI per facility with separate KMS keys and IAM roles.
• Audit trails: every retrieval, model call, and tool invocation logged with user, patient, and timestamp.
• Data residency: keep PHI inside the region of patient consent.
• BAA coverage: confirm a signed BAA is on file before any PHI touches the stack.

Data minimization is worth adding as an unwritten sixth control: retrieve only the fields a given use case needs. A talent-management query has no legitimate reason to touch clinical notes, and enforcing that boundary at the retriever level, not just in application logic, narrows the blast radius of any future bug.

Compliance is a property of the architecture, not a checklist at the end. Build it in from the first commit.

A signed BAA is necessary but not sufficient. HIPAA’s technical safeguards require that access to PHI be both authenticated and auditable at the level of the individual retrieval, not just the application. Every call from the LangChain orchestrator to Neptune or OpenSearch should be tagged with the requesting clinician’s IAM identity and the patient ID being queried, flowing into CloudTrail alongside the Bedrock invocation.

Teams that skip this step discover during an audit that they can prove the application accessed PHI, but not which user, for which patient, for what reason, a gap that turns a routine review into a remediation project.

The fix for healthcare RAG is architectural, not procedural: pass identity and purpose as first-class parameters through every layer of the retrieval chain instead of bolting on logging after the fact.

Healthcare RAG in Practice: Oncology Patient Summarization Agent

To make the architecture concrete, consider an oncology ward admitting 200 patients per day. Each patient has a complex history: prior visits, chemotherapy regimens, labs, months of clinical notes. The attending oncologist has 15 minutes per patient. Without augmentation those minutes are spent chart-hunting; with a RAG agent, on decisions.

The trigger is almost always an EHR event, not a user-initiated search. When the clinician opens a chart, the EHR fires a webhook carrying patient ID and visit reason, and that event, not a typed query, is what kicks off the retrieval pipeline described below.

A RAG agent connected to the EHR retrieves the patient’s last 5 visits, current medications, and lab values, then generates a structured risk-stratification summary, all in under 3 seconds.

Three seconds is the constraint. The clinician opens the chart; the EHR fires an event with patient ID and visit reason. A LangChain orchestrator dispatches parallel retrievals: Neptune for structured history (last 5 visits, medications, labs), OpenSearch for relevant unstructured notes.

When both return, the orchestrator merges them, adds a system prompt instructing the Bedrock LLM to summarize only from provided context and flag missing data, and calls Bedrock. The LLM returns a structured summary: clinical synopsis, active concerns, risk tier with rationale, and recommended next steps.

from langchain.agents import AgentExecutor, create_openai_tools_agent
from langchain_aws import ChatBedrock, NeptuneGraphQAChain
from langchain_opensearch import OpenSearchVectorStore
from langchain.tools import Tool

# 1. Structured retrieval from Neptune knowledge graph
neptune_qa = NeptuneGraphQAChain.from_llm(
    llm=ChatBedrock(model_id="anthropic.claude-3-5-sonnet"),
    graph=neptune_graph_connection,
)
structured_tool = Tool(
    name="patient_history_graph",
    description="Retrieve structured patient history: visits, meds, labs.",
    func=lambda q: neptune_qa.invoke({"query": q.replace("PATIENT_ID", patient_id)}),
)

# 2. Unstructured retrieval from OpenSearch vector store
notes_store = OpenSearchVectorStore(
    opensearch_url=OS_URL,
    index_name="clinical_notes_encrypted",
    embedding_function=bedrock_embeddings,
)
unstructured_tool = Tool(
    name="clinical_notes_vector_search",
    description="Semantic search over discharge summaries and physician notes.",
    func=lambda q: notes_store.similarity_search(q, k=8, filter={"patient_id": patient_id}),
)

# 3. Orchestrator: parallel retrieval, then Bedrock synthesis
oncology_agent = AgentExecutor.from_agent_and_tools(
    agent=create_openai_tools_agent(bedrock_llm, [structured_tool, unstructured_tool], SYSTEM_PROMPT),
    tools=[structured_tool, unstructured_tool],
    max_iterations=3,
    early_stopping_method="generate",
)
summary = oncology_agent.invoke({"input": f"Risk-stratify patient {patient_id} for {visit_reason}"})

Three engineering details matter, each addressable directly in code. Patient-ID scoping means every retriever call (the Neptune query, the OpenSearch similarity search) receives patient_id as a hard filter applied before ranking, not as a post-hoc check; a filter applied after retrieval is a data leak waiting to happen.

The system prompt instructs the model to cite the source passage for every clinical claim and to respond with an explicit “insufficient data” flag rather than a low-confidence guess. Refusal is a feature, not a failure mode, in healthcare RAG.

The latency budget is protected by three levers together: parallel (not sequential) retrieval across Neptune and OpenSearch, a conservative k (eight passages), and model selection matched to task complexity: Haiku-class models for routine summarization, escalating to Sonnet-class only when reasoning is required.

A fourth detail is often missed until production: the evaluation harness for this agent needs its own held-out set, distinct from the general healthcare RAG benchmark, because oncology summarization has a narrower failure mode (omitted medication changes) that a general clinical QA benchmark won’t surface.

Weekly regression runs against fifty to one hundred oncology-specific gold summaries catch drift introduced by a retriever config change or a model version bump before it ever reaches a clinician.

The success metric is clinician trust: the percentage of summaries read without re-checking the raw chart. Systems that hallucinate get turned off and never turned back on.

Healthcare RAG: Common Mistakes to Avoid

• Using the base LLM without grounding: an ungrounded LLM in a clinical setting is a liability.
• Vector-only retrieval, no knowledge graph: loses temporal and relational structure.
• No patient_id scoping: always filter at the index level.
• Single shared KMS key across tenants: use per-tenant customer-managed keys.
• Optimizing for accuracy, ignoring refusal: a confident wrong answer is worse than “I don’t have enough data.”
• Skipping the BAA check: verify before any PHI touches the stack.
• No human-in-the-loop: every action with clinical consequence requires human sign-off.

Of these, the two that surface latest and cost the most are the missing patient_id scope and the missing refusal path, both pass every functional test in a staging environment with a single synthetic patient, and both fail only under multi-tenant production load or on the first genuinely out-of-distribution clinical question. Treat both as launch blockers for any healthcare RAG deployment, not backlog items to revisit after go-live.

Healthcare RAG: Best Practices

• Default to the three-pillar architecture: Bedrock, OpenSearch, Neptune.
• Scope every retriever by patient_id at the index level: hard filter.
• Fine-tune the Bedrock LLM on your institution’s clinical vocabulary.
• Use LangChain during development; migrate to Bedrock Agents when stable.
• Build refusal into the system prompt and measure refusal rate as a health metric.
• Encrypt every store with customer-managed KMS keys; per-tenant keys for multi-tenant.
• Log every retrieval, model call, and tool invocation with patient_id, user_id, timestamp.
• Run a weekly regression evaluation suite on held-out clinical questions.
• Keep a human in the loop for every action with clinical consequence.

Turning “run a weekly regression evaluation suite” into a working practice requires a held-out set that never leaks into training or prompt engineering. The moment engineers tune a prompt against the same questions used to measure accuracy, the score stops measuring anything except overfitting.

Healthcare RAG teams typically maintain two disjoint sets: an engineering set used freely during development, and a locked evaluation set, ideally curated by a clinician outside the engineering team, whose scores are the only ones reported to stakeholders.

Measuring clinical accuracy also means picking the right unit of measurement. A single overall accuracy percentage hides failure modes that matter. A system can score ninety-five percent and still be dangerous if the five percent it gets wrong clusters around one drug class or patient population.

Healthcare RAG evaluation suites should stratify results by use case, by data source (structured Neptune facts vs. unstructured OpenSearch retrieval), and, where cohort size allows, by patient subgroup. An architecture that performs well on average but poorly for an underrepresented population is not production-ready.

Healthcare RAG: Key Takeaways

• Healthcare RAG raises the accuracy bar: grounding and refusal are non-negotiable.
• Three retrieval backends: Bedrock, OpenSearch, Neptune.
• Three production use cases: patient augmentation, re-admission risk, talent management.
• LangChain orchestrates; Bedrock stabilizes: migrate stable flows to Bedrock Agents.
• Compliance is architectural: HIPAA-eligible services, customer-managed KMS keys, per-tenant isolation from the first commit.
• Human-in-the-loop is mandatory: healthcare RAG augments clinicians.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on Amazon Web Services lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which Amazon Web Services, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. Amazon Web Services offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. Amazon Web Services is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an Amazon Web Services default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. Amazon Web Services HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on Cloud Platforms lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which Cloud Platforms, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. Cloud Platforms offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. Cloud Platforms is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an Cloud Platforms default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. Cloud Platforms HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on Enterprise Infrastructure lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which Enterprise Infrastructure, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. Enterprise Infrastructure offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. Enterprise Infrastructure is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an Enterprise Infrastructure default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. Enterprise Infrastructure HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on Managed Cloud Services lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which Managed Cloud Services, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. Managed Cloud Services offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. Managed Cloud Services is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an Managed Cloud Services default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. Managed Cloud Services HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

HIPAA, BAA, and the Shared Responsibility Line

Healthcare RAG on AWS lives inside a compliance boundary defined by HIPAA, the Business Associate Agreement, and the shared responsibility model. The BAA is the contract under which AWS, as a business associate of a covered entity, agrees to safeguard protected health information processed through covered services. AWS offers a standard BAA for covered services, and the practical first step is confirming every service in the planned healthcare RAG architecture appears on that list. Bedrock, S3, OpenSearch Service, RDS, Lambda, and SageMaker are covered; a service missing from the list cannot touch PHI without breaking the compliance boundary.

The shared responsibility line is where most architecture reviews spend their time. AWS is responsible for the security of the cloud, the physical infrastructure, the service-level controls, and the configuration options it exposes. The customer is responsible for security in the cloud, the data, the IAM policies, the encryption settings, and the audit logging. For a healthcare RAG system, the customer side of the line is where PHI flows: the S3 bucket holding clinical source documents, the OpenSearch index holding embeddings, the CloudWatch log streams that might capture prompt text, and the Bedrock invocation logs that capture completion text.

Encryption at rest and in transit is a HIPAA technical safeguard and an AWS default for the covered services, but defaults are not the same as configuration. Confirming that the S3 bucket has a KMS key, that the OpenSearch domain has node-to-node encryption and encryption at rest enabled, that the RDS instance has storage encryption, and that the Bedrock invocation logging destination bucket has its own KMS key is a checklist that takes an hour and closes the most common gaps. The KMS keys themselves live in an account the customer controls, which keeps key administration on the customer side of the shared-responsibility line.

Audit logging is the other technical safeguard that has to be configured deliberately. CloudTrail records the API calls that change the system, CloudWatch Logs records the runtime behavior of the RAG pipeline, and Bedrock invocation logging records the prompts and completions. For healthcare RAG, all three land in destinations with their own access controls, retention policy, and (where required) immutability. A compliance audit asks to see who accessed what PHI and when; the answer is only as good as the logging that was turned on before the access happened.

De-identification and the PHI Boundary in the Corpus

Not every document in a healthcare RAG corpus needs to carry PHI, and the design choice about where to draw the de-identification line shapes the whole architecture. A corpus of de-identified clinical notes, where the 18 HIPAA identifiers have been stripped or generalized, can run on a wider set of services and poses a smaller breach risk if it leaks. A corpus that retains PHI, because the clinical question genuinely depends on patient identity or on identifiers that cannot be removed, has to stay entirely inside the HIPAA-covered boundary for its whole lifecycle.

De-identification is hard to do well and harder to verify. AWS HealthLake provides a managed entity-recognition pipeline that detects and removes the standard identifiers, and a number of teams augment it with a domain-specific pass for clinical jargon, abbreviation expansion, and the edge cases where a single note mentions a rare condition specific enough to re-identify a patient in a small population. The rule of thumb is that de-identification is a process with measurable residual risk, not a binary state; the team tracks the false-negative rate of the entity recognizer on a held-out clinical set and treats the residual PHI rate as a known, accepted risk.

The retrieval path inherits the boundary. If the corpus is de-identified, the embeddings, the index, and the prompt context are all de-identified, and a leak of any one layer exposes only the de-identified text. If the corpus retains PHI, then every layer, embeddings included, has to stay inside the boundary, which constrains the choice of embedding model, the choice of vector store, and even whether the embedding call can leave the account. Some teams run open embedding models on SageMaker inside the account precisely to keep PHI-bearing text from crossing into a managed embedding endpoint.

Citation, Grounding Thresholds, and Clinical Fidelity

A healthcare RAG answer that is fluent but unsupported is worse than no answer, because it carries the surface authority of a confident clinical statement. The structural answer to this risk is citation: every claim in the generated answer is tied to a specific retrieved passage, and the user interface surfaces the tie so a clinician can verify the source before acting on it. Bedrock Knowledge Bases returns the source chunks alongside the generated answer, which is the raw material for the citation layer; the application builds the citations from there.

The grounding threshold is the configuration that decides when the model is allowed to answer at all. Below a threshold of retrieved-chunk relevance, the system declines and returns a fallback message rather than risking an ungrounded answer. Setting the threshold is a clinical-safety decision, not just a retrieval-quality one, because the cost of a wrong answer in a clinical context is much higher than the cost of a refused answer. Teams that ship healthcare RAG usually start with a conservative threshold and loosen it only as eval evidence accumulates that the refusal rate is too high for genuinely answerable questions.

Fidelity is the property that the answer reflects the source faithfully, without paraphrase drift, omission of a qualifier, or smoothing-over of a contradiction in the evidence. The eval harness measures fidelity by comparing each generated claim against its cited source, flagging any claim that goes beyond, contradicts, or selectively quotes the source. A fidelity failure is a finding even when the answer happens to be clinically correct, because the process has broken down in a way that will eventually produce a wrong answer.

The human-in-the-loop checkpoint is the last line of defense and the one that should never be removed for clinical decisions. A healthcare RAG system that drafts a summary, surfaces citations, and then hands the draft to a clinician for review before it reaches the chart is operating in its correct role: amplifying the clinician’s reach, not replacing their judgment. Removing the clinician review to save latency is the kind of optimization that works right up until the first patient harm event, and the corrective is to treat the review step as inviolable.

Evaluation Datasets and the Drift Problem

A healthcare RAG system without an evaluation dataset is a black box that nobody can reason about. The eval set is a curated collection of clinical questions, the retrieved sources that should answer them, the expected answer shape, and the failure modes to watch for. Building the eval set is a clinician-and-engineer collaboration: the clinician supplies the questions and the ground truth, the engineer supplies the retrieval and generation pipeline, and together they grade a sample of outputs against the rubric.

Public clinical eval sets like MedQA, MedMCQA, and PubMedQA give a starting point for the question-and-answer shape, but they are not a substitute for a domain-specific eval set drawn from the actual corpus and the actual clinical use case. A system that answers multiple-choice medical board questions correctly can still fail at the specific task of summarizing oncology notes for a tumor board, because the two tasks draw on different evidence and different answer shapes. The eval set has to reflect the real workload or it measures the wrong thing.

Drift is the slow failure mode that the eval set catches. The corpus ages as clinical guidelines update, new studies publish, and old recommendations are withdrawn; the embedding model ages as newer models change the geometry of the retrieval space; the clinical question mix ages as the user base shifts. Any of these can degrade answer quality without an obvious incident, and the defense is a periodic re-run of the eval set with the results trended over time. A drop in eval-set pass rate after a corpus refresh or an embedding-model upgrade is the early signal that prevents a quiet clinical-quality regression.

Healthcare RAG: Frequently Asked Questions

Why is RAG valuable in healthcare specifically?

RAG grounds LLM responses in authoritative clinical data, reducing hallucinations and making answers auditable. Grounding is a prerequisite, not an enhancement.

How does the patient re-admission use case use RAG?

It retrieves structured history from Neptune and unstructured text from OpenSearch, then feeds both to a fine-tuned Bedrock LLM that generates a re-admission risk score and natural-language recommendation.

Which foundation model provider anchors the healthcare RAG architecture?

Amazon Bedrock hosts the LLM, provides managed Agents and Knowledge Bases, supports fine-tuning, HIPAA-eligible with customer-managed KMS keys. Never uses prompts or completions to train base models.

Which services in this guide are HIPAA-eligible?

Bedrock, OpenSearch, Neptune, and SageMaker: the foundation-model, vector-database, knowledge-graph, and fine-tuning layers. All covered under a signed AWS BAA.

Should I use a Bedrock Agent or LangChain for orchestration?

Start with LangChain during development for flexibility. Migrate to Bedrock Agents when the pattern stabilizes. Most production systems run a mix.

Healthcare RAG on AWS, anchored by Bedrock, OpenSearch, and Neptune, turns a generic LLM into a clinician-grade assistant. Scope every retriever by patient, build refusal into the prompt, audit every call, keep a human in the loop.