AI Stack: Products & Software

by krishna
May 1, 2026

The AI Stack: Products & Software reference maps Well-Architected pillars to modern AWS, Azure, and open-source technologies used to build production AI agents with MCP, RAG, and LLM systems. It compares ingestion, transformation, retrieval, inference, evaluation, and observability layers to help teams choose managed vs open architectures.

Pillar	Capability	AWS	Azure	Open Source
Operational Excellence	Agent Orchestration	Bedrock Agents, Step Functions, EventBridge Pipes	Azure AI Foundry Agents, Logic Apps, Semantic Kernel	LangGraph, CrewAI, AutoGen
	MCP / Tool Serving	Lambda + API Gateway, Bedrock Converse API Tool Use	Azure Functions, API Management, Azure OpenAI Function Calling	MCP SDK, FastMCP, OpenAPI tool servers
	Workflow	Step Functions, MWAA	Fabric Data Pipelines, Data Factory	Dagster, Prefect, Airflow
	Transformation	Glue 5.0, EMR, dbt Cloud	Fabric, Databricks, Data Factory, dbt Cloud	dbt Core, SQLMesh
	Observability	CloudWatch, Bedrock Agent Traces, X-Ray, OpenTelemetry	Azure Monitor, App Insights, AI Foundry Tracing	Langfuse, Helicone, Arize Phoenix, OpenTelemetry
	Evaluation	Bedrock Model Evaluation Jobs, SageMaker Clarify	AI Foundry Evaluations	Ragas, DeepEval, Promptfoo, TruLens
	Prompt Mgmt	Bedrock Prompt Management	Prompt Flow, AI Foundry	DSPy, Promptfoo, Instructor, Pezzo
	Feature Store	SageMaker Feature Store	Azure ML Feature Store	Feast
	Memory	DynamoDB, ElastiCache for Redis	Cosmos DB, Azure Cache for Redis	Zep, Mem0, LanceDB
	Model Routing	Bedrock Intelligent Prompt Routing	Azure AI Model Router	LiteLLM, OpenRouter, Portkey
Security	Identity	AWS IAM	Microsoft Entra ID	Keycloak, Open Policy Agent
	Data Protection	KMS, Macie	Key Vault, Purview	HashiCorp Vault
	Secrets	Secrets Manager	Key Vault	FastAPI + OAuth2, MCP Auth spec
	Guardrails	Bedrock Guardrails	Azure AI Content Safety	NeMo Guardrails, Llama Guard
	Governance	SageMaker Model Registry, Model Cards	Azure ML Registry	MLflow, Weights & Biases
Reliability	LLM Serving	Bedrock, SageMaker Endpoints	Azure OpenAI, Azure ML Endpoints	vLLM, TGI, Ollama, NVIDIA NIM
	Lakehouse	S3 + Iceberg via Athena/EMR	ADLS + Delta on Fabric/Databricks, Iceberg on OneLake	Iceberg, Delta Lake, Hudi
	Vector DB	OpenSearch Serverless, Aurora pgvector	Azure AI Search	Qdrant, Weaviate, Milvus, LanceDB
	Streaming	Kinesis Data Streams, SQS	Event Hubs	Kafka, Redpanda
Performance	Inference	Inferentia2	Azure OpenAI Provisioned Throughput	TensorRT-LLM, vLLM
	Warehouse	Redshift Serverless, Redshift Spectrum	Synapse, Fabric Warehouse	ClickHouse, DuckDB
	RAG	Bedrock Knowledge Bases	Azure AI Search	LlamaIndex, Haystack 2
	Embeddings	Titan Embeddings v2, Cohere	text-embedding-3-large	BGE-M3, E5
	Rerank	Cohere Rerank on Bedrock	Azure AI Search Semantic Ranker	ColBERT, BGE-Reranker
	Hybrid	OpenSearch	Azure AI Search Hybrid	Weaviate, Vespa
Cost	Storage	S3 Intelligent-Tiering	ADLS Cool/Archive	Iceberg compaction
	Caching	ElastiCache, Bedrock Prompt Caching	Azure Cache for Redis, Azure OpenAI Prompt Caching	GPTCache, Redis Semantic Cache
	Routing	Bedrock Intelligent Prompt Routing	Azure AI Model Router	LiteLLM
Sustainability	Compute	Graviton, Spot, Inferentia	Ampere Altra, Spot VMs, Sweden Central	Quantization: AWQ, GPTQ, GGUF
Sustainability	Optimization	SageMaker Neo, Model Distillation	ONNX Runtime	Pruning, Small models 3B/7B

Medallion Architecture

The architecture flows raw data from S3/ADLS through Iceberg/Delta Lake Bronze tables for ACID ingestion, then uses dbt in Silver to clean and conform entities, which feed Gold-layer data marts and vector stores for analytics and RAG. The Gold data marts and vectors are exposed as tools via an MCP server, enabling AI agents to query structured data and retrieve grounded context for LLM inference.

1. Raw Layer

Storage: Amazon S3, Azure ADLS Gen2

Data Sources: PDFs, API JSON, Database CDC, App Logs, SaaS exports

Format: Unstructured, Semi-structured

Ingestion: Apache NiFi, Airbyte, Amazon Kinesis, Azure Event Hubs, AWS Glue Streaming, Debezium

→

2. Bronze — Iceberg / Delta Lake

Table Format: Apache Iceberg, Delta Lake

Pattern: Append-only, ACID transactions, Schema Evolution

Partitioning: By date, source_system, tenant_id

Capabilities: Time Travel, Incremental Processing

AWS: S3 + Athena/EMR
Azure: ADLS + Fabric/Databricks
OSS: Apache Spark, Trino

3. Silver — dbt Transformation

Orchestration: Apache Airflow, Amazon MWAA, Prefect, Dagster

Transform: dbt Core/Cloud, SQLMesh

Operations: Deduplicate, Clean, Standardize, Conform Dimensions

Quality: Data Tests, Contracts, Documentation

Tests: dbt tests, Great Expectations
Lineage: OpenLineage, DataHub, Purview

→

4. Gold — Data Marts + Vectors

Data Warehouse: Amazon Redshift, Azure Synapse, ClickHouse

Data Marts: Star Schema, Dimensional Models, Aggregates

Vector Store: Qdrant, OpenSearch, Azure AI Search

Embeddings: Titan, Azure OpenAI, BGE-M3

Semantic Layer: dbt Semantic Layer, Cube.js

RAG: Bedrock Knowledge Bases, LlamaIndex
BI: PowerBI, QuickSight

5. Agent Layer — MCP Tool Serving

MCP Server: AWS Lambda + API Gateway, Azure Functions, FastAPI + FastMCP

Agent Framework: Bedrock Agents, Azure AI Foundry, LangGraph

Tools Exposed: SQL query Data Mart, Vector similarity search, REST API calls

LLM Inference: Claude on Bedrock, GPT-4o on Azure OpenAI, Llama 3.1 on vLLM

Memory: DynamoDB, Cosmos DB, Redis for conversation state

Guardrails: Bedrock Guardrails, Azure Content Safety
Observability: Langfuse, CloudWatch

Well-Architected Mapping:

Operational Excellence: Airflow + dbt for CI/CD pipelines
Security: IAM, Lake Formation, KMS encryption at each layer
Reliability: Iceberg ACID + Multi-AZ Redshift
Performance: Partitioned Iceberg, Redshift RA3, Vector HNSW indexes
Cost: S3 Intelligent-Tiering, Spot EMR, Redshift Serverless
Sustainability: Graviton, Iceberg compaction, Serverless compute

References

The AI Stack: Products & Software reference maps Well-Architected pillars to modern AWS, Azure, and open-source technologies used to build production AI agents with MCP, RAG, and LLM systems. It compares ingestion, transformation, retrieval, inference, evaluation, and observability layers to help teams choose managed vs open architectures. Pillar Capability AWS Azure Open Source Operational Excellence…