AI Stack: Products & Software
The AI Stack: Products & Software reference table maps each Well-Architected Framework pillar to the specific AWS, Azure, and open-source technologies needed to build production AI agents with MCP, RAG, and LLM services. It provides a side-by-side comparison across operational excellence, security, reliability, performance, cost, and sustainability to help teams choose managed or open-source components for ingestion, transformation, vectors, inference, and observability.
| Well-Architected Pillar | Capability | AWS Native | Azure Native | Open Source |
|---|---|---|---|---|
| Operational Excellence Automate, monitor, iterate |
Agent Orchestration | Amazon Bedrock Agents + AWS Step Functions | Azure AI Foundry Agent Service + Logic Apps | LangChain, CrewAI, AutoGen, LangGraph |
| MCP Implementation | AWS Lambda + API Gateway (HTTP/WebSocket) | Azure Functions + API Management | FastMCP, MCP SDK (Python/TS) | |
| Workflow Orchestration | Amazon MWAA (Apache Airflow), Step Functions, EventBridge | Azure Data Factory, Microsoft Fabric Data Pipelines, Databricks Workflows | Apache Airflow, Prefect, Dagster | |
| Data Transformation | AWS Glue, Amazon EMR, dbt Cloud | Azure Data Factory, Microsoft Fabric, Azure Databricks, dbt Cloud | dbt Core, SQLMesh, Apache Spark | |
| Observability | Amazon CloudWatch, AWS X-Ray, Bedrock Agent Traces | Azure Monitor, Application Insights, Prompt Flow Tracing | Langfuse, LangSmith, OpenTelemetry, Grafana, Prometheus | |
| LLM Evaluation & Testing | Bedrock Model Evaluation Jobs | Azure AI Foundry Evaluations | Ragas, DeepEval, Promptfoo, TruLens | |
| Prompt Management | Bedrock Prompt Management | Azure Prompt Flow | PromptLayer, LangChain Hub, Pezzo | |
| Feature / Context Store | Amazon SageMaker Feature Store | Azure ML Feature Store | Feast, Tecton OSS | |
| Memory Layer (Agents) | Amazon DynamoDB, ElastiCache for Redis | Azure Cosmos DB, Azure Cache for Redis | Redis, Mem0, Zep | |
| Security Protect data, systems, assets |
Identity & Access | AWS IAM, AWS Lake Formation | Microsoft Entra ID, Azure RBAC | Keycloak, Apache Ranger, Open Policy Agent |
| Data Protection | AWS KMS, Amazon Macie, VPC Endpoints, S3 Encryption | Azure Key Vault, Microsoft Purview, Private Link | HashiCorp Vault, SOPS, Transparent Data Encryption | |
| Secret & Tool Security | AWS Secrets Manager, Bedrock Agent Action Groups | Azure Key Vault, Managed Identities | FastAPI + OAuth2, MCP Auth | |
| AI Guardrails | Amazon Bedrock Guardrails | Azure AI Content Safety | NeMo Guardrails, Llama Guard, Guardrails AI | |
| Model Governance | Amazon SageMaker Model Registry, Model Cards | Azure ML Registry, Responsible AI Dashboard | MLflow, Weights & Biases | |
| Data Lineage & Catalog | AWS Glue Data Catalog, Amazon DataZone | Microsoft Purview, Unity Catalog | OpenLineage, DataHub, Amundsen | |
| Reliability Recover from failure |
LLM Service | Amazon Bedrock Multi-AZ, SageMaker Endpoints + Auto Scaling | Azure OpenAI PTU, Azure ML Managed Online Endpoints | vLLM, TGI, Ray Serve on Kubernetes |
| Data Lakehouse | Amazon S3 + Apache Iceberg via Athena/EMR + Delta Lake via Glue 4.0 | ADLS Gen2 + Delta Lake on Azure Databricks + Iceberg on Synapse Serverless | Apache Iceberg, Delta Lake, Apache Hudi on S3/MinIO | |
| Vector Database | Amazon OpenSearch Serverless, Aurora PostgreSQL + pgvector | Azure AI Search, Azure Cosmos DB for PostgreSQL | Qdrant, Milvus, Weaviate, Chroma | |
| Data Ingestion | Amazon Kinesis, SQS, AWS Glue Streaming, AppFlow | Azure Event Hubs, Service Bus, Azure Data Factory | Apache Kafka, Apache NiFi, Airbyte, Debezium | |
| Caching Layer | Amazon ElastiCache for Redis | Azure Cache for Redis | Redis, DragonflyDB, KeyDB | |
| Performance Efficiency Use resources efficiently |
Inference | AWS Inferentia2, Bedrock Latency-Optimized Inference | Azure ML + NC/H100 VMs, Azure OpenAI PTU | vLLM PagedAttention, TensorRT-LLM, Ollama |
| Data Warehouse | Amazon Redshift Serverless, Redshift RA3, Redshift Spectrum | Azure Synapse Analytics, Microsoft Fabric Warehouse | ClickHouse, Apache Druid, DuckDB | |
| Data Marts | Redshift Datasharing, Redshift Serverless workgroups | Synapse Dedicated SQL Pools, Fabric Lakehouse Shortcuts | dbt Semantic Layer, Cube.js, Trino, StarRocks | |
| RAG + Retrieval | Amazon Bedrock Knowledge Bases, Amazon Kendra | Azure AI Search | LlamaIndex, Haystack, RAGFlow | |
| Embeddings | Amazon Titan Embeddings v2, Cohere on Bedrock | Azure OpenAI text-embedding-3-large | BGE-M3, E5, Nomic Embed, SentenceTransformers | |
| Reranking | Cohere Rerank on Bedrock, SageMaker hosted models | Azure AI Search Semantic Ranker | BGE-Reranker, Cohere API, ColBERT, Cross-Encoders | |
| Hybrid Search | Amazon OpenSearch (BM25 + k-NN) | Azure AI Search Hybrid | Weaviate, Vespa, Qdrant Hybrid | |
| Context Optimization | AWS Lambda preprocessing | Azure Prompt Flow transforms | LLMLingua, Semantic Chunking, Unstructured.io | |
| Cost Optimization Avoid unnecessary cost |
Storage + Table Format | Amazon S3 Intelligent-Tiering + Apache Iceberg via Athena/EMR | ADLS Gen2 Cool/Archive + Iceberg on Synapse/Databricks | Apache Iceberg, Delta Lake on S3/MinIO |
| Token Management | Bedrock Token Usage Metrics, CloudWatch | Azure Cost Management + Billing, Azure OpenAI Metrics | tiktoken, Tokenizers, Langfuse Cost Tracking | |
| Response Caching | API Gateway Cache + ElastiCache, Bedrock Prompt Caching | Azure Front Door + Redis, Azure OpenAI Prompt Caching | GPTCache, Redis Semantic Cache | |
| Model Routing | Bedrock multi-model routing | Azure AI Model Router | LiteLLM, OpenRouter, RouteLLM | |
| Distillation / Fine-tuning | Amazon SageMaker JumpStart, Bedrock Custom Models | Azure ML, Azure OpenAI Fine-tuning | LoRA, QLoRA, PEFT, Axolotl | |
| Sustainability Minimize environmental impact |
Compute & Region | AWS Graviton3, Inferentia, eu-west-1, Spot Instances, S3 Iceberg compaction | Azure Ampere Altra VMs, Sweden Central, Spot VMs, Delta Optimize | Quantization (AWQ, GPTQ, GGUF), Iceberg/Delta compaction, DuckDB |
| Model Optimization | Bedrock model selection, SageMaker Neo | Azure model optimization, ONNX Runtime | Model Distillation, Pruning, ONNX, Smaller 7B/3B models |
Medallion Architecture:
The architecture flows raw data from S3/ADLS through Iceberg/Delta Lake Bronze tables for ACID ingestion, then uses dbt in Silver to clean and conform entities, which feed Gold-layer data marts and vector stores for analytics and RAG. The Gold data marts and vectors are exposed as tools via an MCP server, enabling AI agents to query structured data and retrieve grounded context for LLM inference.
1. Raw Layer
Storage: Amazon S3, Azure ADLS Gen2
Data Sources: PDFs, API JSON, Database CDC, App Logs, SaaS exports
Format: Unstructured, Semi-structured
2. Bronze – Iceberg/Delta Lake
Table Format: Apache Iceberg, Delta Lake
Pattern: Append-only, ACID transactions, Schema Evolution
Partitioning: By date, source_system, tenant_id
Capabilities: Time Travel, Incremental Processing
Azure: ADLS + Synapse/Databricks
OSS: Apache Spark, Trino
3. Silver – dbt Transformation
Orchestration: Apache Airflow, Amazon MWAA, Prefect, Dagster
Transform: dbt Core/Cloud, SQLMesh
Operations: Deduplicate, Clean, Standardize, Conform Dimensions
Quality: Data Tests, Contracts, Documentation
Lineage: OpenLineage, DataHub, Purview
4. Gold – Data Marts + Vectors
Data Warehouse: Amazon Redshift, Azure Synapse, ClickHouse
Data Marts: Star Schema, Dimensional Models, Aggregates
Vector Store: Qdrant, OpenSearch, Azure AI Search
Embeddings: Titan, Azure OpenAI, BGE-M3
Semantic Layer: dbt Semantic Layer, Cube.js
BI: PowerBI, QuickSight
5. Agent Layer – MCP Tool Serving
MCP Server: AWS Lambda + API Gateway, Azure Functions, FastAPI + FastMCP
Agent Framework: Bedrock Agents, Azure AI Foundry, LangGraph
Tools Exposed: SQL query Data Mart, Vector similarity search, REST API calls
LLM Inference: Claude on Bedrock, GPT-4o on Azure OpenAI, Llama 3.1 on vLLM
Memory: DynamoDB, Cosmos DB, Redis for conversation state
Observability: Langfuse, CloudWatch
- Operational Excellence: Airflow + dbt for CI/CD pipelines
- Security: IAM, Lake Formation, KMS encryption at each layer
- Reliability: Iceberg ACID + Multi-AZ Redshift
- Performance: Partitioned Iceberg, Redshift RA3, Vector HNSW indexes
- Cost: S3 Intelligent-Tiering, Spot EMR, Redshift Serverless
- Sustainability: Graviton, Iceberg compaction, Serverless compute
Reference:
- AWS Well-Architected Framework
- Microsoft Azure Well-Architected Framework
- AWS Machine Learning Lens
- NIST AI Risk Management Framework
The AI Stack: Products & Software reference table maps each Well-Architected Framework pillar to the specific AWS, Azure, and open-source technologies needed to build production AI agents with MCP, RAG, and LLM services. It provides a side-by-side comparison across operational excellence, security, reliability, performance, cost, and sustainability to help teams choose managed or open-source components…