AWS ETL Tools Cheat Sheet

1When to Use What

The decision matrix. Pick the row that matches your problem.

2AWS Glue

ServerlessApache SparkBatch

TL;DR

Managed Spark behind an API. You write PySpark, AWS runs the cluster. Includes a Hive-compatible Data Catalog (central metastore for your lake).

Architecture

S3 Raw

→

Crawler

→

Catalog

→

ETL Job

→

S3 Clean

Cheat Snippet

aws glue start-job-run \
  --job-name my-etl \
  --arguments='--input=s3://raw/,--output=s3://clean/'

Pricing

$0.44/DPU-hour. 1 DPU = 4 vCPU + 16GB. Min 2 DPUs. Crawlers billed separately.

3Amazon EMR

ProvisionedSpark / Hadoop / Presto

TL;DR

Raw Hadoop/Spark clusters you control. Pick instance types, install custom JARs, tune spark.executor.memory. EMR Serverless exists but you lose fine control.

Use Case

• • 50TB+ daily click stream processing
• • ML model training on Spark MLlib
• • Interactive Presto analytics on massive datasets

Cheat Snippet

aws emr create-cluster \
  --release-label emr-7.0.0 \
  --applications Name=Spark Name=Hive \
  --instance-type m5.xlarge --instance-count 5

Pricing

EC2 cost + EMR surcharge (~20%). Use spot for task nodes (up to 90% savings, risk of interruption).

4Amazon Athena

ServerlessPresto / TrinoAd-hoc SQL

TL;DR

Presto-as-a-service. Point at S3, write SQL, get results. No infrastructure. Technically ELT (query-time transform) but everyone searches "ETL" for it.

Cost Comparison

Cheat Snippet

CREATE TABLE clean WITH (
  format='PARQUET',
  partitioned_by=ARRAY['dt']
) AS SELECT user_id, event, dt
FROM raw WHERE dt > '2026-01-01';

5Kinesis (Streams + Firehose)

ServerlessStreaming

TL;DR

Data Streams: Managed Kafka-like shards. You read with consumers (Lambda, KCL).

Firehose: Zero-admin delivery pipe. Buffers → dumps to S3/Redshift/OpenSearch.

Data Analytics: Managed Apache Flink for streaming SQL/Java.

Streaming Architecture

Producers→

Streams

→

Firehose

→

S3 Lake

Cheat Snippet

aws firehose create-delivery-stream \
  --delivery-stream-name clicks \
  --s3-destination-configuration \
    BucketARN=arn:aws:s3:::lake,\
    CompressionFormat=SNAPPY

6AWS DMS

ProvisionedCDCMigration

TL;DR

Reads the source DB's transaction log (binlog/WAL/redo) and replays on target. Full-load + ongoing CDC replication. NOT a transformation engine -it moves data.

Migration Flow

Oracle

→

SCT

→

DMS

→

Aurora

Cheat Snippet

aws dms create-replication-task \
  --replication-task-id ora-to-pg \
  --source-endpoint-arn $SRC \
  --target-endpoint-arn $TGT \
  --migration-type full-load-and-cdc

7Service Comparison

8Glue: Data Lake ETL

What is AWS Glue? Managed Spark for batch ETL. Includes a Data Catalog shared by Athena, EMR, Redshift.

Hub-and-Spoke Architecture

RDS

S3 CSV

DynamoDB

Data Catalog

central metastore

Athena

Redshift

EMR

9EMR: Big Data at Scale

What is Amazon EMR? Full Hadoop/Spark/Presto clusters. You pick instances, install JARs, tune memory.

Cluster Processing (50TB/day)

50TB Raw

EMR Spark Cluster

Master Node

core

core

spot

spot

Parquet

10Athena: SQL on S3

What is Amazon Athena? Serverless Presto/Trino. Point at S3, write SQL, get results. Pay per TB scanned.

Query → Scan → Results

Analyst

SELECT...

Athena

dt=1

dt=2

dt=3

dt=4

S3 partitioned Parquet

$5/TB

scanned

CTAS materialization

CREATE TABLE clean_events
WITH (format='PARQUET', partitioned_by=ARRAY['dt'])
AS SELECT user_id, action, ts, dt
FROM raw_events
WHERE dt > '2026-01-01' AND action IS NOT NULL;

11Kinesis: Streaming Pipelines

What is Amazon Kinesis? Streaming family. Data Streams = managed shards. Firehose = delivery. Flink = streaming SQL.

Multi-shard Fan-out

P1

P2

P3

Data Streams

Shard 0

Shard 1

Shard 2

Lambda

Firehose

Flink

→

S3 Lake

→

Query

12DMS: Database Migration

What is AWS DMS? Reads source DB transaction logs and replays on target. Full-load + ongoing CDC.

Zero-Downtime Migration

Oraclesource

binlog

DMS

full-load

CDC

Aurora PGtarget

lag:0ms

13Batch ETL Pattern

Standard nightly ETL: raw → transform → load

S3 Raw

→

Glue Job

→

S3 Clean

→

Redshift

Crawler discovers schema → PySpark transforms → COPY loads → analysts query

14Streaming ETL Pattern

Real-time clickstream / IoT ingestion

IoT→

Kinesis

→

Firehose

→

S3 Parquet

→

Athena

Buffer 60s → Snappy compress → partition by dt → SQL analytics

15Key Limits & Quotas

16Lake Formation & Governance

• • Manages permissions instead of S3 bucket policies
• • Tag-based access control (LF-Tags)
• • Cross-account data sharing without S3 replication
• • Works with Glue, Athena, Redshift Spectrum, EMR

Rules = [
  ColumnValues "age" between 0 and 150,
  Completeness "email" > 0.95,
  IsUnique "user_id"
]

17Cost Cheat Sheet

What you actually pay (us-east-1, 2026)

Pro tip: Glue + Athena on Parquet is the cheapest serverless stack. EMR on spot is cheapest at petabyte scale.