Module Review: Streams & Lambda

[!NOTE] This module explores the core principles of Module Review: Streams & Lambda, deriving solutions from first principles and hardware constraints to build world-class, production-ready expertise.

1. Key Takeaways

Event-Driven Architecture: DynamoDB Streams allow you to decouple your database from your downstream consumers (Search, Analytics, Notifications).
Strict Ordering: Changes to a single Item (Partition Key) are strictly ordered.
Lambda Integration: The Event Source Mapping (ESM) polls shards for you. You must tune BatchSize and BatchWindow for optimal throughput/latency.
Error Handling: A single failed record can block the entire shard (“Poison Pill”). Always use BisectBatchOnFunctionError and return BatchItemFailures.
Monitoring: Watch IteratorAge. If it grows, your consumers are too slow or failing.

2. Flashcards

Retention Period

How long does DynamoDB Streams keep data?

24 Hours

Data is deleted after 24 hours. For longer retention (up to 365 days), use Kinesis Data Streams.

IteratorAge

What does a high IteratorAge indicate?

Processing Lag

It means your consumer is processing old records. You are falling behind the write rate of the table.

BisectBatchOnFunctionError

What does this setting do?

Splits Failed Batches

If a batch fails, AWS splits it into two halves and retries them separately to isolate the bad record.

Stream View Type

Which view type gives you both the before and after state of an item?

NEW_AND_OLD_IMAGES

It provides the full item as it appeared before and after the modification.

Idempotency

Why must stream consumers be idempotent?

At-Least-Once Delivery

DynamoDB Streams guarantees at-least-once delivery. Retries may cause the same record to be delivered multiple times.

Concurrency Limit

How many Lambda instances can process a single shard at once?

Exactly One

One Lambda invocation per shard to guarantee strict ordering of records.

3. Production Checklist

Before going live with DynamoDB Streams, ensure you have configured the following:

Error Handling: Enable BisectBatchOnFunctionError. Partial Failures: Handler returns BatchItemFailures for failed records. Dead Letter Queue: Configure an SQS DLQ for records that fail all retries. Monitoring: Create a CloudWatch Alarm for IteratorAge > 10000 (10s). Batch Tuning: Adjust BatchWindow (e.g., 1-10s) to reduce invocations if low latency isn't critical. Throughput: Ensure downstream services (e.g., OpenSearch, SQL) can handle the write throughput of your table.

4. Next Steps

You have mastered the event-driven side of DynamoDB. In the final module, we will explore advanced optimization techniques, including Time To Live (TTL), Transactions, and Capacity Planning.

Module 5: Optimization & Best Practices →