Tools: Part 7: Decoupled Architecture

1. Goal of Today’s Architecture ## 2. Final Architecture ## 3. Step-by-Step ## Step 1 — User Authentication with Cognito ## 4. Step 2 — Secure API with API Gateway ## 5. Step 3 — Lambda API Backend ## 6. Step 4 — Event Streaming with Kinesis ## 7. Step 5 — Storage in S3 ## 8. Security Improvements We Added ## 9. Why Kinesis Was Added ## 10. Key DevOps Concepts Demonstrated ## Authentication ## API security ## Serverless compute ## Event-driven architecture ## Durable storage ## Decoupling ## 11. How to Explain This in an Interview ## 12. Real Systems That Use This Architecture Today we enhanced an existing serverless architecture by adding: This allows the system to: The full architecture now looks like this: We create a user authentication system. Then we exchanged that code for a token using: The access token is used to access APIs. We configured Amazon API Gateway. API Gateway was configured with: JWT Authorizer using Cognito. Only authenticated users can call the API. The API triggers AWS Lambda. Lambda responsibilities: We integrated Amazon Kinesis Data Streams. Lambda sends events using: Kinesis acts as an event buffer. Now we added security + streaming. Kinesis allows decoupling. This lab demonstrates real production concepts: API Gateway authorizers Producer / Consumer pattern We implemented a secure event-driven serverless architecture. Users authenticate using Cognito and receive a JWT token. API Gateway validates the token before allowing access to the API. The API triggers a Lambda function which generates a file key and sends an event to a Kinesis Data Stream. The stream decouples ingestion from processing and allows scalable event handling. Downstream services consume the events and store the results in S3. This design improves scalability, security, and fault tolerance. This architecture is used in: Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse CODE_BLOCK: Client │ │ Login ▼ Amazon Cognito (Authentication) │ │ JWT Token ▼ Amazon API Gateway (API + JWT validation) │ │ Authorized request ▼ AWS Lambda (API Logic) │ │ PutRecord() ▼ Amazon Kinesis Data Stream (Event pipeline) │ │ Stream processing ▼ Worker / Lambda │ ▼ Amazon S3 (File storage) Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: Client │ │ Login ▼ Amazon Cognito (Authentication) │ │ JWT Token ▼ Amazon API Gateway (API + JWT validation) │ │ Authorized request ▼ AWS Lambda (API Logic) │ │ PutRecord() ▼ Amazon Kinesis Data Stream (Event pipeline) │ │ Stream processing ▼ Worker / Lambda │ ▼ Amazon S3 (File storage) CODE_BLOCK: Client │ │ Login ▼ Amazon Cognito (Authentication) │ │ JWT Token ▼ Amazon API Gateway (API + JWT validation) │ │ Authorized request ▼ AWS Lambda (API Logic) │ │ PutRecord() ▼ Amazon Kinesis Data Stream (Event pipeline) │ │ Stream processing ▼ Worker / Lambda │ ▼ Amazon S3 (File storage) CODE_BLOCK: http://localhost:3000/?code=xxxx Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: http://localhost:3000/?code=xxxx CODE_BLOCK: http://localhost:3000/?code=xxxx COMMAND_BLOCK: curl /oauth2/token Enter fullscreen mode Exit fullscreen mode COMMAND_BLOCK: curl /oauth2/token COMMAND_BLOCK: curl /oauth2/token CODE_BLOCK: id_token access_token refresh_token Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: id_token access_token refresh_token CODE_BLOCK: id_token access_token refresh_token CODE_BLOCK: API request → token validation → allow/deny Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: API request → token validation → allow/deny CODE_BLOCK: API request → token validation → allow/deny COMMAND_BLOCK: curl -X POST API_URL \ -H "Authorization: Bearer ACCESS_TOKEN" Enter fullscreen mode Exit fullscreen mode COMMAND_BLOCK: curl -X POST API_URL \ -H "Authorization: Bearer ACCESS_TOKEN" COMMAND_BLOCK: curl -X POST API_URL \ -H "Authorization: Bearer ACCESS_TOKEN" CODE_BLOCK: { "message": "File uploaded and event sent", "file_key": "uploads/uuid.txt" } Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: { "message": "File uploaded and event sent", "file_key": "uploads/uuid.txt" } CODE_BLOCK: { "message": "File uploaded and event sent", "file_key": "uploads/uuid.txt" } CODE_BLOCK: PutRecord() Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: PutRecord() CODE_BLOCK: PutRecord() CODE_BLOCK: Lambda │ │ PutRecord ▼ Kinesis Stream Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: Lambda │ │ PutRecord ▼ Kinesis Stream CODE_BLOCK: Lambda │ │ PutRecord ▼ Kinesis Stream CODE_BLOCK: uploads/cc501bd4-e4b2-43bb-b0a7-0297fb0930ba.txt Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: uploads/cc501bd4-e4b2-43bb-b0a7-0297fb0930ba.txt CODE_BLOCK: uploads/cc501bd4-e4b2-43bb-b0a7-0297fb0930ba.txt CODE_BLOCK: Client → API Gateway → Lambda → S3 Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: Client → API Gateway → Lambda → S3 CODE_BLOCK: Client → API Gateway → Lambda → S3 CODE_BLOCK: Client ↓ Cognito (Authentication) ↓ API Gateway (Authorization) ↓ Lambda ↓ Kinesis (Event streaming) ↓ Worker processing ↓ S3 Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: Client ↓ Cognito (Authentication) ↓ API Gateway (Authorization) ↓ Lambda ↓ Kinesis (Event streaming) ↓ Worker processing ↓ S3 CODE_BLOCK: Client ↓ Cognito (Authentication) ↓ API Gateway (Authorization) ↓ Lambda ↓ Kinesis (Event streaming) ↓ Worker processing ↓ S3 CODE_BLOCK: API → Direct processing Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: API → Direct processing CODE_BLOCK: API → Direct processing CODE_BLOCK: API → Event Stream → Worker Enter fullscreen mode Exit fullscreen mode CODE_BLOCK: API → Event Stream → Worker CODE_BLOCK: API → Event Stream → Worker - Authentication layer → Cognito - Streaming/event layer → Kinesis - authenticate users securely - accept API requests - process events asynchronously - store files in S3 - scale automatically - Amazon Cognito - User logs in through Cognito Hosted UI - Cognito authenticates the user - Cognito returns a JWT token - receive API request - generate file key - push event to Kinesis - return response - async processing - durable storage - scalability - asynchronous processing - high scalability - fault tolerance - replay capability - image upload pipelines - video processing - IoT ingestion systems - log processing platforms - ML data pipelines