Another E2E Solution delivered. This time with CI/CD, AWS EventBridge and ECS Fargate

To wrap up the year, I built my latest E2E project. It is a side project however it will help us at work. We have a service that uploads documents from a third-party system. This integration requires authentication, but the system enforces a monthly password rotation. When the password expires, uploads and downloads start failing, which quickly turns into a operational issue. To remove the need for manual updates and the risk of someone simply forgetting, I built an automation to handle this end to end. The solution is a Python worker using Selenium with headless Chromium, executed on a schedule and backed by a full CI/CD pipeline. On every push to the main branch, GitHub Actions assumes an AWS IAM Role via OIDC (no access keys involved), builds the Docker image, and pushes it to Amazon ECR. The workflow then registers a new ECS Task Definition revision, updating only the container image. This is the Architecture design solution: Execution is handled by Amazon EventBridge, which triggers the task every 29 days: The task runs on ECS Fargate in a public subnet, with a public IP and outbound traffic allowed. When triggered, Fargate starts the container, runs automation.py, launches Selenium with Chromium and Chromedriver, logs into the system, performs the password rotation, and exits. On success, the task finishes automatically with exit code 0. If an exception occurs, logs are sent to CloudWatch and the error is reported to a Slack alerts channel. Archtecture decisions:
I chose to run the task in a public subnet for simplicity and cost reasons. Since the worker only needs outbound internet access and does not expose any inbound ports, there’s no additional risk as long as the security group has no inbound rules. This also avoids the cost and complexity of running a NAT Gateway, which would be required with private subnets. Using ECS Fargate instead of Lambda was also a deliberate decision. Running Selenium with Chromium on Lambda usually requires custom layers and fine-tuning, and it’s easy to hit limits around memory, package size, or execution time. With Fargate, the entire environment is packaged in the Docker image, with predictable runtime behavior and flexible CPU and memory allocation, which makes this kind of workload much easier to operate. In the end, this is a simple batch worker. It runs on a schedule, does one job, and exits. For headless browser automation, this approach turned out to be more straightforward and reliable. Templates let you quickly answer FAQs or store snippets for re-use. Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink. Hide child comments as well For further actions, you may consider blocking this person and/or reporting abuse