--- title: "Things I have automated at work" description: "Seven automations built for a security team: firewall reviews, AMI patch tickets, Slack bots, document permission checks and a tagging compliance CLI." date: 2025-02-15 tags: ["AWS","CDK","TypeScript","Python","Golang","Wiz","Jupyter","Zapier","SOC 2"] url: https://adampie.dev/things-i-have-automated-at-work/ --- # Things I have automated at work --- ## Reviewing firewall rules across AWS, GCP and Azure __Tools:__ AWS CDK, Wiz, AWS, Google Drive, Slack __Languages:__ TypeScript, Python To ensure SOC 2 compliance, we conduct multiple firewall reviews throughout the year. Automation has increased the frequency of these reviews and reduced the time required from a full day to a maximum of one hour. Wiz exports firewall rules from AWS, GCP, and Azure to AWS S3. Lambdas process the raw file and compare it with previous reports to generate a diff of rules, writing back to S3. A different lambda copies that file to Google Drive for easy collaboration. A Slack message is then also sent to both the Security team and the Security Champions for review. ### Process ```mermaid sequenceDiagram Wiz->>S3: Export S3<<->>Lambda: Process S3<<->>Lambda: Diff Lambda->>Google Drive: Upload Lambda-->>Slack: Send notification ``` ### Example ![Firewall Review](FirewallReview.png) --- ## Closing tickets referencing AMIs that need patching that are no longer used __Tools:__ AWS CDK, Wiz, AWS, Jira __Languages:__ TypeScript, Python EKS and Karpenter allow nodes to have short lifecycles, which is great for new AMIs. The vulnerability management program creates tickets to monitor AMIs with critical or high CVEs, and using a newer AMI can resolve many of those issues, but some require manual intervention. Some tickets may remain “in progress” despite no longer using the AMI. This automation gets all the Jira Issues part of the VM Program containing AMIs and checks against Wiz to see if it’s still in use. If it’s no longer found in the current snapshot of resources, the ticket is closed and a Slack message sent. This saves us a bit of time and effort as it stops us from following up on AMIs we no longer use. ### Process ```mermaid sequenceDiagram Lambda<<->>Jira: Get issues Lambda<<->>Wiz: Check AMIs Lambda->>Jira: Resolve issue Lambda-->>Slack: Send notification ``` ### Example ![AMI Checker](AMIChecker.png) --- ## A Slackbot that tags the relevant team on new messages in the Security channel __Tools:__ AWS CDK, Jupyter, Zapier, Slack, AWS __Languages:__ TypeScript, Python This is my favourite thing I’ve automated so far. Our Security Org has a few teams, with people spread across multiple countries and time-zones. We use a shared Slack channel for company-wide messaging, but we sometimes miss an odd message or two. To prevent that from happening, I had the idea of having a bot tag the relevant team on incoming messages. It took me a few weeks to review the past two years of Slack messages in the security channel. I added a “classify” reaction and stored the message and the relevant team in a sheet using a Zap. Once I had enough data, I started researching on how to classify text and came across [scikit-learn](https://scikit-learn.org/stable/). A Jupiter notebook and multiple tutorials later, I had something that somewhat resembled what I was after. I then set up a Zap to trigger off a new Slack message, which would make a POST request to my API endpoint to classify the message. To not cause any notification fatigue from misclassifications, I set it to not tag the team on confidence ratings below 60%. Every so often, I add more data or try new things with scikit-learn and end up AB testing the new model against the current one. ### Process Labelling of data ```mermaid sequenceDiagram Slack->>Zapier: "Classify" reaction added to message Zapier->>Zapier: Store message and team in Tables ``` Tagging team on new messages ```mermaid sequenceDiagram Slack->>Zapier: New message in Security channel Zapier<<->>AWS: POST request to model Zapier->>Slack: Tag team ``` ### Example ![CCBot](CCBot.png) --- ## Checking permissions of shared documents shared in Slack __Tools:__ Zapier, Slack __Languages:__ Python One thing I see people do is scope internal documents to “Anyone with the link” rather than “Anyone in the company”. These links can often end up in places they shouldn’t, so I built a Zap to check the permissions of the document when a link is posted in Slack. The Zap filters links matching our “document management system,” and then, using ‘Code by Zapier,’ it makes a GET request to the URL to check for a HTTP 200 (public) or HTTP 302 (redirects to a login page) response. ### Process ```mermaid sequenceDiagram Slack->>Zapier: New message containing "tool" URL Zapier->>Tool: GET request to URL Tool-->>Zapier: HTTP 200 - NO AUTH Tool-->>Zapier: HTTP 302 - AUTH Zapier->>Slack: Notification for "NO AUTH" ``` ### Example ![Public Document](PublicDocument.png) --- ## A Slackbot for GitLab Merge Requests including their statuses __Tools:__ AWS CDK, GitLab, Slack __Languages:__ Golang, TypeScript Each morning, I review Wiz and review any code review requests assigned to me overnight (we’re a distributed team). To see what interesting things other teams in my Zone are doing, I built a Slackbot that posts a message to a channel when a new merge request is opened. The bot also adds reactions to the message depending on whether the merge request is approved, merged, or closed. As it catches all Merge Request events from GitLab, I ended up setting up multiple feed channels for: * Security * SRE * Data ### Process ```mermaid sequenceDiagram GitLab->>AWS: Merge Request Event Note over AWS,Slack: MR Opened AWS->>Slack: New Slack message Note over AWS,Slack: MR Approved AWS->>Slack: "Approved" reaction added Note over AWS,Slack: MR Merged AWS->>Slack: All reactions removed AWS->>Slack: "Merged" reaction removed Note over AWS,Slack: MR Closed AWS->>Slack: All reactions removed AWS->>Slack: "Closed" reaction removed ``` ### Example ![MR Approved](MRApproved.png) ![MR Merged](MRMerged.png) --- ## A CLI tool to check tagging and labelling compliance __Languages:__ Golang This was a tool I built when testing Cursor. I wanted to see how far I could get with writing as little code as possible. It’s quite a useful tool that I run from time to time to check if we’ve missed any tags or labels in our code. It supports Terraform, Helm and Dockerfiles, which covers most of our use-cases. ### Example ![Check Terraform](CheckTerraform.png) ![Check Helm](CheckHelm.png) ![Check Dockerfile](CheckDockerfile.png) --- ## Reacting to a Slack message to create a ticket __Tools:__ Zapier, Slack This is something most teams have set up at Zapier. It’s a simple Zap which triggers off a Slack reaction to a message and creates a Jira ticket. It saves us a bit of time and effort, as we can pre-configure the random required fields beforehand. ### Process ```mermaid sequenceDiagram Slack->>Zapier: Reaction added Zapier<<->>Jira: Create Issue Zapier->>Slack: Comment Issue ``` ### Example ![Slack to Jira](SlackToJira.png)