Wire the user-bot Lambda to Slack: bot token in SSM, real DMs, end-to-end test #212

Overview

We need the user-bot Lambda to actually DM new IAM users their temporary console password, so that new members can sign in without a lead handing the password over by hand. #209 built and deployed the bot and a tested SlackMessageSender, but the deployed function is wired to a stub that sends nothing, because no Slack app or bot token existed yet.

Action Items

How it fits together.

• A Slack app's bot token is stored in an SSM Parameter Store SecureString, /user-bot/slack-bot-token, in us-east-1 next to the function.
• Terraform creates the parameter with a placeholder, and the real token is set by hand, so it never appears in git or in Terraform state.
• At cold start the Lambda reads the token and builds SlackMessageSender.
• If the token is missing, unreadable or still the placeholder, the bot refuses before touching the password: it logs an error and throws, and the user's password is unchanged. There is no silent fallback to the stub.

Delivered as two PRs. The order is less critical than in #209: if PR 2 deploys before the token is set, the bot simply refuses.

Before starting: the Slack app (needs a Slack workspace admin)

• At api.slack.com/apps, choose Create New App → From scratch in the Hack for LA workspace. Name it e.g. "AWS User Bot".
• OAuth & Permissions → Bot Token Scopes: add chat:write and nothing else. Posting to a member ID makes Slack open the DM itself, so no other scope is needed.
• App Home: turn on the Messages tab. Without it, users who try to reply see "sending messages to this app has been turned off".
• Install to Workspace, then keep the Bot User OAuth Token (xoxb-…) for the step after PR 1. Do not paste it into this issue, a PR, or Slack.

PR 1: Terraform

• In terraform/user-bot.tf, add an aws_ssm_parameter with:

  • region = local.user_bot_region, name = "/user-bot/slack-bot-token", type = "SecureString" (default aws/ssm key)
  • value_wo = "placeholder-set-by-hand" and value_wo_version = 1
  • lifecycle { prevent_destroy = true }

  Why value_wo and not value: with value, the provider reads the decrypted parameter back into Terraform state on every refresh, so the token would sit in the state bucket. With the write-only value_wo, the provider's read sets value to null (resourceParameterRead in provider v6.64.0), and the value is only written when value_wo_version changes.

  Put both of these traps in a comment above the resource:

  • Never change value_wo_version. That rewrites the placeholder over the real token.
  • Do not change other arguments such as description in place. The provider then sends an empty value and the apply fails.

• Execution role (aws_iam_role_policy.user_bot): add ssm:GetParameter on this one parameter's ARN.

  • No kms:Decrypt statement should be needed: the AWS-managed aws/ssm key's key policy allows decryption through SSM for principals in the account.
  • Confirm that in the end-to-end test below rather than assuming it.

• Function (aws_lambda_function.user_bot): add environment { variables = { SLACK_TOKEN_PARAMETER = <the parameter's name> } }. The function's ignore_changes covers only its code, so Terraform applies this normally.

• Regenerate terraform/README.md with terraform-docs v0.20.0 (terraform-docs -c .terraform.docs.yml . in terraform/). Keep the existing | ---- | separator style so the diff shows only the new rows.

After PR 1 merges: set the token

• After PR 1 merges and Terraform Apply succeeds, store the real token. Keep it out of shell history by reading it from a prompt:

  read -rs SLACK_TOKEN && MSYS_NO_PATHCONV=1 aws ssm put-parameter --region us-east-1 \
    --name /user-bot/slack-bot-token --type SecureString --overwrite --value "$SLACK_TOKEN"; unset SLACK_TOKEN

  MSYS_NO_PATHCONV=1 is only needed in Git Bash on Windows, which otherwise rewrites /user-bot/... into a Windows path.
• Confirm it took without decrypting: aws ssm get-parameter --region us-east-1 --name /user-bot/slack-bot-token --query Parameter.Version should be 2.
• Confirm Terraform did not notice: a fresh terraform plan shows no change to the parameter.

PR 2: the Lambda

• Add @aws-sdk/client-ssm. In a new src/slack-token.ts, read the parameter named by SLACK_TOKEN_PARAMETER with WithDecryption: true.
  • Cache it in module scope for the life of the execution environment, but clear the cache on failure so the next invocation retries instead of reusing a rejected promise.
  • Treat these as failures:
    • env var unset
    • parameter missing
    • access denied
    • a value that does not start with xoxb-, which catches the placeholder
• In src/handler.ts, replace sender: MessageSender in Dependencies with getSender: () => Promise<MessageSender>.
  • Call it after the tag checks and before generatePassword / UpdateLoginProfile.
  • If it throws: log an error with the user name (never the token), throw, and leave the password untouched.
  • This ordering is the whole point. Resolving the token at send time would mean a reset password that nobody receives.
• In src/index.ts, wire getSender to build a SlackMessageSender from the token.
  • StubMessageSender stays in the codebase for local runs and tests but is no longer deployed. Its comment and the README's "Not sending DMs yet" note should say so.
• Unit tests (SSM mocked with aws-sdk-client-mock). Cover at least:
  • token read and cached across two invocations: one GetParameter call
  • cache cleared after a failure, so the next call retries
  • env var unset
  • ParameterNotFound
  • AccessDeniedException
  • placeholder value
  • for each failure: the handler throws, UpdateLoginProfile is never called, nothing is sent, and the token appears in no log or thrown error
• Update lambda/user-bot/README.md: where the token lives, how to rotate it (the same put-parameter command), and the refuse-before-reset behaviour.

After both PRs merge: end-to-end test

This replaces #209's last two action items, which were planned against the stub.

• After PR 2 merges, confirm Deploy user-bot Lambda ran green.
• Open a PR adding two throwaway users to terraform/aws-users.tf, and merge it:
  • user-bot-test with slack_id set to your own member ID
  • user-bot-test-noslack with no slack_id
• Confirm the DM arrived from the app, containing:
  • the sign-in page
  • user-bot-test
  • a temporary password, which signs in at the sign-in page and immediately forces a password change
• In /aws/lambda/user-bot (us-east-1), confirm:
  • user-bot-test was handed to the Slack sender with no errors
  • user-bot-test-noslack was skipped with reason no-slack-id
  • the password appears nowhere in the log
• Remove both throwaway users in a follow-up PR, and confirm Terraform Apply deleted them.

Resources/Instructions

• The bot and its design: Add a user-bot Lambda that DMs new IAM users a temporary console password #209, lambda/user-bot/README.md, terraform/user-bot.tf
• Where the sender plugs in: lambda/user-bot/src/index.ts, src/handler.ts, src/senders/slack.ts
• Terraform aws_ssm_parameter, value_wo and write-only arguments
• Slack: creating an app and bot token scopes, chat.postMessage
• Parameter Store SecureString and the aws/ssm key
• Related, not duplicates: Define Automations for Slack operations devops#67 (a Slack app for backend monitoring alerts, using incoming webhooks); Slack app for monitoring backend services devops#1 (closed: requirements for a Slack bot platform)