I use Terraform and ECS, so it's probably a bit different, but perhaps ChatGPT can help you translate. I store the database url in a AWS Secrets Manager secret and reference it like this:
"secrets": [
{
"name": "DATABASE_URL",
"valueFrom": "${aws_secretsmanager_secret_version.ndap_1.arn}:WINDMILL_DATABASE_URL::"
}
],