diff --git a/backlog.md b/backlog.md index ecffa5a..81d0964 100644 --- a/backlog.md +++ b/backlog.md @@ -49,6 +49,15 @@ 21. Спрятать сеть за балансировщиком нагрузки 22. Автопродление TLS падает при перезагрузке nginx; сертификат действует до 14.10.2026. (Исправить reload внутри контейнера и проверить systemctl start an-chat-ssl-renew.service до успешного завершения.) 23. WireGuard-only SSH. +25. Перенести секреты из .env в KMS Selectel. +26. Запрет входа под root: В /etc/ssh/sshd_config установите PermitRootLogin no. Заходите под обычным пользователем (например, deploy) и используйте sudo для админских задач. +27. Удалите все ненужные пакеты, компиляторы (gcc, make) и сервисы. Чем меньше программ на сервере, тем меньше потенциальных уязвимостей. +28. Монтирование с флагами безопасности: Разделы диска (особенно /tmp и /var/tmp) следует монтировать с флагами noexec (запрет запуска исполняемых файлов) и nosuid (игнорирование битов setuid). +29. Systemd-ограничения: используйте директивы в юните + NoNewPrivileges=yes # Запрещает повышение привилегий через setuid + ProtectSystem=strict # Делает всю ОС доступной только для чтения + PrivateTmp=yes # Дает процессу свой изолированный /tmp + ProtectHome=yes # Скрывает домашние директории пользователей 24. Добавить логи (Для Python-сервисов добавить OTLP Log Exporter: api-backend; sms-service; sms-worker. Подключить LoggerProvider, BatchLogRecordProcessor и bounded queue. Передавать resource attributes: service.name; service.version; deployment.environment; service.namespace=han-chat.) Экспортировать структурированные поля request_id, trace_id, span_id, severity и event name. Оставить stdout как аварийный локальный журнал. Добавить canary-тесты, запрещающие экспорт токенов, cookie, телефонов, email, текстов сообщений, SQL и object keys.). 25. Nginx metrics/tracing в signoz diff --git a/codebase/backend/.env.example b/codebase/backend/.env.example index 25f599d..76ebed9 100644 --- a/codebase/backend/.env.example +++ b/codebase/backend/.env.example @@ -1,4 +1,7 @@ -# Copy to .env, replace every placeholder, then run: ./scripts/validate-env .env +# Non-secret deployment configuration only. +# Copy to .env, replace placeholders, then run: ./scripts/validate-env .env +# Secrets are supplied at runtime by deployment/secrets/han-secrets. +SECRETS_SOURCE=file APP_ENV=production-like RELEASE_VERSION=change-me-release LOG_LEVEL=INFO @@ -12,22 +15,15 @@ BITRIX_SYNC_IMAGE=han-chat-bitrix-sync:local KEYCLOAK_IMAGE=han-chat-keycloak:local SMS_SERVICE_IMAGE=han-chat-sms-service:local -# Managed PostgreSQL is external to Compose. All production DSNs must verify TLS. +# Managed PostgreSQL is external to Compose. Credential-bearing DSNs are secrets. HAN_PG_HOST=managed-pg.private.example HAN_PG_PORT=5433 HAN_PG_DATABASE=han_chat PG_CA_HOST_PATH=/opt/han-chat/secrets/pg/ca.pem -# PgBouncer session mode uses each role's database-level search_path; do not pass libpq options. -DATABASE_URL=postgresql+asyncpg://han_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -BITRIX_DATABASE_URL=postgresql://bitrix_local_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -BITRIX_SYNC_APP_DATABASE_URL=postgresql://bitrix_sync_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -BITRIX_SYNC_DATABASE_URL=postgresql://bitrix_sync_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -MESSAGE_SAFETY_DATABASE_URL=postgresql://message_safety_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -SMS_DATABASE_URL=postgresql+asyncpg://sms_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem +# PgBouncer session mode uses database-level search_path; runtime DSNs must not pass options/currentSchema. KEYCLOAK_DB_URL=jdbc:postgresql://managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem KEYCLOAK_DB_SCHEMA=keycloak KEYCLOAK_DB_USERNAME=keycloak_user -KEYCLOAK_DB_PASSWORD=change-me PUBLIC_HOST=chat.example.ru PUBLIC_WEB_URL=https://chat.example.ru @@ -70,61 +66,21 @@ KEYCLOAK_REALM=han-chat KEYCLOAK_AUDIENCE=han-chat-api # На мок среде true,true. На продакшн false,false. KEYCLOAK_OTP_MOCK_ENABLED=true -KEYCLOAK_OTP_MOCK_CODE=change-me KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false -# Обязательны только при KEYCLOAK_YANDEX_CAPTCHA_ENABLED=true. +# Public site key; the server key stays in Secrets Manager when CAPTCHA is enabled. KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY= -KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY= -# (openssl rand -hex 32) -KEYCLOAK_OTP_HMAC_KEY=change-me KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC=300 KEYCLOAK_SETTINGS_BRIDGE_URL=http://api-backend:8000/internal/settings/v1/otp KEYCLOAK_SMS_SERVICE_URL=http://sms-service:8080 -# Должен совпадать с SMS_SERVICE_TOKEN. -KEYCLOAK_SMS_SERVICE_TOKEN=change-me KEYCLOAK_ADMIN=bootstrap-admin -# (openssl rand -hex 32) -KEYCLOAK_ADMIN_PASSWORD=change-me -# (openssl rand -hex 32) -CURSOR_HMAC_SECRET=change-me - -# Redis пароль 1 (openssl rand -hex 32) -REDIS_API_PASSWORD=change-me -REDIS_URL=redis://api_backend:change-me@redis:6379/0 -REDIS_REALTIME_URL=redis://api_backend:change-me@redis:6379/1 -# Redis пароль 2 (openssl rand -hex 32) -REDIS_SAFETY_PASSWORD=change-me -MESSAGE_SAFETY_REDIS_URL=redis://message_safety:change-me@redis:6379/2 -# Redis пароль 3 (openssl rand -hex 32) -REDIS_HEALTH_PASSWORD=change-me REDIS_MAXMEMORY=384mb REDIS_EVICTION_POLICY=volatile-lru -#token1 (openssl rand -hex 32) -MESSAGE_SAFETY_SERVICE_TOKEN=change-me -#token2 (openssl rand -hex 32) -BITRIX_LOCAL_APP_INTERNAL_TOKEN=change-me -BITRIX_INTERNAL_API_TOKEN=change-me -#token3 (openssl rand -hex 32) -BITRIX_API_FORWARD_TOKEN=change-me -BITRIX_API_INBOX_TOKEN=change-me -#token4 (openssl rand -hex 32) -BITRIX_SYNC_SERVICE_TOKEN=change-me -#token5 (openssl rand -hex 32) -KEYCLOAK_SETTINGS_BRIDGE_TOKEN=change-me -#token6 (openssl rand -hex 32), должен совпадать с KEYCLOAK_SMS_SERVICE_TOKEN -SMS_SERVICE_TOKEN=change-me -# Тестовый продюсер Internal Notifications API. В БД хранится только hash. -NOTIFICATIONS_TOKEN_PRODUCER_TEST=change-me - -# i-Digital Direct. Перед production заменить placeholders согласованными значениями. +# i-Digital Direct non-secret configuration. IDGTL_SMS_BASE_URL=https://direct.i-dgtl.ru -IDGTL_SMS_API_KEY=change-me IDGTL_SMS_CALLBACK_PUBLIC_URL=https://chat.example.ru/callbacks/idgtl/sms -IDGTL_SMS_CALLBACK_USERNAME=change-me -IDGTL_SMS_CALLBACK_PASSWORD=change-me BITRIX_LOCAL_APP_BASE_URL=http://bitrix-local-app:8080 BITRIX_API_INBOX_PATH=/internal/openlines/v1/inbox @@ -139,28 +95,17 @@ MESSAGE_SAFETY_RULES_VERSION=2026-01-01 #Отключение синхронизации (при отключенной синхронизации параметры ниже не работают) BITRIX_SYNC_ENABLED=false BITRIX_SYNC_CRM_BASE_URL=https://example.bitrix24.ru -BITRIX_SYNC_CRM_WEBHOOK_URL=change-me BITRIX_SYNC_CONTACT_MAP_INTERVAL_SEC=60 BITRIX_SYNC_CONTACT_UPDATE_INTERVAL_SEC=30 BITRIX_SYNC_CRM_MAX_CONCURRENCY=2 BITRIX_SYNC_CONTACT_LIST_BATCH_SIZE=50 -# (openssl rand -hex 32) -BITRIX_SYNC_WEBHOOK_TOKEN=change-me - -# Сначала регистрируется приложение в Битрикс24, потом тут прописываются секреты -BITRIX_CLIENT_ID=change-me -BITRIX_CLIENT_SECRET=change-me -BITRIX_APPLICATION_TOKEN=change-me - +BITRIX_CLIENT_ID=change-me-client-id BITRIX_CONNECTOR_ID=han_mobile_app BITRIX_CONNECTOR_NAME=HAN Mobile App BITRIX_OPEN_LINE_ID= BITRIX_EXPECTED_DOMAIN=<ваш-портал>.bitrix24.ru BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix -# ENCRYPTION_KEY обязательно через base64 -# Generate with: python -c "import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())" -BITRIX_TOKEN_ENCRYPTION_KEY=change-me BITRIX_HTTP_TIMEOUT_SEC=10 BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC=20 @@ -168,10 +113,6 @@ SELECTEL_S3_ENDPOINT_URL=https://s3.ru-7.storage.selcloud.ru SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine -SELECTEL_S3_ACCESS_KEY=change-me -SELECTEL_S3_SECRET_KEY=change-me -SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY=change-me -SELECTEL_S3_QUARANTINE_READ_SECRET_KEY=change-me OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317 OTEL_SERVICE_NAME_API=api-backend @@ -181,7 +122,6 @@ SMS_METRICS_PORT=9464 #Если есть внешний OTLP-сервис, замените (Точный формат авторизации зависит от провайдера): #Если внешнего OTLP-сервиса пока нет, otlp.example.invalid:4317 можно временно оставить, но Collector будет постоянно пытаться подключиться, писать предупреждения и накапливать очередь. OTEL_REMOTE_ENDPOINT=otlp.example.invalid:4317 -OTEL_REMOTE_AUTH_HEADER=change-me # true только для plaintext OTLP внутри доверенной приватной сети (например self-hosted SigNoz). OTEL_REMOTE_TLS_INSECURE=false # SDK отправляет все spans локальному Collector; решение о хранении принимает tail_sampling. diff --git a/codebase/backend/api-backend/Dockerfile b/codebase/backend/api-backend/Dockerfile index d0a95d8..0cf92f7 100644 --- a/codebase/backend/api-backend/Dockerfile +++ b/codebase/backend/api-backend/Dockerfile @@ -13,7 +13,8 @@ COPY --from=builder /build/dist/*.whl /tmp/ RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl COPY alembic.ini ./ COPY alembic ./alembic +COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint USER 10001:10001 EXPOSE 8000 -ENTRYPOINT ["uvicorn"] -CMD ["app.main:app", "--host", "0.0.0.0", "--port", "8000", "--no-proxy-headers"] +ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"] +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--no-proxy-headers"] diff --git a/codebase/backend/api-backend/alembic/env.py b/codebase/backend/api-backend/alembic/env.py index 6eb1636..d101cbb 100644 --- a/codebase/backend/api-backend/alembic/env.py +++ b/codebase/backend/api-backend/alembic/env.py @@ -1,4 +1,5 @@ import asyncio +import os from logging.config import fileConfig from sqlalchemy import pool @@ -6,12 +7,11 @@ from sqlalchemy import pool from alembic import context from app.db import Base from app.postgres import create_postgres_engine -from app.settings import get_settings config = context.config if config.config_file_name: fileConfig(config.config_file_name) -database_url = get_settings().database_url +database_url = os.environ["DATABASE_URL"] config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%")) target_metadata = Base.metadata diff --git a/codebase/backend/api-backend/app/logging_security.py b/codebase/backend/api-backend/app/logging_security.py new file mode 100644 index 0000000..db055fb --- /dev/null +++ b/codebase/backend/api-backend/app/logging_security.py @@ -0,0 +1,47 @@ +from __future__ import annotations + +import re +from collections.abc import Mapping +from typing import Any + +REDACTED = "[REDACTED]" +_SENSITIVE_KEY = re.compile( + r"(authorization|cookie|password|passwd|secret|token|api[_-]?key|" + r"database[_-]?url|redis[_-]?url|dsn|callback[_-]?url)", + re.IGNORECASE, +) +_URI_USERINFO = re.compile(r"(?P[a-z][a-z0-9+.-]*://)[^/@\s]+@", re.IGNORECASE) +_QUERY_SECRET = re.compile( + r"(?P[?&](?:token|access_token|api_key|key|secret|password)=)[^&#\s]+", + re.IGNORECASE, +) +_AUTH_VALUE = re.compile(r"\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+", re.IGNORECASE) + + +def sanitize_text(value: str) -> str: + value = _URI_USERINFO.sub(r"\g[REDACTED]@", value) + value = _QUERY_SECRET.sub(r"\g[REDACTED]", value) + return _AUTH_VALUE.sub(r"\1 [REDACTED]", value) + + +def sanitize_value(value: Any) -> Any: + if isinstance(value, str): + return sanitize_text(value) + if isinstance(value, Mapping): + return { + str(key): REDACTED if _SENSITIVE_KEY.search(str(key)) else sanitize_value(item) + for key, item in value.items() + } + if isinstance(value, list): + return [sanitize_value(item) for item in value] + if isinstance(value, tuple): + return tuple(sanitize_value(item) for item in value) + return value + + +def redact_event( + _logger: Any, + _method_name: str, + event_dict: dict[str, Any], +) -> dict[str, Any]: + return sanitize_value(event_dict) diff --git a/codebase/backend/api-backend/app/main.py b/codebase/backend/api-backend/app/main.py index 9a1843d..5609b1b 100644 --- a/codebase/backend/api-backend/app/main.py +++ b/codebase/backend/api-backend/app/main.py @@ -50,6 +50,7 @@ from app.integrations import ( S3Client, SafetyClient, ) +from app.logging_security import redact_event from app.metrics import AUTH_BOOTSTRAP, HTTP_DURATION, HTTP_REQUESTS, RATE_LIMIT_DECISIONS from app.notification_routes import router as notification_router from app.notification_service import synchronize_source_tokens @@ -96,6 +97,7 @@ def configure_logging(level: str) -> None: processors=[ structlog.contextvars.merge_contextvars, add_trace_context, + redact_event, structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"), structlog.stdlib.add_log_level, structlog.processors.JSONRenderer(), @@ -342,7 +344,11 @@ async def http_error(request: Request, exc: StarletteHTTPException): @app.exception_handler(Exception) async def unhandled_error(request: Request, exc: Exception): - log.exception("request.failed", error_code="internal_error") + log.error( + "request.failed", + error_code="internal_error", + error_type=type(exc).__name__, + ) return error_response(request, "internal_error", "Internal server error", 500) diff --git a/codebase/backend/api-backend/container-entrypoint.sh b/codebase/backend/api-backend/container-entrypoint.sh new file mode 100644 index 0000000..7e6f3ff --- /dev/null +++ b/codebase/backend/api-backend/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec "$@" diff --git a/codebase/backend/api-backend/docker-compose.yml b/codebase/backend/api-backend/docker-compose.yml index 40b73e2..98562bc 100644 --- a/codebase/backend/api-backend/docker-compose.yml +++ b/codebase/backend/api-backend/docker-compose.yml @@ -5,39 +5,54 @@ services: expose: - "8000" environment: + HAN_SECRET_VARS: >- + DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN + BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN + KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY + CURSOR_HMAC_SECRET + DATABASE_URL_FILE: /run/secrets/api_database_url + REDIS_URL_FILE: /run/secrets/api_redis_url + REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url + MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token + BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token + BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token + KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token + SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key + SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key + CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret APP_ENV: ${APP_ENV} API_PORT: ${API_PORT:-8000} LOG_LEVEL: ${LOG_LEVEL:-INFO} - DATABASE_URL: ${DATABASE_URL} - REDIS_URL: ${REDIS_URL} - REDIS_REALTIME_URL: ${REDIS_REALTIME_URL} KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL} KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL} KEYCLOAK_REALM: ${KEYCLOAK_REALM} KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE} MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL} - MESSAGE_SAFETY_SERVICE_TOKEN: ${MESSAGE_SAFETY_SERVICE_TOKEN} MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5} MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2} MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300} MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD: ${MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD:-5} MESSAGE_SAFETY_CIRCUIT_OPEN_SEC: ${MESSAGE_SAFETY_CIRCUIT_OPEN_SEC:-30} BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL} - BITRIX_LOCAL_APP_INTERNAL_TOKEN: ${BITRIX_LOCAL_APP_INTERNAL_TOKEN} - BITRIX_API_INBOX_TOKEN: ${BITRIX_API_INBOX_TOKEN} BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20} BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD: ${BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD:-5} BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC: ${BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC:-30} - KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN} SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL} SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS} SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS} SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE} - SELECTEL_S3_ACCESS_KEY: ${SELECTEL_S3_ACCESS_KEY} - SELECTEL_S3_SECRET_KEY: ${SELECTEL_S3_SECRET_KEY} OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT} - CURSOR_HMAC_SECRET: ${CURSOR_HMAC_SECRET} - NOTIFICATIONS_TOKEN_PRODUCER_TEST: ${NOTIFICATIONS_TOKEN_PRODUCER_TEST} + secrets: + - api_database_url + - api_redis_url + - api_redis_realtime_url + - message_safety_service_token + - bitrix_local_app_internal_token + - bitrix_api_inbox_token + - keycloak_settings_bridge_token + - selectel_s3_access_key + - selectel_s3_secret_key + - cursor_hmac_secret healthcheck: test: - CMD @@ -48,6 +63,8 @@ services: timeout: 3s retries: 5 restart: unless-stopped + ulimits: + core: {soft: 0, hard: 0} networks: - backend - observability @@ -55,3 +72,25 @@ services: networks: backend: observability: + +secrets: + api_database_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL + api_redis_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_URL + api_redis_realtime_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_REALTIME_URL + message_safety_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_SERVICE_TOKEN + bitrix_local_app_internal_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_LOCAL_APP_INTERNAL_TOKEN + bitrix_api_inbox_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_INBOX_TOKEN + keycloak_settings_bridge_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN + selectel_s3_access_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_ACCESS_KEY + selectel_s3_secret_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_SECRET_KEY + cursor_hmac_secret: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/CURSOR_HMAC_SECRET diff --git a/codebase/backend/api-backend/tests/unit/test_logging_security.py b/codebase/backend/api-backend/tests/unit/test_logging_security.py new file mode 100644 index 0000000..99e4aae --- /dev/null +++ b/codebase/backend/api-backend/tests/unit/test_logging_security.py @@ -0,0 +1,31 @@ +from app.logging_security import REDACTED, redact_event, sanitize_text + + +def test_redacts_sensitive_fields_recursively() -> None: + event = { + "authorization": "Bearer top-secret", + "nested": { + "database_url": "postgresql://user:password@db/app", + "safe": "kept", + }, + } + + redacted = redact_event(None, "info", event) + + assert redacted["authorization"] == REDACTED + assert redacted["nested"]["database_url"] == REDACTED + assert redacted["nested"]["safe"] == "kept" + + +def test_redacts_credentials_embedded_in_text() -> None: + value = ( + "POST https://callback-user:callback-password@example.test/cb" + "?token=query-secret Authorization=Bearer header-secret" + ) + + redacted = sanitize_text(value) + + assert "callback-password" not in redacted + assert "query-secret" not in redacted + assert "header-secret" not in redacted + assert redacted.count(REDACTED) == 3 diff --git a/codebase/backend/bitrix-local-app/Dockerfile b/codebase/backend/bitrix-local-app/Dockerfile index 899ee5a..1d39317 100644 --- a/codebase/backend/bitrix-local-app/Dockerfile +++ b/codebase/backend/bitrix-local-app/Dockerfile @@ -6,7 +6,9 @@ COPY app ./app COPY alembic ./alembic COPY alembic.ini pyproject.toml ./ RUN pip install --no-cache-dir . +COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint USER app EXPOSE 8080 HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)" +ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"] diff --git a/codebase/backend/bitrix-local-app/app/main.py b/codebase/backend/bitrix-local-app/app/main.py index c973dcb..1121720 100644 --- a/codebase/backend/bitrix-local-app/app/main.py +++ b/codebase/backend/bitrix-local-app/app/main.py @@ -766,7 +766,7 @@ def create_app(settings: Settings | None = None) -> FastAPI: timeout=cfg.bitrix_http_timeout_sec, ) except Exception as exc: - logger.exception( + logger.error( "outbound delivery failed", extra={ "request_id": request.state.request_id, @@ -1153,8 +1153,11 @@ async def worker_loop(app: FastAPI, kind: str) -> None: await process_outbound(app) else: await process_setup(app) - except Exception: - logger.exception("worker iteration failed", extra={"worker_kind": kind}) + except Exception as exc: + logger.error( + "worker iteration failed", + extra={"worker_kind": kind, "error_type": type(exc).__name__}, + ) try: await asyncio.wait_for(app.state.stop.wait(), app.state.settings.bitrix_worker_poll_sec) except TimeoutError: @@ -1197,7 +1200,7 @@ async def process_inbox(app: FastAPI) -> None: ) await session.commit() except Exception as exc: - logger.exception( + logger.error( "inbound forward failed", extra={ "inbox_id": str(row.id), @@ -1275,7 +1278,7 @@ async def process_outbound(app: FastAPI) -> None: timeout=app.state.settings.bitrix_http_timeout_sec, ) except Exception as exc: - logger.exception( + logger.error( "outbound retry failed", extra={ "outbound_id": str(row.id), diff --git a/codebase/backend/bitrix-local-app/container-entrypoint.sh b/codebase/backend/bitrix-local-app/container-entrypoint.sh new file mode 100644 index 0000000..7e6f3ff --- /dev/null +++ b/codebase/backend/bitrix-local-app/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec "$@" diff --git a/codebase/backend/bitrix-sync/Dockerfile b/codebase/backend/bitrix-sync/Dockerfile index 899ee5a..1d39317 100644 --- a/codebase/backend/bitrix-sync/Dockerfile +++ b/codebase/backend/bitrix-sync/Dockerfile @@ -6,7 +6,9 @@ COPY app ./app COPY alembic ./alembic COPY alembic.ini pyproject.toml ./ RUN pip install --no-cache-dir . +COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint USER app EXPOSE 8080 HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)" +ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"] diff --git a/codebase/backend/bitrix-sync/container-entrypoint.sh b/codebase/backend/bitrix-sync/container-entrypoint.sh new file mode 100644 index 0000000..7e6f3ff --- /dev/null +++ b/codebase/backend/bitrix-sync/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec "$@" diff --git a/codebase/backend/deployment/DEPLOYMENT_GUIDE.ru.md b/codebase/backend/deployment/DEPLOYMENT_GUIDE.ru.md index 5f6a41e..2b9569d 100644 --- a/codebase/backend/deployment/DEPLOYMENT_GUIDE.ru.md +++ b/codebase/backend/deployment/DEPLOYMENT_GUIDE.ru.md @@ -262,7 +262,7 @@ han-chat-documents ## 8. Создание файла окружения -На VM: +`.env` содержит только несекретную конфигурацию. На VM: ```sh cd /opt/han-chat/backend @@ -272,7 +272,18 @@ chmod 600 .env nano .env ``` -Замените все `change-me` и адреса `example.*`. +Замените несекретные адреса `example.*`. Не добавляйте в `.env` пароли, токены, +ключи, credential-bearing DSN или пути `*_FILE`. Установите отдельно проверенный +launcher `deployment/secrets/han-secrets` из ops-пакета. Его интерфейс: + +```sh +deployment/secrets/han-secrets run --config .env -- +``` + +Launcher читает `SECRETS_SOURCE=file|selectel`, устанавливает +`HAN_SECRETS_ACTIVE=1`, выдаёт значения только дочернему процессу и не печатает +их. Рекомендуемый `HAN_RUNTIME_SECRET_MANIFEST` содержит только пары +`SECRET_KEY=/absolute/protected/path`; файлы имеют mode `0400`/`0600`. ### 8.1. Основные адреса @@ -295,7 +306,7 @@ KEYCLOAK_AUDIENCE=han-chat-api ### 8.2. PostgreSQL -Укажите фактические host, port, database, пользователей и пароли: +В `.env` укажите только host, port, database и путь к публичному CA: ```dotenv HAN_PG_HOST= @@ -303,17 +314,13 @@ HAN_PG_PORT=6432 HAN_PG_DATABASE=han_chat PG_CA_HOST_PATH=/opt/han-chat/backend/secrets/pg/ca.pem -DATABASE_URL=postgresql+asyncpg://han_app:@:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -BITRIX_DATABASE_URL=postgresql://bitrix_local_app:@:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -BITRIX_SYNC_DATABASE_URL=postgresql://bitrix_sync_user:@:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem -MESSAGE_SAFETY_DATABASE_URL=postgresql://message_safety_app:@:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem - -KEYCLOAK_DB_URL=jdbc:postgresql://:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem KEYCLOAK_DB_SCHEMA=keycloak -KEYCLOAK_DB_USERNAME=keycloak_user -KEYCLOAK_DB_PASSWORD= ``` +Все service DSN, включая JDBC и backup DSN, формирует secret backend. Runtime +validator проверяет `verify-full`, `sslrootcert` и запрет `options/currentSchema` +без вывода строк подключения. + Порт `5433` используется с PgBouncer в режиме `session`. Не добавляйте `options=-csearch_path...` или JDBC-параметр `currentSchema`: они передают startup parameter `search_path`, который Selectel PgBouncer отклоняет. Для @@ -323,9 +330,12 @@ Keycloak схема задаётся отдельно через `KEYCLOAK_DB_SC Если пароль содержит `@`, `:`, `/`, `?`, `#` или `%`, его необходимо URL-кодировать внутри PostgreSQL URL. -### 8.3. Генерация секретов +### 8.3. Подготовка runtime-секретов -Для обычных токенов используйте: +Генерируйте секреты вне shell history средствами secret manager. Не выполняйте +`export TOKEN=...` и не вставляйте значения в команды. Имена обязательных +runtime-переменных определены в `scripts/validate-env`; парные токены связываются +в secret backend. ```sh openssl rand -hex 32 @@ -337,7 +347,8 @@ openssl rand -hex 32 python3 -c 'import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())' ``` -Заполните все секреты. Две пары должны совпадать: +Ни один из этих секретов не добавляется в `.env`. Пары проверяются runtime +validator без вывода значений. ```dotenv BITRIX_LOCAL_APP_INTERNAL_TOKEN= @@ -361,7 +372,8 @@ KEYCLOAK_ADMIN_PASSWORD= ### 8.4. Redis -Создайте три разных пароля и продублируйте их в URL: +Создайте три разных пароля в secret backend; там же сформируйте Redis URL. +Следующий блок описывает логический контракт и не является содержимым `.env`: ```dotenv REDIS_API_PASSWORD= @@ -380,11 +392,10 @@ MESSAGE_SAFETY_REDIS_URL=redis://message_safety:@redis:63 ```dotenv KEYCLOAK_OTP_MOCK_ENABLED=true KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true -KEYCLOAK_OTP_MOCK_CODE=<ТЕСТОВЫЙ_КОД_НЕ_КОРОЧЕ_16_СИМВОЛОВ> ``` -Этот код будет вводиться пользователем при тестовой авторизации. Не используйте -его как production-механизм доставки OTP. +`KEYCLOAK_OTP_MOCK_CODE` хранится только в secret backend. Не используйте mock +как production-механизм доставки OTP. Целевой real mode задаёт `modules/module-11-idgtl-sms.md`: Keycloak генерирует и локально проверяет OTP, `sms-service` надёжно записывает заказ/журнал, worker вызывает i-Digital Direct, callback обновляет только delivery journal. Нельзя просто установить `KEYCLOAK_OTP_MOCK_ENABLED=false`. @@ -397,13 +408,10 @@ SELECTEL_S3_ENDPOINT_URL=https://s3.storage.selcloud.ru SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents - -SELECTEL_S3_ACCESS_KEY= -SELECTEL_S3_SECRET_KEY= -SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY= -SELECTEL_S3_QUARANTINE_READ_SECRET_KEY= ``` +Обе пары S3 credentials хранятся только в secret backend. + API backend принудительно использует virtual-hosted addressing: `https://.s3.storage.selcloud.ru/`. Это обязательно для браузерных presigned PUT и CORS в Selectel; path-style URL для этого сценария не @@ -411,20 +419,17 @@ API backend принудительно использует virtual-hosted addre ### 8.7. Bitrix24 -До установки локального приложения заполните: +До установки локального приложения загрузите credentials в secret backend. +В `.env` остаются только несекретные connector/public URL параметры: ```dotenv -BITRIX_CLIENT_ID= -BITRIX_CLIENT_SECRET= BITRIX_CONNECTOR_ID=han_mobile_app BITRIX_OPEN_LINE_ID=8 BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix -BITRIX_TOKEN_ENCRYPTION_KEY= ``` -`BITRIX_APPLICATION_TOKEN` можно окончательно задать после создания приложения в -Bitrix24. До этого используйте отдельное случайное значение, проходящее -валидацию окружения. +`BITRIX_APPLICATION_TOKEN` сохраняется в secret backend после создания +приложения и никогда не помещается в `.env`. ### 8.8. TLS и наблюдаемость @@ -448,8 +453,11 @@ ACME_EMAIL= ```sh cd /opt/han-chat/backend ./scripts/validate-env .env -docker compose --env-file .env config --quiet -docker compose --env-file .env config --services +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet +sudo deployment/secrets/han-compose config --services python3 -m unittest discover -s tests -v ``` @@ -670,7 +678,8 @@ curl -i https://chat.example.ru/internal/safety/v1/messages/check https://chat.example.ru/ ``` -Для тестовой авторизации используйте значение `KEYCLOAK_OTP_MOCK_CODE` из `.env`. +Для тестовой авторизации получите mock code утверждённым защищённым способом, +не читая его из `.env` и не помещая в shell history. ## 16. Подключение Bitrix24 @@ -685,7 +694,7 @@ Placement: https://chat.example.ru/bitrix/placement После установки: 1. Получите и сохраните application token. -2. Запишите его в `BITRIX_APPLICATION_TOKEN` файла `.env`. +2. Сохраните его как `BITRIX_APPLICATION_TOKEN` в secret backend. 3. Пересоздайте сервис: ```sh @@ -735,7 +744,7 @@ docker compose --env-file .env up -d --force-recreate 1. Создайте backup/PITR marker PostgreSQL. 2. Сохраните текущие image digests. 3. Получите новый код. -4. Проверьте `.env`. +4. Проверьте несекретный `.env` и runtime secret set. 5. Пересоберите образы. 6. Примените миграции и seed. 7. Пересоздайте сервисы. diff --git a/codebase/backend/deployment/RUNBOOK.md b/codebase/backend/deployment/RUNBOOK.md index 2a52608..43e0584 100644 --- a/codebase/backend/deployment/RUNBOOK.md +++ b/codebase/backend/deployment/RUNBOOK.md @@ -60,16 +60,20 @@ verify key-based deploy access in a separate SSH session. ## Gate 6 — environment and secrets ```sh -umask 077 cp .env.example .env -chmod 600 .env -# Replace placeholders using a protected editor/secret manager. +# Replace non-secret configuration placeholders only. ./scripts/validate-env .env -docker compose --env-file .env config --quiet +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet ``` -- [ ] Token pairs match, PG verifies TLS, public URLs are HTTPS. -- [ ] Mock OTP risk is accepted and all secrets are unique >=128-bit values. +- [ ] `SECRETS_SOURCE=file|selectel`; `.env` contains no secret keys or credential-bearing DSNs. +- [ ] `deployment/secrets/han-secrets` sets `HAN_SECRETS_ACTIVE=1`, does not log values, + and optionally exposes a paths-only `HAN_RUNTIME_SECRET_MANIFEST`. +- [ ] Runtime token pairs match, PG verifies TLS, public URLs are HTTPS. +- [ ] Mock OTP risk is accepted and runtime secrets are unique >=128-bit values. - [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` is unique and supplied only through secret/env; the `producer_test` source seed stores only its hash. - [ ] `FRONTEND_DEV_PROXY_ENABLED=false` and Safety/nginx timeout budgets match. @@ -211,8 +215,7 @@ For local acceptance only, start the redacted debug collector with: Provider backup/PITR is authoritative. A supplemental verified logical dump: ```sh -PG_BACKUP_DSN='postgresql://...?...sslmode=verify-full&sslrootcert=...' \ - deployment/scripts/backup.sh /opt/han-chat/backups +deployment/scripts/backup.sh /opt/han-chat/backups ``` Quarterly, restore PG and S3 into an isolated VPC, deploy the same image digests, @@ -225,10 +228,13 @@ Only roll back to images compatible with the current schema: ```sh SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \ - deployment/scripts/rollback.sh /secure/path/previous-release.env -ENV_FILE=/secure/path/previous-release.env deployment/scripts/smoke.sh + deployment/scripts/rollback.sh +deployment/scripts/smoke.sh ``` +Rollback reuses the current runtime secret set and non-secret config. Do not +create or restore an environment snapshot. + ## Real SMS rollout addendum This runbook remains mock-only until module-11 artifacts exist. An SMS release requires schema/role `sms`, versioned migrations and an active approved `auth_otp` seed, `sms-service`/worker, the exact callback route, paired service tokens, Direct `TOKEN_1`, approved sender/template, separate callback credentials, a reconfirmed callback source IP, and a static worker egress IP. diff --git a/codebase/backend/deployment/RUNBOOK.ru.md b/codebase/backend/deployment/RUNBOOK.ru.md index d1600ba..b57d734 100644 --- a/codebase/backend/deployment/RUNBOOK.ru.md +++ b/codebase/backend/deployment/RUNBOOK.ru.md @@ -63,16 +63,21 @@ sudo deployment/scripts/setup-vm.sh ## Этап 6 — окружение и секреты ```sh -umask 077 cp .env.example .env -chmod 600 .env -# Замените заглушки через защищенный редактор или менеджер секретов. +# Замените только несекретные placeholders. ./scripts/validate-env .env -docker compose --env-file .env config --quiet +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet ``` -- [ ] Парные токены совпадают, PostgreSQL проверяет TLS, публичные URL используют HTTPS. -- [ ] Риск mock OTP принят; все секреты уникальны и содержат не менее 128 бит энтропии. +- [ ] `SECRETS_SOURCE=file|selectel`; `.env` не содержит secret keys и DSN с credentials. +- [ ] `deployment/secrets/han-secrets` устанавливает `HAN_SECRETS_ACTIVE=1`, + не пишет значения в лог и при возможности передаёт paths-only manifest + через `HAN_RUNTIME_SECRET_MANIFEST`. +- [ ] Runtime-пары токенов совпадают, PostgreSQL проверяет TLS. +- [ ] Риск mock OTP принят; runtime-секреты уникальны и содержат не менее 128 бит энтропии. - [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` сгенерирован отдельно, передан только через secret/env; seed `notification_sources.code='producer_test'` содержит только его hash. - [ ] Установлено `FRONTEND_DEV_PROXY_ENABLED=false`; таймауты Safety и nginx согласованы. @@ -233,8 +238,7 @@ deployment/scripts/smoke.sh Основным механизмом являются backup/PITR провайдера. Дополнительный проверенный логический дамп: ```sh -PG_BACKUP_DSN='postgresql://...?...sslmode=verify-full&sslrootcert=...' \ - deployment/scripts/backup.sh /opt/han-chat/backups +deployment/scripts/backup.sh /opt/han-chat/backups ``` Ежеквартально восстанавливайте PostgreSQL и S3 в изолированной VPC, развертывайте те же @@ -248,10 +252,13 @@ smoke-тесты и фиксируйте фактические RPO/RTO. Redis ```sh SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \ - deployment/scripts/rollback.sh /secure/path/previous-release.env -ENV_FILE=/secure/path/previous-release.env deployment/scripts/smoke.sh + deployment/scripts/rollback.sh +deployment/scripts/smoke.sh ``` +Откат использует текущие runtime-секреты и текущий несекретный config. Snapshot +старого `.env` не создаётся и не восстанавливается. + ## Дополнение: rollout реальной SMS-авторизации Текущий runbook остаётся mock-only, пока артефакты module-11 не реализованы. Для SMS release обязательны: schema/role `sms`, migrations/seed active approved `auth_otp`, `sms-service`/worker, exact callback route, парные service tokens, Direct `TOKEN_1`, согласованные sender/template, отдельные callback credentials, подтверждённый callback source IP и статический egress IP worker. diff --git a/codebase/backend/deployment/docker-compose.jobs.yml b/codebase/backend/deployment/docker-compose.jobs.yml index 789de1b..8388424 100644 --- a/codebase/backend/deployment/docker-compose.jobs.yml +++ b/codebase/backend/deployment/docker-compose.jobs.yml @@ -1,83 +1,123 @@ -x-no-sms-secrets: &no-sms-secrets - SMS_DATABASE_URL: "" - SMS_SERVICE_TOKEN: "" - KEYCLOAK_SMS_SERVICE_TOKEN: "" - IDGTL_SMS_API_KEY: "" - IDGTL_SMS_CALLBACK_USERNAME: "" - IDGTL_SMS_CALLBACK_PASSWORD: "" +x-api-job-secrets: &api-job-secrets + - api_database_url + - api_redis_url + - api_redis_realtime_url + - message_safety_service_token + - bitrix_local_app_internal_token + - bitrix_api_inbox_token + - keycloak_settings_bridge_token + - selectel_s3_access_key + - selectel_s3_secret_key + - cursor_hmac_secret + +x-api-job-environment: &api-job-environment + HAN_SECRET_VARS: >- + DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN + BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN + KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY + CURSOR_HMAC_SECRET + DATABASE_URL_FILE: /run/secrets/api_database_url + REDIS_URL_FILE: /run/secrets/api_redis_url + REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url + MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token + BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token + BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token + KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token + SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key + SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key + CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret + APP_ENV: ${APP_ENV:-production-like} + LOG_LEVEL: ${LOG_LEVEL:-INFO} + KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL} + KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL:-http://keycloak:8080/auth} + KEYCLOAK_REALM: ${KEYCLOAK_REALM:-han-chat} + KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE:-han-chat-api} + MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL:-http://message-safety:8080} + MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5} + MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2} + MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300} + BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL:-http://bitrix-local-app:8080} + BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20} + SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL} + SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS} + SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS} + SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE} + TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-127.0.0.1/32} services: migrate-api: image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local} profiles: ["ops"] - env_file: - - path: ../.env - required: false - environment: *no-sms-secrets - entrypoint: [] + environment: + HAN_SECRET_VARS: DATABASE_URL + DATABASE_URL_FILE: /run/secrets/api_database_url + secrets: + - api_database_url command: ["alembic", "upgrade", "head"] volumes: - ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro networks: [backend, egress] restart: "no" security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} migrate-bitrix-local: image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local} profiles: ["ops"] - env_file: - - path: ../.env - required: false - environment: *no-sms-secrets - entrypoint: [] + environment: + HAN_SECRET_VARS: BITRIX_DATABASE_URL + BITRIX_DATABASE_URL_FILE: /run/secrets/bitrix_database_url + secrets: + - bitrix_database_url command: ["alembic", "upgrade", "head"] volumes: - ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro networks: [backend, egress] restart: "no" security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} migrate-bitrix-sync: image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local} profiles: ["ops"] - env_file: - - path: ../.env - required: false - environment: *no-sms-secrets - entrypoint: [] + environment: + HAN_SECRET_VARS: BITRIX_SYNC_DATABASE_URL + BITRIX_SYNC_DATABASE_URL_FILE: /run/secrets/bitrix_sync_database_url + secrets: + - bitrix_sync_database_url command: ["alembic", "upgrade", "head"] volumes: - ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro networks: [backend, egress] restart: "no" security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} migrate-sms: image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local} profiles: ["ops"] environment: - SMS_DATABASE_URL: ${SMS_DATABASE_URL} - SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN} - IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru} - IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL} - IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME} - IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD} - entrypoint: [] + HAN_SECRET_VARS: SMS_DATABASE_URL + SMS_DATABASE_URL_FILE: /run/secrets/sms_database_url + secrets: + - sms_database_url command: ["alembic", "upgrade", "head"] volumes: - ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro networks: [backend, egress] restart: "no" security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} seed-settings: image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local} profiles: ["ops"] - env_file: - - path: ../.env - required: false - environment: *no-sms-secrets - entrypoint: [] + environment: *api-job-environment + secrets: *api-job-secrets command: - /bin/sh - -ec @@ -91,6 +131,8 @@ services: networks: [backend, egress] restart: "no" security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} toolbox: image: curlimages/curl:8.11.1 diff --git a/codebase/backend/deployment/scripts/backup.sh b/codebase/backend/deployment/scripts/backup.sh index 5aba26a..3d174d7 100644 --- a/codebase/backend/deployment/scripts/backup.sh +++ b/codebase/backend/deployment/scripts/backup.sh @@ -1,10 +1,22 @@ #!/bin/sh set -eu +cd "$(dirname "$0")/../.." +CONFIG_FILE=${CONFIG_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} -if [ -z "${PG_BACKUP_DSN:-}" ]; then - echo "PG_BACKUP_DSN is required (managed PostgreSQL TLS DSN, supplied via secure environment)." >&2 +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi + +if [ -z "${PG_BACKUP_DSN_FILE:-}" ] || [ ! -r "$PG_BACKUP_DSN_FILE" ]; then + echo "PG_BACKUP_DSN_FILE is required from the secret launcher." >&2 exit 64 fi +PG_BACKUP_DSN=$(cat "$PG_BACKUP_DSN_FILE") case "$PG_BACKUP_DSN" in *sslmode=verify-full*sslrootcert=*) ;; *) echo "PG_BACKUP_DSN must enforce sslmode=verify-full and sslrootcert." >&2; exit 64 ;; @@ -16,7 +28,9 @@ mkdir -p "$output_dir" stamp=$(date -u +%Y%m%dT%H%M%SZ) archive="$output_dir/han-chat-$stamp.dump" -pg_dump --dbname="$PG_BACKUP_DSN" --format=custom --no-owner --no-privileges --file="$archive" +PGDATABASE="$PG_BACKUP_DSN" pg_dump \ + --format=custom --no-owner --no-privileges --file="$archive" +unset PG_BACKUP_DSN pg_restore --list "$archive" >/dev/null sha256sum "$archive" > "$archive.sha256" chmod 600 "$archive" "$archive.sha256" diff --git a/codebase/backend/deployment/scripts/migrate.sh b/codebase/backend/deployment/scripts/migrate.sh index 9edd82a..8515f53 100644 --- a/codebase/backend/deployment/scripts/migrate.sh +++ b/codebase/backend/deployment/scripts/migrate.sh @@ -1,20 +1,34 @@ #!/bin/sh set -eu cd "$(dirname "$0")/../.." +CONFIG_FILE=${CONFIG_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} + +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi if [ "${PITR_MARKER_CONFIRMED:-false}" != "true" ]; then echo "Refusing migration: create provider PITR marker, then set PITR_MARKER_CONFIRMED=true" >&2 exit 64 fi -./scripts/validate-env "${ENV_FILE:-.env}" -docker compose --env-file "${ENV_FILE:-.env}" config --quiet -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-api alembic current -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-local alembic current -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-sync alembic current -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-sms alembic current -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-api -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-local -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-sync -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-sms +if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then + ./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST" +else + ./scripts/validate-env "$CONFIG_FILE" --runtime-env +fi +docker compose --env-file "$CONFIG_FILE" config --quiet +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-api alembic current +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-local alembic current +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-sync alembic current +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-sms alembic current +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-api +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-local +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-sync +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-sms echo "Migrations completed; record revisions in release evidence." diff --git a/codebase/backend/deployment/scripts/rollback.sh b/codebase/backend/deployment/scripts/rollback.sh index 05cc645..fe7777d 100644 --- a/codebase/backend/deployment/scripts/rollback.sh +++ b/codebase/backend/deployment/scripts/rollback.sh @@ -1,21 +1,34 @@ #!/bin/sh set -eu cd "$(dirname "$0")/../.." +CONFIG_FILE=${CONFIG_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} -previous_env=${1:-} -if [ -z "$previous_env" ] || [ ! -r "$previous_env" ]; then - echo "Usage: $0 /secure/path/previous-release.env" >&2 +previous_release=${1:-} +if [ -z "$previous_release" ]; then + echo "Usage: $0 " >&2 exit 64 fi +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi if [ "${SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED:-false}" != "true" ]; then echo "Refusing rollback: set SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true after migration review." >&2 exit 64 fi -./scripts/validate-env "$previous_env" -docker compose --env-file "$previous_env" config --quiet -docker compose --env-file "$previous_env" up -d --remove-orphans -docker compose --env-file "$previous_env" ps +if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then + ./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST" +else + ./scripts/validate-env "$CONFIG_FILE" --runtime-env +fi +RELEASE_VERSION="$previous_release" docker compose --env-file "$CONFIG_FILE" config --quiet +RELEASE_VERSION="$previous_release" docker compose --env-file "$CONFIG_FILE" up -d --remove-orphans +docker compose --env-file "$CONFIG_FILE" ps echo "Application images rolled back without Alembic downgrade." -echo "Run deployment/scripts/smoke.sh with ENV_FILE=$previous_env and verify outbox/inbox idempotency." +echo "Run deployment/scripts/smoke.sh and verify outbox/inbox idempotency." diff --git a/codebase/backend/deployment/scripts/seed.sh b/codebase/backend/deployment/scripts/seed.sh index 8e3c4d7..b319286 100644 --- a/codebase/backend/deployment/scripts/seed.sh +++ b/codebase/backend/deployment/scripts/seed.sh @@ -1,7 +1,18 @@ #!/bin/sh set -eu cd "$(dirname "$0")/../.." +CONFIG_FILE=${ENV_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} -./scripts/validate-env "${ENV_FILE:-.env}" -docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm seed-settings +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi + +./scripts/validate-env "$CONFIG_FILE" \ + --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST" +docker compose --env-file "$CONFIG_FILE" --profile ops run --rm seed-settings echo "Seed and mandatory-settings validation completed." diff --git a/codebase/backend/deployment/scripts/setup-vm.sh b/codebase/backend/deployment/scripts/setup-vm.sh index dd5ac71..8ed8541 100644 --- a/codebase/backend/deployment/scripts/setup-vm.sh +++ b/codebase/backend/deployment/scripts/setup-vm.sh @@ -435,6 +435,46 @@ configure_application_security() { fi } +install_secret_loader_if_possible() { + step "Загрузчик секретов" + local source_dir="${DEPLOY_DIR}/deployment/secrets" + if [[ ! -f "${source_dir}/secrets_loader.py" || ! -f "${source_dir}/han-secrets" ]]; then + log "Проект еще не скопирован: загрузчик секретов будет установлен при повторном запуске" + return + fi + chmod 0750 "${source_dir}/han-secrets" "${source_dir}/han-compose" + + install -d -m 0700 -o root -g root \ + /etc/han \ + /etc/han/secrets \ + /etc/han/credentials + install -d -m 0755 -o root -g root \ + /usr/local/lib/han-secrets \ + /usr/local/share/doc/han-secrets + install -m 0750 -o root -g root \ + "${source_dir}/secrets_loader.py" \ + /usr/local/lib/han-secrets/secrets_loader.py + install -m 0750 -o root -g root \ + "${source_dir}/han-secrets" \ + /usr/local/lib/han-secrets/han-secrets + install -m 0750 -o root -g root \ + "${source_dir}/han-compose" \ + /usr/local/bin/han-compose + install -m 0644 -o root -g root \ + "${source_dir}/han-secrets@.service" \ + /etc/systemd/system/han-secrets@.service + install -m 0644 -o root -g root \ + "${source_dir}/SELECTEL_RUNBOOK.ru.md" \ + /usr/local/share/doc/han-secrets/SELECTEL_RUNBOOK.ru.md + if [[ ! -e /etc/han/secrets/production.selectel.json.example ]]; then + install -m 0600 -o root -g root \ + "${source_dir}/config.example.json" \ + /etc/han/secrets/production.selectel.json.example + fi + systemctl daemon-reload + log "Загрузчик установлен, но не включен: сначала выполните SELECTEL_RUNBOOK.ru.md" +} + install_ssl_timer_if_possible() { step "Таймер продления TLS" local renew_script="${DEPLOY_DIR}/deployment/scripts/ssl-renew.sh" @@ -504,14 +544,18 @@ HSTS max-age: ${HSTS_MAX_AGE_SECONDS} ${DEPLOY_DIR} 3. Поместите CA PostgreSQL: ${DEPLOY_DIR}/secrets/pg/ca.pem -4. Создайте и заполните окружение: +4. Создайте только несекретный config: cd ${DEPLOY_DIR} cp .env.example .env chmod 600 .env ./scripts/validate-env .env -5. Продолжите с Gate 7 в: +5. Настройте Selectel, encrypted bootstrap credential и fallback map: + deployment/secrets/SELECTEL_RUNBOOK.ru.md +6. Выполняйте Compose только через: + sudo deployment/secrets/han-compose +7. Продолжите с Gate 7 в: deployment/RUNBOOK.ru.md -6. После копирования проекта повторно запустите этот скрипт для установки TLS-таймера. +8. После копирования проекта повторно запустите этот скрипт для установки unit-файлов. Важно: членство в группе docker начнет действовать после нового входа в систему. EOF @@ -538,6 +582,7 @@ main() { configure_docker_firewall configure_ssh configure_application_security + install_secret_loader_if_possible install_ssl_timer_if_possible verify summary diff --git a/codebase/backend/deployment/scripts/smoke.sh b/codebase/backend/deployment/scripts/smoke.sh index ffe96f6..d8e9999 100644 --- a/codebase/backend/deployment/scripts/smoke.sh +++ b/codebase/backend/deployment/scripts/smoke.sh @@ -2,11 +2,23 @@ set -eu cd "$(dirname "$0")/../.." -ENV_FILE=${ENV_FILE:-.env} -./scripts/validate-env "$ENV_FILE" +CONFIG_FILE=${CONFIG_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi +if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then + ./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST" +else + ./scripts/validate-env "$CONFIG_FILE" --runtime-env +fi env_value() { - python3 - "$ENV_FILE" "$1" <<'PY' + python3 - "$CONFIG_FILE" "$1" <<'PY' import sys from pathlib import Path diff --git a/codebase/backend/deployment/scripts/ssl-renew.sh b/codebase/backend/deployment/scripts/ssl-renew.sh index d92df6e..0049d7b 100644 --- a/codebase/backend/deployment/scripts/ssl-renew.sh +++ b/codebase/backend/deployment/scripts/ssl-renew.sh @@ -1,13 +1,23 @@ #!/bin/sh set -eu cd "$(dirname "$0")/../.." +CONFIG_FILE=${CONFIG_FILE:-.env} +SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets} + +if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then + [ -x "$SECRETS_LAUNCHER" ] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi lock=/tmp/han-chat-cert-renew.lock exec 9>"$lock" flock -n 9 || { echo '{"event":"tls.renew.skipped","reason":"lock_busy"}'; exit 0; } compose() { - docker compose --env-file .env "$@" + docker compose --env-file "$CONFIG_FILE" "$@" } nginx_container="$(compose ps --status running --quiet nginx)" diff --git a/codebase/backend/deployment/scripts/verify-observability.sh b/codebase/backend/deployment/scripts/verify-observability.sh index 75ac39f..9d591be 100644 --- a/codebase/backend/deployment/scripts/verify-observability.sh +++ b/codebase/backend/deployment/scripts/verify-observability.sh @@ -1,5 +1,18 @@ #!/usr/bin/env bash set -Eeuo pipefail +cd "$(dirname "$0")/../.." +CONFIG_FILE="${CONFIG_FILE:-.env}" +SECRETS_LAUNCHER="${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}" + +if [[ "${HAN_SECRETS_ACTIVE:-0}" != 1 ]]; then + [[ -x "$SECRETS_LAUNCHER" ]] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi + +compose() { docker compose --env-file "$CONFIG_FILE" "$@"; } NETWORK="${OBSERVABILITY_NETWORK:-han-chat-observability}" COLLECTOR_SERVICE="${COLLECTOR_SERVICE:-otel-collector}" @@ -8,7 +21,7 @@ errors=0 ok() { printf 'OK %s\n' "$*"; } fail() { printf 'FAIL %s\n' "$*" >&2; errors=$((errors + 1)); } -collector_id="$(docker compose ps -q "$COLLECTOR_SERVICE" 2>/dev/null || true)" +collector_id="$(compose ps -q "$COLLECTOR_SERVICE" 2>/dev/null || true)" if [[ -n "$collector_id" ]] && [[ "$(docker inspect --format '{{.State.Status}}' "$collector_id")" == running ]]; then ok "Collector service is running" @@ -17,7 +30,7 @@ else fi for service in sms-service sms-worker; do - if docker compose exec -T "$service" python - <<'PY' >/dev/null 2>&1 + if compose exec -T "$service" python - <<'PY' >/dev/null 2>&1 import os import urllib.request port = os.environ.get("SMS_METRICS_PORT", "9464") if "worker" in os.environ.get("OTEL_SERVICE_NAME", "") else "8080" @@ -38,7 +51,7 @@ docker run --rm --network "$NETWORK" \ || fail "telemetrygen failed" bad_logs="$( - docker compose logs --since=10m "$COLLECTOR_SERVICE" 2>&1 | + compose logs --since=10m "$COLLECTOR_SERVICE" 2>&1 | grep -Ei 'queue is full|connection refused|tls:|Unauthenticated|Permanent error' || true )" if [[ -z "$bad_logs" ]]; then diff --git a/codebase/backend/deployment/secrets/SELECTEL_RUNBOOK.ru.md b/codebase/backend/deployment/secrets/SELECTEL_RUNBOOK.ru.md new file mode 100644 index 0000000..0ac7ba9 --- /dev/null +++ b/codebase/backend/deployment/secrets/SELECTEL_RUNBOOK.ru.md @@ -0,0 +1,206 @@ +# Selectel Secrets Manager для HAN Chat + +## Модель + +`han-secrets` читает только `SECRETS_SOURCE=selectel|file` из обычного `.env`, +выбирает соответствующую root-only JSON-карту и вызывает `secrets_loader.py`. +Загрузчик получает project-scoped IAM token, читает объявленные секреты и +создаёт в `/run/han-chat/secrets`: + +- отдельные файлы с каноническими именами для Compose secrets; +- узкие service dotenv-файлы для диагностики состава без вывода значений; +- `manifest` вида `NAME=/absolute/path`, используемый валидатором. + +Каталог `/run` находится в tmpfs и имеет режим `0700`. Канонические файлы имеют +`0444`: локальные пользователи не могут пройти через root-only каталог, а +не-root UID контейнера может прочитать только явно смонтированный Compose +secret. Значения не передаются через Docker Config.Env, argv, общий `.env` или +логи. После полного root/docker-компромисса runtime-значения извлекаемы — это +ограничение модели, а не гарантия Secret Manager. + +Сбой Selectel никогда автоматически не включает file fallback. Уже работающие +контейнеры продолжают использовать текущие значения; новый sync завершается +fail-closed. + +## 1. Ресурсы Selectel + +1. Создайте отдельный проект `han-chat-secrets-prod`. +2. Создайте сервисного пользователя `han-chat-secrets-reader`. +3. Назначьте ему `member` только в этом проекте. Не выдавайте account scope, + `iam.admin` и доступ к другим production-ресурсам. Если Selectel добавит + отдельную read-only роль Secrets Manager, замените `member` на неё. +4. Ограничьте обращения к `api.selectel.ru` исходящим IP ВМ, если функция + доступна в аккаунте. +5. Включите экспорт audit logs. Контролируйте события `secrets.secret*` и + `secrets.secret_version*`; alert на delete, смену current version вне окна, + массовые чтения и обращения не от штатного пользователя/IP. +6. Выполните canary и убедитесь, что provider audit содержит metadata операции, + но не value, Base64 payload, IAM token или response body. + +Secrets Manager принимает project-scoped IAM token в `X-Auth-Token`. Token +живёт до 24 часов, но загрузчик использует его только в памяти одного запуска. +TLS и redirect policy отключать нельзя. + +## 2. Каталог секретов + +Скопируйте `config.example.json` в +`/etc/han/secrets/production-like.selectel.json` и замените account, username, +project, region и `remote`. Каноническое имя слева обязано совпадать с +Compose/validator; `remote` — неизменяемый ключ в Selectel. + +Используйте консервативные provider keys с дефисами, например: + +- `han-chat-prod-pg-han-app-dsn`, `han-chat-prod-pg-bitrix-local-dsn`, + `han-chat-prod-pg-bitrix-sync-dsn`, `han-chat-prod-pg-sms-dsn`, + `han-chat-prod-pg-keycloak-password`, `han-chat-prod-pg-backup-dsn`; +- `han-chat-prod-redis-api-password`, `han-chat-prod-redis-safety-password`, + `han-chat-prod-redis-health-password`, а также три credential-bearing URL; +- `han-chat-prod-message-safety-token`, `han-chat-prod-bitrix-internal-token`, + `han-chat-prod-bitrix-forward-token`, `han-chat-prod-bitrix-sync-token`, + `han-chat-prod-keycloak-settings-token`, `han-chat-prod-sms-service-token`; +- Keycloak bootstrap password, OTP HMAC и mock code только для среды, где mock + действительно включён; +- Bitrix client secret, application token и token encryption key; +- пары S3 app access/secret key; +- i-Digital API key и callback username/password. + +Одинаковые пары env (`BITRIX_LOCAL_APP_INTERNAL_TOKEN` / +`BITRIX_INTERNAL_API_TOKEN`, `BITRIX_API_FORWARD_TOKEN` / +`BITRIX_API_INBOX_TOKEN`, `SMS_SERVICE_TOKEN` / +`KEYCLOAK_SMS_SERVICE_TOKEN`) должны ссылаться на один `remote`. + +`literal: ""` разрешён только для заведомо пустого optional-параметра, например +OTLP auth при self-hosted SigNoz или выключенного CAPTCHA server key. Секреты +не записывайте в JSON. Не заводите planned/unused Message Safety DB, quarantine +S3 и Bitrix sync credentials до появления потребляющего кода. + +Загружайте значения в Selectel через скрытый prompt/stdin. Не передавайте value +позиционным аргументом CLI и не включайте `set -x`/`curl -v`. + +## 3. Установка на ВМ + +Повторный запуск `deployment/scripts/setup-vm.sh` после копирования проекта +устанавливает loader, launcher, systemd template и этот runbook. Вручную: + +```sh +sudo install -d -m 0700 /etc/han/secrets /etc/han/credentials +sudo install -d -m 0755 /usr/local/lib/han-secrets +sudo install -m 0750 secrets_loader.py han-secrets /usr/local/lib/han-secrets/ +sudo install -m 0750 han-compose /usr/local/bin/han-compose +sudo install -m 0644 han-secrets@.service /etc/systemd/system/ +sudo install -m 0600 config.example.json \ + /etc/han/secrets/production-like.selectel.json +``` + +Обычный `/opt/han-chat/backend/.env` содержит только несекретные параметры. +Штатный режим: + +```dotenv +SECRETS_SOURCE=selectel +APP_ENV=production-like +``` + +## 4. Bootstrap credential + +Не храните пароль service user в `.env` или JSON. На целевой ВМ: + +```sh +sudo systemd-creds encrypt --name=selectel-service-user-password - \ + /etc/han/credentials/production.selectel-password.cred +sudo chmod 0600 /etc/han/credentials/production.selectel-password.cred +``` + +Введите пароль через интерактивный stdin. Unit передаёт расшифрованный файл +через приватный `$CREDENTIALS_DIRECTORY`. Root всё равно может его извлечь; +после инцидента credential и все доступные ему секреты необходимо ротировать. + +## 5. Проверка и запуск + +Все команды, которым нужны Compose secrets, запускайте от root через wrapper: + +```sh +cd /opt/han-chat/backend +sudo ./scripts/validate-env .env +sudo systemctl daemon-reload +sudo systemctl enable han-secrets@production.service +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet +sudo deployment/secrets/han-compose up -d --wait +``` + +Selectel sync запускается именно unit-файлом: только он предоставляет +расшифрованный bootstrap credential через `$CREDENTIALS_DIRECTORY`. +`han-compose` и ops-скрипты используют уже синхронизированный manifest и +отказываются работать, если `SECRETS_SOURCE`/loader config не совпадают с +runtime state. После смены source, provider version или JSON-карты сначала +выполняйте `systemctl restart han-secrets@production.service`. + +Не используйте `docker compose config` без `--quiet`, `docker inspect` для +поиска конфигурации, `env`, `strace`, core dump или debug HTTP proxy. Проверка +приёмки должна подтвердить отсутствие canary value в `docker inspect`, stdout, +json logs, traces и shell history. + +На выделенной только под HAN Chat ВМ после canary и проверки file fallback +установите fail-closed ordering: + +```sh +sudo install -d -m 0755 /etc/systemd/system/docker.service.d +sudo install -m 0644 \ + deployment/secrets/docker-han-secrets.conf.example \ + /etc/systemd/system/docker.service.d/han-secrets.conf +sudo systemctl daemon-reload +sudo systemctl restart docker +``` + +После этого проведите reboot rehearsal: materializer должен завершиться до +autorestart контейнеров. Ошибка Selectel намеренно блокирует Docker. На ВМ с +другими workloads такой глобальный `Requires=` запрещён: нужен отдельный Docker +daemon/VM, иначе fail-closed HAN остановит несвязанные системы. + +## 6. Явный file fallback + +Подготовьте отдельную карту +`/etc/han/secrets/production-like.file.json`: скопируйте Selectel-карту, +установите `"mode": "file"`, удалите `selectel` и `http`, добавьте: + +```json +"file": { + "path": "/etc/han/break-glass/secrets.env", + "max_bytes": 1048576 +} +``` + +`secrets` map остаётся тем же. Для записей с `literal: ""` строка в fallback +не нужна; остальные канонические ключи обязательны. Fallback parser не исполняет +shell: запрещены `export`, substitutions, multiline, неизвестные и дублирующиеся +ключи. Файл — `root:root 0600`. + +При инциденте доставьте recovery-файл из защищённой офлайн-копии и только затем +явно измените `.env`: + +```dotenv +SECRETS_SOURCE=file +``` + +Перезапустите `han-secrets@production.service`, затем выполните +validate/recreate через wrappers. После восстановления Selectel верните +`SECRETS_SOURCE=selectel`, снова перезапустите unit, повторите проверки и +удалите recovery-файл. +Не храните его постоянно на ВМ: это вернуло бы исходный риск монолитного `.env`. + +## 7. Ротация и rollback + +1. Добавьте новую версию секрета, не меняя имя. +2. Для canary при необходимости временно pin числовой `version` в JSON-карте. +3. Выполните sync, validation и smoke без вывода конфигурации. +4. Сделайте версию current, удалите pin, снова sync и пересоздайте только + потребителей. +5. Для rollback активируйте предыдущую provider version; не храните snapshot + старого `.env`. + +Selectel не позволяет удалить отдельную версию — только секрет целиком. Старые +значения должны быть отозваны в PostgreSQL/S3/Bitrix/i-Digital после окна +rollback. Все значения из прежнего `.env` считайте раскрытыми и ротируйте после +перехода. diff --git a/codebase/backend/deployment/secrets/config.example.json b/codebase/backend/deployment/secrets/config.example.json new file mode 100644 index 0000000..0b317cf --- /dev/null +++ b/codebase/backend/deployment/secrets/config.example.json @@ -0,0 +1,190 @@ +{ + "version": 1, + "mode": "selectel", + "runtime_dir": "/run/han-chat/secrets", + "http": { + "timeout_seconds": 10, + "retries": 3, + "max_response_bytes": 1048576 + }, + "selectel": { + "account_id": "123456", + "username": "han-secrets-reader", + "project_name": "han-production", + "region": "ru-9", + "interface": "public", + "password_file": "selectel-service-user-password" + }, + "secrets": { + "DATABASE_URL": { + "remote": "DATABASE_URL", + "consumers": ["api-backend", "api-migrate"], + "max_bytes": 4096 + }, + "BITRIX_DATABASE_URL": { + "remote": "BITRIX_DATABASE_URL", + "consumers": ["bitrix-local-app", "bitrix-local-migrate"], + "max_bytes": 4096 + }, + "BITRIX_SYNC_DATABASE_URL": { + "remote": "BITRIX_SYNC_DATABASE_URL", + "consumers": ["bitrix-sync", "bitrix-sync-migrate"], + "max_bytes": 4096 + }, + "SMS_DATABASE_URL": { + "remote": "SMS_DATABASE_URL", + "consumers": ["sms-service", "sms-worker", "sms-migrate"], + "max_bytes": 4096 + }, + "KEYCLOAK_DB_PASSWORD": { + "remote": "KEYCLOAK_DB_PASSWORD", + "consumers": ["keycloak"], + "max_bytes": 1024 + }, + "KEYCLOAK_ADMIN_PASSWORD": { + "remote": "KEYCLOAK_ADMIN_PASSWORD", + "consumers": ["keycloak"], + "max_bytes": 1024 + }, + "CURSOR_HMAC_SECRET": { + "remote": "CURSOR_HMAC_SECRET", + "consumers": ["api-backend"], + "max_bytes": 1024 + }, + "KEYCLOAK_OTP_HMAC_KEY": { + "remote": "KEYCLOAK_OTP_HMAC_KEY", + "consumers": ["keycloak"], + "max_bytes": 1024 + }, + "KEYCLOAK_OTP_MOCK_CODE": { + "remote": "KEYCLOAK_OTP_MOCK_CODE", + "consumers": ["keycloak"], + "max_bytes": 1024 + }, + "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY": { + "remote": "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", + "consumers": ["keycloak"], + "max_bytes": 1024 + }, + "BITRIX_TOKEN_ENCRYPTION_KEY": { + "remote": "BITRIX_TOKEN_ENCRYPTION_KEY", + "consumers": ["api-backend", "bitrix-local-app", "bitrix-sync"], + "max_bytes": 1024 + }, + "REDIS_API_PASSWORD": { + "remote": "REDIS_API_PASSWORD", + "consumers": ["redis"], + "max_bytes": 1024 + }, + "REDIS_URL": { + "remote": "REDIS_URL", + "consumers": ["api-backend", "delivery-worker", "cleanup-worker"], + "max_bytes": 4096 + }, + "REDIS_REALTIME_URL": { + "remote": "REDIS_REALTIME_URL", + "consumers": ["api-backend"], + "max_bytes": 4096 + }, + "REDIS_SAFETY_PASSWORD": { + "remote": "REDIS_SAFETY_PASSWORD", + "consumers": ["redis"], + "max_bytes": 1024 + }, + "MESSAGE_SAFETY_REDIS_URL": { + "remote": "MESSAGE_SAFETY_REDIS_URL", + "consumers": ["message-safety", "safety-recovery-worker"], + "max_bytes": 4096 + }, + "REDIS_HEALTH_PASSWORD": { + "remote": "REDIS_HEALTH_PASSWORD", + "consumers": ["redis", "redis-exporter"], + "max_bytes": 1024 + }, + "MESSAGE_SAFETY_SERVICE_TOKEN": { + "remote": "MESSAGE_SAFETY_SERVICE_TOKEN", + "consumers": ["api-backend", "message-safety"], + "max_bytes": 1024 + }, + "BITRIX_LOCAL_APP_INTERNAL_TOKEN": { + "remote": "BITRIX_LOCAL_APP_INTERNAL_TOKEN", + "consumers": ["api-backend", "bitrix-local-app"], + "max_bytes": 1024 + }, + "BITRIX_INTERNAL_API_TOKEN": { + "remote": "BITRIX_LOCAL_APP_INTERNAL_TOKEN", + "consumers": ["api-backend", "bitrix-local-app"], + "max_bytes": 1024 + }, + "BITRIX_API_FORWARD_TOKEN": { + "remote": "BITRIX_API_FORWARD_TOKEN", + "consumers": ["api-backend", "bitrix-local-app"], + "max_bytes": 1024 + }, + "BITRIX_API_INBOX_TOKEN": { + "remote": "BITRIX_API_FORWARD_TOKEN", + "consumers": ["api-backend", "bitrix-local-app"], + "max_bytes": 1024 + }, + "BITRIX_SYNC_SERVICE_TOKEN": { + "remote": "BITRIX_SYNC_SERVICE_TOKEN", + "consumers": ["api-backend", "bitrix-sync"], + "max_bytes": 1024 + }, + "KEYCLOAK_SETTINGS_BRIDGE_TOKEN": { + "remote": "KEYCLOAK_SETTINGS_BRIDGE_TOKEN", + "consumers": ["api-backend", "keycloak"], + "max_bytes": 1024 + }, + "SMS_SERVICE_TOKEN": { + "remote": "SMS_SERVICE_TOKEN", + "consumers": ["sms-service", "sms-worker", "keycloak"], + "max_bytes": 1024 + }, + "KEYCLOAK_SMS_SERVICE_TOKEN": { + "remote": "SMS_SERVICE_TOKEN", + "consumers": ["sms-service", "sms-worker", "keycloak"], + "max_bytes": 1024 + }, + "IDGTL_SMS_API_KEY": { + "remote": "IDGTL_SMS_API_KEY", + "consumers": ["sms-service", "sms-worker"], + "max_bytes": 1024 + }, + "IDGTL_SMS_CALLBACK_USERNAME": { + "remote": "IDGTL_SMS_CALLBACK_USERNAME", + "consumers": ["sms-service", "sms-worker"], + "max_bytes": 1024 + }, + "IDGTL_SMS_CALLBACK_PASSWORD": { + "remote": "IDGTL_SMS_CALLBACK_PASSWORD", + "consumers": ["sms-service", "sms-worker"], + "max_bytes": 1024 + }, + "BITRIX_CLIENT_SECRET": { + "remote": "BITRIX_CLIENT_SECRET", + "consumers": ["bitrix-local-app"], + "max_bytes": 1024 + }, + "BITRIX_APPLICATION_TOKEN": { + "remote": "BITRIX_APPLICATION_TOKEN", + "consumers": ["bitrix-local-app"], + "max_bytes": 1024 + }, + "SELECTEL_S3_SECRET_KEY": { + "remote": "SELECTEL_S3_SECRET_KEY", + "consumers": ["api-backend"], + "max_bytes": 1024 + }, + "SELECTEL_S3_ACCESS_KEY": { + "remote": "SELECTEL_S3_ACCESS_KEY", + "consumers": ["api-backend"], + "max_bytes": 1024 + }, + "OTEL_REMOTE_AUTH_HEADER": { + "literal": "", + "consumers": ["otel-collector"], + "max_bytes": 4096 + } + } +} diff --git a/codebase/backend/deployment/secrets/docker-han-secrets.conf.example b/codebase/backend/deployment/secrets/docker-han-secrets.conf.example new file mode 100644 index 0000000..2e9d358 --- /dev/null +++ b/codebase/backend/deployment/secrets/docker-han-secrets.conf.example @@ -0,0 +1,6 @@ +[Unit] +# Enable only on a dedicated HAN Chat Docker host after the Selectel canary and +# file fallback have both passed. A failed secret sync intentionally blocks +# Docker startup so containers cannot race an empty /run directory. +Requires=han-secrets@production.service +After=han-secrets@production.service diff --git a/codebase/backend/deployment/secrets/han-compose b/codebase/backend/deployment/secrets/han-compose new file mode 100644 index 0000000..85f9a39 --- /dev/null +++ b/codebase/backend/deployment/secrets/han-compose @@ -0,0 +1,15 @@ +#!/bin/sh +set -eu + +DEPLOY_DIR=${HAN_DEPLOY_DIR:-/opt/han-chat/backend} +SOURCE_DIR=$(CDPATH= cd -- "$(dirname "$0")" && pwd) +[ ! -f "$SOURCE_DIR/../../docker-compose.yml" ] || \ + DEPLOY_DIR=$(CDPATH= cd -- "$SOURCE_DIR/../.." && pwd) +cd "$DEPLOY_DIR" + +CONFIG_FILE=${CONFIG_FILE:-.env} +LAUNCHER=${HAN_SECRETS_LAUNCHER:-/usr/local/lib/han-secrets/han-secrets} +[ -x "$LAUNCHER" ] || LAUNCHER=deployment/secrets/han-secrets + +exec "$LAUNCHER" run --config "$CONFIG_FILE" -- \ + docker compose --env-file "$CONFIG_FILE" "$@" diff --git a/codebase/backend/deployment/secrets/han-secrets b/codebase/backend/deployment/secrets/han-secrets new file mode 100644 index 0000000..8e6e8f5 --- /dev/null +++ b/codebase/backend/deployment/secrets/han-secrets @@ -0,0 +1,175 @@ +#!/usr/bin/env python3 +"""Synchronize runtime secrets and execute a command without exporting their values.""" + +from __future__ import annotations + +import argparse +import json +import os +import subprocess +import sys +import tempfile +from pathlib import Path + +from secrets_loader import LoaderError, load_json, run + + +def load_public_config(path: Path) -> dict[str, str]: + values: dict[str, str] = {} + try: + lines = path.read_text(encoding="utf-8").splitlines() + except OSError as exc: + raise LoaderError( + f"cannot read non-secret config: {exc.strerror or exc.__class__.__name__}" + ) from None + for number, raw in enumerate(lines, 1): + line = raw.strip() + if not line or line.startswith("#"): + continue + if "=" not in line: + raise LoaderError(f"non-secret config has invalid syntax at line {number}") + key, value = line.split("=", 1) + key = key.strip() + if not key or key in values: + raise LoaderError(f"non-secret config has an invalid key at line {number}") + values[key] = value.strip() + return values + + +def loader_config_path( + public: dict[str, str], + explicit: Path | None, + environ: dict[str, str], +) -> tuple[str, Path]: + source = public.get("SECRETS_SOURCE") + if source not in {"selectel", "file"}: + raise LoaderError("SECRETS_SOURCE must explicitly be selectel or file") + if explicit is not None: + return source, explicit + override = environ.get(f"HAN_SECRETS_{source.upper()}_CONFIG") + if override: + return source, Path(override) + environment = public.get("APP_ENV", "production") + return source, Path(f"/etc/han/secrets/{environment}.{source}.json") + + +def prepare_environment( + config_path: Path, + source: str, + environ: dict[str, str], + *, + synchronize: bool, +) -> dict[str, str]: + document = load_json(config_path) + if document.get("mode") != source: + raise LoaderError("selected loader configuration mode does not match SECRETS_SOURCE") + runtime = document.get("runtime_dir") + if not isinstance(runtime, str) or not Path(runtime).is_absolute(): + raise LoaderError("loader configuration has an invalid runtime_dir") + runtime_dir = Path(runtime) + manifest = runtime_dir / "manifest" + state_path = runtime_dir / "state.json" + consumers: list[str] = [] + if synchronize or (source == "file" and not state_path.is_file()): + consumers = run(config_path, environ=environ) + state = { + "version": 1, + "source": source, + "loader_config": str(config_path.resolve()), + } + descriptor, temporary = tempfile.mkstemp( + prefix=".state.", suffix=".tmp", dir=runtime_dir + ) + temporary_path = Path(temporary) + try: + os.chmod(temporary_path, 0o600) + with os.fdopen(descriptor, "w", encoding="utf-8") as stream: + descriptor = -1 + json.dump(state, stream, separators=(",", ":")) + stream.write("\n") + stream.flush() + os.fsync(stream.fileno()) + os.replace(temporary_path, state_path) + except BaseException: + if descriptor >= 0: + os.close(descriptor) + temporary_path.unlink(missing_ok=True) + raise + elif not state_path.is_file(): + raise LoaderError( + "runtime secrets are not synchronized; restart han-secrets systemd unit" + ) + else: + state = load_json(state_path) + if ( + state.get("version") != 1 + or state.get("source") != source + or state.get("loader_config") != str(config_path.resolve()) + ): + raise LoaderError( + "runtime secret state does not match selected source/config; synchronize first" + ) + if not manifest.is_file(): + raise LoaderError("runtime secret manifest was not materialized") + manifest_entries: dict[str, str] = {} + for line in manifest.read_text(encoding="utf-8").splitlines(): + key, value_path = line.split("=", 1) + manifest_entries[key] = value_path + specs = document.get("secrets") + if not isinstance(specs, dict) or set(manifest_entries) != set(specs): + raise LoaderError("runtime secret manifest does not match loader configuration") + child = dict(environ) + child["HAN_SECRETS_ACTIVE"] = "1" + child["HAN_RUNTIME_SECRET_DIR"] = str(runtime_dir) + child["HAN_RUNTIME_SECRET_MANIFEST"] = str(manifest) + for key, value_path in manifest_entries.items(): + if not Path(value_path).is_file(): + raise LoaderError("runtime secret manifest references a missing file") + child[f"{key}_FILE"] = value_path + if synchronize or consumers: + print( + f"han-secrets: synchronized {len(consumers)} service scope(s) from {source}", + file=sys.stderr, + ) + return child + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("action", choices=("sync", "run")) + parser.add_argument("--config", type=Path, default=Path(".env")) + parser.add_argument("--loader-config", type=Path) + arguments, command = parser.parse_known_args(argv) + if command and command[0] == "--": + command.pop(0) + if arguments.action == "run" and not command: + parser.error("run requires a command after --") + if arguments.action == "sync" and command: + parser.error("sync does not accept a command") + + environment = dict(os.environ) + try: + public = load_public_config(arguments.config) + source, loader_config = loader_config_path( + public, arguments.loader_config, environment + ) + child = prepare_environment( + loader_config, + source, + environment, + synchronize=arguments.action == "sync", + ) + except (LoaderError, OSError, ValueError, json.JSONDecodeError) as exc: + message = str(exc) if isinstance(exc, LoaderError) else exc.__class__.__name__ + print(f"han-secrets: {message}", file=sys.stderr) + return 1 + if arguments.action == "sync": + return 0 + if os.name == "nt": + return subprocess.call(command, env=child) + os.execvpe(command[0], command, child) + return 127 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/codebase/backend/deployment/secrets/han-secrets@.service b/codebase/backend/deployment/secrets/han-secrets@.service new file mode 100644 index 0000000..eb07efd --- /dev/null +++ b/codebase/backend/deployment/secrets/han-secrets@.service @@ -0,0 +1,42 @@ +[Unit] +Description=Materialize HAN service secrets (%i) +Documentation=file:/usr/local/share/doc/han-secrets/SELECTEL_RUNBOOK.ru.md +Wants=network-online.target +After=network-online.target +Before=han-stack@%i.service + +[Service] +Type=oneshot +User=root +Group=root +UMask=0077 +RuntimeDirectory=han-chat/secrets +RuntimeDirectoryMode=0700 +ExecStart=/usr/bin/python3 /usr/local/lib/han-secrets/han-secrets sync --config /opt/han-chat/backend/.env +LoadCredentialEncrypted=selectel-service-user-password:/etc/han/credentials/%i.selectel-password.cred +RemainAfterExit=yes +StandardOutput=null +StandardError=journal +SyslogIdentifier=han-secrets-%i +NoNewPrivileges=yes +PrivateTmp=yes +PrivateDevices=yes +ProtectSystem=strict +ProtectHome=yes +ProtectKernelTunables=yes +ProtectKernelModules=yes +ProtectKernelLogs=yes +ProtectControlGroups=yes +ProtectClock=yes +RestrictRealtime=yes +RestrictSUIDSGID=yes +LockPersonality=yes +MemoryDenyWriteExecute=yes +LimitCORE=0 +SystemCallArchitectures=native +RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 +CapabilityBoundingSet= +AmbientCapabilities= + +[Install] +WantedBy=multi-user.target diff --git a/codebase/backend/deployment/secrets/secrets_loader.py b/codebase/backend/deployment/secrets/secrets_loader.py new file mode 100644 index 0000000..c2fd271 --- /dev/null +++ b/codebase/backend/deployment/secrets/secrets_loader.py @@ -0,0 +1,682 @@ +#!/usr/bin/env python3 +"""Materialize narrowly scoped service dotenv files from Selectel Secrets Manager.""" + +from __future__ import annotations + +import argparse +import base64 +import binascii +import json +import os +import random +import re +import ssl +import stat +import sys +import tempfile +import time +import urllib.error +import urllib.parse +import urllib.request +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Callable, Mapping, NoReturn + +DEFAULT_IDENTITY_URL = "https://cloud.api.selcloud.ru/identity/v3/auth/tokens" +MAX_CONFIG_BYTES = 1_048_576 +MAX_HTTP_BYTES = 1_048_576 +MAX_SECRET_BYTES = 65_536 +RETRYABLE_STATUS = frozenset({408, 425, 429, 500, 502, 503, 504}) +ENV_NAME_RE = re.compile(r"^[A-Z][A-Z0-9_]*$") +SERVICE_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_.-]*$") +DOTENV_LINE_RE = re.compile(r"^([A-Z][A-Z0-9_]*)=(.*)$") + + +class LoaderError(Exception): + """An expected, already-redacted loader failure.""" + + +def fail(message: str) -> NoReturn: + raise LoaderError(message) + + +def _object(value: Any, label: str) -> dict[str, Any]: + if not isinstance(value, dict): + fail(f"{label} must be an object") + return value + + +def _only_keys(value: Mapping[str, Any], allowed: set[str], label: str) -> None: + unknown = sorted(set(value) - allowed) + if unknown: + fail(f"{label} contains unsupported fields: {', '.join(unknown)}") + + +def _required_string(value: Mapping[str, Any], key: str, label: str) -> str: + item = value.get(key) + if not isinstance(item, str) or not item: + fail(f"{label}.{key} must be a non-empty string") + return item + + +def _bounded_int(value: Any, label: str, minimum: int, maximum: int) -> int: + if isinstance(value, bool) or not isinstance(value, int) or not minimum <= value <= maximum: + fail(f"{label} must be an integer from {minimum} through {maximum}") + return value + + +def read_limited(path: Path, limit: int, label: str) -> bytes: + try: + with path.open("rb") as stream: + data = stream.read(limit + 1) + except OSError as exc: + fail(f"cannot read {label}: {exc.strerror or exc.__class__.__name__}") + if len(data) > limit: + fail(f"{label} exceeds {limit} bytes") + return data + + +def require_private_regular_file(path: Path, label: str) -> None: + try: + metadata = path.lstat() + except OSError as exc: + fail(f"cannot inspect {label}: {exc.strerror or exc.__class__.__name__}") + if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode): + fail(f"{label} must be a regular file and not a symlink") + if os.name != "nt" and stat.S_IMODE(metadata.st_mode) & 0o077: + fail(f"{label} must not be accessible by group or other users") + + +def load_json(path: Path) -> dict[str, Any]: + raw = read_limited(path, MAX_CONFIG_BYTES, "configuration") + try: + document = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + fail("configuration is not valid UTF-8 JSON") + return _object(document, "configuration") + + +def credential_value(selectel: Mapping[str, Any], environ: Mapping[str, str]) -> str: + methods = sum(key in selectel for key in ("password_file", "password_env")) + if methods != 1: + fail("selectel must set exactly one of password_file or password_env") + if "password_env" in selectel: + variable = _required_string(selectel, "password_env", "selectel") + if not ENV_NAME_RE.fullmatch(variable): + fail("selectel.password_env is not a valid environment variable name") + value = environ.get(variable) + if value is None or not value: + fail(f"credential environment variable {variable} is not set") + return value + + configured = Path(_required_string(selectel, "password_file", "selectel")) + if configured.is_absolute(): + path = configured + else: + directory = environ.get("CREDENTIALS_DIRECTORY") + if not directory: + fail("relative password_file requires CREDENTIALS_DIRECTORY") + path = Path(directory) / configured + require_private_regular_file(path, "credential") + raw = read_limited(path, 16_384, "credential") + try: + value = raw.decode("utf-8") + except UnicodeDecodeError: + fail("credential is not valid UTF-8") + value = value.removesuffix("\n").removesuffix("\r") + if not value or "\n" in value or "\r" in value or "\x00" in value: + fail("credential must contain exactly one non-empty text line") + return value + + +class NoRedirect(urllib.request.HTTPRedirectHandler): + def redirect_request(self, req: Any, fp: Any, code: int, msg: str, headers: Any, newurl: str) -> None: + return None + + +@dataclass(frozen=True) +class HTTPResult: + status: int + headers: Mapping[str, str] + body: bytes + + +class HTTPClient: + def __init__( + self, + *, + timeout: float, + retries: int, + max_response_bytes: int, + cafile: str | None = None, + opener: Any | None = None, + sleeper: Callable[[float], None] = time.sleep, + jitter: Callable[[], float] = random.random, + ) -> None: + self.timeout = timeout + self.retries = retries + self.max_response_bytes = max_response_bytes + self.sleeper = sleeper + self.jitter = jitter + if opener is None: + try: + context = ssl.create_default_context(cafile=cafile) + except (OSError, ssl.SSLError) as exc: + fail(f"cannot initialize TLS trust store: {exc.__class__.__name__}") + self.opener = urllib.request.build_opener( + urllib.request.HTTPSHandler(context=context), NoRedirect() + ) + else: + self.opener = opener + + def request( + self, + method: str, + url: str, + *, + headers: Mapping[str, str] | None = None, + body: bytes | None = None, + expected: frozenset[int], + ) -> HTTPResult: + parsed = urllib.parse.urlsplit(url) + if parsed.scheme != "https" or not parsed.netloc or parsed.username or parsed.password: + fail("provider endpoint must be an HTTPS URL without embedded credentials") + request = urllib.request.Request( + url, data=body, headers=dict(headers or {}), method=method + ) + for attempt in range(self.retries + 1): + try: + with self.opener.open(request, timeout=self.timeout) as response: + status = int(response.status) + content_length = response.headers.get("Content-Length") + if content_length: + try: + if int(content_length) > self.max_response_bytes: + fail("provider response exceeds configured limit") + except ValueError: + fail("provider returned an invalid Content-Length") + response_body = response.read(self.max_response_bytes + 1) + if len(response_body) > self.max_response_bytes: + fail("provider response exceeds configured limit") + if status not in expected: + fail(f"provider request failed with HTTP {status}") + return HTTPResult(status, response.headers, response_body) + except urllib.error.HTTPError as exc: + status = int(exc.code) + if status not in RETRYABLE_STATUS or attempt >= self.retries: + fail(f"provider request failed with HTTP {status}") + except (urllib.error.URLError, TimeoutError, OSError): + if attempt >= self.retries: + fail("provider request failed after retries") + delay = min(8.0, 0.25 * (2**attempt)) * (0.5 + self.jitter()) + self.sleeper(delay) + fail("provider request failed") + + +def parse_json_response(result: HTTPResult, label: str) -> dict[str, Any]: + try: + return _object(json.loads(result.body.decode("utf-8")), label) + except (UnicodeDecodeError, json.JSONDecodeError): + fail(f"{label} is not valid JSON") + + +def project_token_and_catalog( + client: HTTPClient, selectel: Mapping[str, Any], password: str +) -> tuple[str, list[Any]]: + identity_url = selectel.get("identity_url", DEFAULT_IDENTITY_URL) + if not isinstance(identity_url, str): + fail("selectel.identity_url must be a string") + account_id = _required_string(selectel, "account_id", "selectel") + username = _required_string(selectel, "username", "selectel") + project_name = _required_string(selectel, "project_name", "selectel") + payload = { + "auth": { + "identity": { + "methods": ["password"], + "password": { + "user": { + "name": username, + "domain": {"name": account_id}, + "password": password, + } + }, + }, + "scope": { + "project": { + "name": project_name, + "domain": {"name": account_id}, + } + }, + } + } + result = client.request( + "POST", + identity_url, + headers={"Content-Type": "application/json", "Accept": "application/json"}, + body=json.dumps(payload, separators=(",", ":")).encode("utf-8"), + expected=frozenset({201}), + ) + token = result.headers.get("X-Subject-Token") + if not isinstance(token, str) or not token: + fail("identity response omitted X-Subject-Token") + document = parse_json_response(result, "identity response") + token_data = document.get("token") + if not isinstance(token_data, dict): + fail("identity response omitted token metadata") + project = token_data.get("project") + if not isinstance(project, dict) or not project.get("id"): + fail("identity token is not project-scoped") + catalog = token_data.get("catalog") + if not isinstance(catalog, list): + fail("identity response omitted service catalog") + return token, catalog + + +def secrets_endpoint(catalog: list[Any], region: str, interface: str) -> str: + matches: list[str] = [] + for service in catalog: + if not isinstance(service, dict) or service.get("type") != "secrets-manager": + continue + endpoints = service.get("endpoints") + if not isinstance(endpoints, list): + continue + for endpoint in endpoints: + if ( + isinstance(endpoint, dict) + and endpoint.get("region") == region + and endpoint.get("interface") == interface + and isinstance(endpoint.get("url"), str) + ): + matches.append(endpoint["url"].rstrip("/")) + if len(matches) != 1: + fail("service catalog did not contain exactly one matching Secrets Manager endpoint") + return matches[0] + + +def decode_secret(document: Mapping[str, Any], name: str, limit: int) -> bytes: + # GET /v1/{name} returns the current value inside ``version`` while + # GET /v1/{name}/versions/{id} returns a version object directly. + payload: Mapping[str, Any] = document + version = document.get("version") + if isinstance(version, dict): + payload = version + encoded = payload.get("value") + if not isinstance(encoded, str): + fail(f"secret {name} response omitted base64 value") + try: + value = base64.b64decode(encoded, validate=True) + except (binascii.Error, ValueError): + fail(f"secret {name} has invalid base64 encoding") + if not value: + fail(f"secret {name} is empty") + if len(value) > limit: + fail(f"secret {name} exceeds its configured limit") + if b"\x00" in value or b"\n" in value or b"\r" in value: + fail(f"secret {name} cannot be represented as a dotenv value") + return value + + +def fetch_selectel( + config: Mapping[str, Any], + specs: Mapping[str, Mapping[str, Any]], + environ: Mapping[str, str], + client_factory: Callable[..., HTTPClient] = HTTPClient, +) -> dict[str, bytes]: + selectel = _object(config.get("selectel"), "selectel") + _only_keys( + selectel, + { + "account_id", + "username", + "project_name", + "region", + "interface", + "password_file", + "password_env", + "identity_url", + "secrets_url", + "ca_file", + }, + "selectel", + ) + http = _object(config.get("http", {}), "http") + _only_keys(http, {"timeout_seconds", "retries", "max_response_bytes"}, "http") + timeout = http.get("timeout_seconds", 10) + if isinstance(timeout, bool) or not isinstance(timeout, (int, float)) or not 0.1 <= timeout <= 60: + fail("http.timeout_seconds must be from 0.1 through 60") + retries = _bounded_int(http.get("retries", 3), "http.retries", 0, 8) + response_limit = _bounded_int( + http.get("max_response_bytes", MAX_HTTP_BYTES), + "http.max_response_bytes", + 1024, + 4 * MAX_HTTP_BYTES, + ) + cafile = selectel.get("ca_file") + if cafile is not None and (not isinstance(cafile, str) or not cafile): + fail("selectel.ca_file must be a non-empty string") + client = client_factory( + timeout=float(timeout), + retries=retries, + max_response_bytes=response_limit, + cafile=cafile, + ) + password = credential_value(selectel, environ) + token, catalog = project_token_and_catalog(client, selectel, password) + region = _required_string(selectel, "region", "selectel") + interface = selectel.get("interface", "public") + if interface not in {"public", "internal"}: + fail("selectel.interface must be public or internal") + override = selectel.get("secrets_url") + if override is not None and (not isinstance(override, str) or not override): + fail("selectel.secrets_url must be a non-empty string") + base_url = override.rstrip("/") if override else secrets_endpoint(catalog, region, interface) + + values: dict[str, bytes] = {} + fetched: dict[tuple[str, int | None], bytes] = {} + for canonical, spec in specs.items(): + if "literal" in spec: + values[canonical] = b"" + continue + remote = _required_string(spec, "remote", f"secrets.{canonical}") + version = spec.get("version") + version_id: int | None = None + if version is not None: + version_id = _bounded_int( + version, f"secrets.{canonical}.version", 1, 2_147_483_647 + ) + cache_key = (remote, version_id) + if cache_key not in fetched: + path = f"/v1/{urllib.parse.quote(remote, safe='')}" + if version_id is not None: + path += f"/versions/{version_id}" + try: + result = client.request( + "GET", + base_url + path, + headers={"X-Auth-Token": token, "Accept": "application/json"}, + expected=frozenset({200}), + ) + document = parse_json_response(result, f"secret {canonical} response") + fetched[cache_key] = decode_secret( + document, canonical, MAX_SECRET_BYTES + ) + except LoaderError as exc: + fail(f"cannot load {canonical}: {exc}") + limit = _bounded_int( + spec.get("max_bytes", MAX_SECRET_BYTES), + f"secrets.{canonical}.max_bytes", + 1, + MAX_SECRET_BYTES, + ) + value = fetched[cache_key] + if len(value) > limit: + fail(f"secret {canonical} exceeds its configured limit") + values[canonical] = value + return values + + +def parse_dotenv(path: Path, expected: set[str], max_bytes: int) -> dict[str, bytes]: + require_private_regular_file(path, "fallback dotenv") + raw = read_limited(path, max_bytes, "fallback dotenv") + try: + text = raw.decode("utf-8") + except UnicodeDecodeError: + fail("fallback dotenv is not valid UTF-8") + values: dict[str, bytes] = {} + for number, line in enumerate(text.splitlines(), 1): + if not line or line.startswith("#"): + continue + match = DOTENV_LINE_RE.fullmatch(line) + if not match: + fail(f"fallback dotenv has invalid syntax at line {number}") + name, encoded_value = match.groups() + if name not in expected: + fail(f"fallback dotenv contains undeclared key {name}") + if name in values: + fail(f"fallback dotenv contains duplicate key {name}") + if encoded_value.startswith('"'): + try: + decoded = json.loads(encoded_value) + except json.JSONDecodeError: + fail(f"fallback dotenv has invalid quoted value at line {number}") + if not isinstance(decoded, str): + fail(f"fallback dotenv has invalid quoted value at line {number}") + value = decoded.encode("utf-8") + elif encoded_value.startswith("'"): + if len(encoded_value) < 2 or not encoded_value.endswith("'"): + fail(f"fallback dotenv has invalid quoted value at line {number}") + value = encoded_value[1:-1].encode("utf-8") + else: + if any(character.isspace() for character in encoded_value) or any( + character in encoded_value for character in ("'", '"', "`", "$", "\\") + ): + fail(f"fallback dotenv requires quoting at line {number}") + value = encoded_value.encode("utf-8") + if not value or b"\x00" in value or b"\n" in value or b"\r" in value: + fail(f"fallback dotenv has an empty or unsafe value for {name}") + values[name] = value + missing = sorted(expected - set(values)) + if missing: + fail(f"fallback dotenv is missing declared keys: {', '.join(missing)}") + return values + + +def validate_specs(config: Mapping[str, Any]) -> dict[str, dict[str, Any]]: + raw_specs = _object(config.get("secrets"), "secrets") + if not raw_specs: + fail("secrets must not be empty") + specs: dict[str, dict[str, Any]] = {} + for canonical, raw_spec in raw_specs.items(): + if not isinstance(canonical, str) or not ENV_NAME_RE.fullmatch(canonical): + fail("every canonical secret name must be an uppercase environment name") + spec = _object(raw_spec, f"secrets.{canonical}") + _only_keys( + spec, + {"remote", "consumers", "max_bytes", "version", "literal"}, + f"secrets.{canonical}", + ) + has_remote = "remote" in spec + has_literal = "literal" in spec + if has_remote == has_literal: + fail(f"secrets.{canonical} must set exactly one of remote or literal") + if has_literal and spec["literal"] != "": + fail(f"secrets.{canonical}.literal may only be an empty string") + consumers = spec.get("consumers") + if not isinstance(consumers, list) or not consumers: + fail(f"secrets.{canonical}.consumers must be a non-empty array") + if len(consumers) != len(set(item for item in consumers if isinstance(item, str))): + fail(f"secrets.{canonical}.consumers contains duplicates or invalid values") + for consumer in consumers: + if not isinstance(consumer, str) or not SERVICE_NAME_RE.fullmatch(consumer): + fail(f"secrets.{canonical}.consumers contains an invalid service name") + specs[canonical] = spec + return specs + + +def dotenv_quote(value: bytes, name: str) -> str: + try: + text = value.decode("utf-8") + except UnicodeDecodeError: + fail(f"secret {name} is not valid UTF-8") + return json.dumps(text, ensure_ascii=False) + + +def materialize(runtime_dir: Path, specs: Mapping[str, Mapping[str, Any]], values: Mapping[str, bytes]) -> None: + try: + runtime_dir.mkdir(mode=0o700, parents=True, exist_ok=True) + if runtime_dir.is_symlink(): + fail("runtime directory must not be a symlink") + os.chmod(runtime_dir, 0o700) + except OSError as exc: + fail(f"cannot prepare runtime directory: {exc.strerror or exc.__class__.__name__}") + consumers = sorted( + {consumer for spec in specs.values() for consumer in spec["consumers"]} + ) + staged: list[tuple[Path, Path]] = [] + try: + for consumer in consumers: + lines = [ + f"{name}={dotenv_quote(values[name], name)}\n" + for name, spec in sorted(specs.items()) + if consumer in spec["consumers"] + ] + descriptor, temporary = tempfile.mkstemp( + prefix=f".{consumer}.", suffix=".tmp", dir=runtime_dir + ) + temporary_path = Path(temporary) + try: + os.chmod(temporary_path, 0o600) + stream = os.fdopen(descriptor, "w", encoding="utf-8", newline="\n") + descriptor = -1 + with stream: + stream.writelines(lines) + stream.flush() + os.fsync(stream.fileno()) + except BaseException: + if descriptor >= 0: + os.close(descriptor) + raise + staged.append((temporary_path, runtime_dir / f"{consumer}.env")) + for temporary_path, destination in staged: + os.replace(temporary_path, destination) + value_paths: dict[str, Path] = {} + for name, value in sorted(values.items()): + descriptor, temporary = tempfile.mkstemp( + prefix=f".{name}.", suffix=".tmp", dir=runtime_dir + ) + temporary_path = Path(temporary) + try: + # Compose implements local secrets as bind mounts. The protected + # 0700 parent prevents host users from traversing to this 0444 + # file while allowing a non-root container UID to read its mount. + os.chmod(temporary_path, 0o444) + with os.fdopen(descriptor, "wb") as stream: + descriptor = -1 + stream.write(value) + stream.flush() + os.fsync(stream.fileno()) + except BaseException: + if descriptor >= 0: + os.close(descriptor) + temporary_path.unlink(missing_ok=True) + raise + destination = runtime_dir / name + os.replace(temporary_path, destination) + value_paths[name] = destination + + manifest_lines = [ + f"{name}={path.resolve()}\n" for name, path in sorted(value_paths.items()) + ] + descriptor, temporary = tempfile.mkstemp( + prefix=".manifest.", suffix=".tmp", dir=runtime_dir + ) + manifest_path = Path(temporary) + try: + os.chmod(manifest_path, 0o600) + with os.fdopen(descriptor, "w", encoding="utf-8", newline="\n") as stream: + descriptor = -1 + stream.writelines(manifest_lines) + stream.flush() + os.fsync(stream.fileno()) + except BaseException: + if descriptor >= 0: + os.close(descriptor) + manifest_path.unlink(missing_ok=True) + raise + os.replace(manifest_path, runtime_dir / "manifest") + if hasattr(os, "O_DIRECTORY"): + directory_fd = os.open(runtime_dir, os.O_RDONLY | os.O_DIRECTORY) + try: + os.fsync(directory_fd) + finally: + os.close(directory_fd) + except OSError as exc: + fail(f"cannot atomically materialize service files: {exc.strerror or exc.__class__.__name__}") + finally: + for temporary_path, _ in staged: + try: + temporary_path.unlink(missing_ok=True) + except OSError: + pass + + +def run( + config_path: Path, + *, + runtime_override: Path | None = None, + environ: Mapping[str, str] | None = None, + client_factory: Callable[..., HTTPClient] = HTTPClient, +) -> list[str]: + os.umask(0o077) + environment = os.environ if environ is None else environ + config = load_json(config_path) + _only_keys(config, {"version", "mode", "runtime_dir", "http", "selectel", "file", "secrets"}, "configuration") + if config.get("version") != 1: + fail("configuration.version must be 1") + mode = config.get("mode") + if mode not in {"selectel", "file"}: + fail("configuration.mode must explicitly be selectel or file") + specs = validate_specs(config) + if runtime_override is None: + configured_runtime = config.get("runtime_dir") + if not isinstance(configured_runtime, str) or not configured_runtime: + fail("configuration.runtime_dir must be a non-empty string") + runtime_dir = Path(configured_runtime) + else: + runtime_dir = runtime_override + if not runtime_dir.is_absolute(): + fail("runtime directory must be an absolute path") + + if mode == "selectel": + if "file" in config: + fail("file settings are forbidden in selectel mode") + values = fetch_selectel(config, specs, environment, client_factory) + else: + if "selectel" in config or "http" in config: + fail("selectel and http settings are forbidden in file mode") + file_config = _object(config.get("file"), "file") + _only_keys(file_config, {"path", "max_bytes"}, "file") + source = Path(_required_string(file_config, "path", "file")) + if not source.is_absolute(): + fail("file.path must be absolute") + max_bytes = _bounded_int( + file_config.get("max_bytes", MAX_CONFIG_BYTES), + "file.max_bytes", + 1, + 4 * MAX_CONFIG_BYTES, + ) + expected = {name for name, spec in specs.items() if "literal" not in spec} + values = parse_dotenv(source, expected, max_bytes) + values.update( + {name: b"" for name, spec in specs.items() if "literal" in spec} + ) + for canonical, value in values.items(): + limit = _bounded_int( + specs[canonical].get("max_bytes", MAX_SECRET_BYTES), + f"secrets.{canonical}.max_bytes", + 1, + MAX_SECRET_BYTES, + ) + if len(value) > limit: + fail(f"secret {canonical} exceeds its configured limit") + + materialize(runtime_dir, specs, values) + return sorted({consumer for spec in specs.values() for consumer in spec["consumers"]}) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description="Materialize per-service secret dotenv files") + parser.add_argument("--config", required=True, type=Path) + parser.add_argument("--runtime-dir", type=Path) + arguments = parser.parse_args(argv) + try: + consumers = run(arguments.config, runtime_override=arguments.runtime_dir) + except LoaderError as exc: + print(f"secrets-loader: {exc}", file=sys.stderr) + return 1 + print(f"secrets-loader: materialized {len(consumers)} service file(s)", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/codebase/backend/infra/compose/application.yml b/codebase/backend/infra/compose/application.yml index 1694b30..ad927e4 100644 --- a/codebase/backend/infra/compose/application.yml +++ b/codebase/backend/infra/compose/application.yml @@ -1,28 +1,59 @@ -x-no-sms-secrets: &no-sms-secrets - SMS_DATABASE_URL: "" - SMS_SERVICE_TOKEN: "" - KEYCLOAK_SMS_SERVICE_TOKEN: "" - IDGTL_SMS_API_KEY: "" - IDGTL_SMS_CALLBACK_USERNAME: "" - IDGTL_SMS_CALLBACK_PASSWORD: "" - KEYCLOAK_YANDEX_CAPTCHA_ENABLED: "" - KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: "" - KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: "" +x-api-secret-environment: &api-secret-environment + HAN_SECRET_VARS: >- + DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN + BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN + KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY + CURSOR_HMAC_SECRET + DATABASE_URL_FILE: /run/secrets/api_database_url + REDIS_URL_FILE: /run/secrets/api_redis_url + REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url + MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token + BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token + BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token + KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token + SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key + SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key + CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret + +x-api-secrets: &api-secrets + - api_database_url + - api_redis_url + - api_redis_realtime_url + - message_safety_service_token + - bitrix_local_app_internal_token + - bitrix_api_inbox_token + - keycloak_settings_bridge_token + - selectel_s3_access_key + - selectel_s3_secret_key + - cursor_hmac_secret x-api-runtime: &api-runtime build: context: ../../api-backend image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local} - env_file: - - path: ../../.env - required: false environment: - <<: *no-sms-secrets - NOTIFICATIONS_TOKEN_PRODUCER_TEST: ${NOTIFICATIONS_TOKEN_PRODUCER_TEST:?NOTIFICATIONS_TOKEN_PRODUCER_TEST is required} + <<: *api-secret-environment APP_ENV: ${APP_ENV:-production-like} + LOG_LEVEL: ${LOG_LEVEL:-INFO} RELEASE_VERSION: ${RELEASE_VERSION:-unknown} + KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL} + KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL:-http://keycloak:8080/auth} + KEYCLOAK_REALM: ${KEYCLOAK_REALM:-han-chat} + KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE:-han-chat-api} + MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL:-http://message-safety:8080} + MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5} + MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2} + MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300} + BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL:-http://bitrix-local-app:8080} + BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20} + SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL} + SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS} + SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS} + SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE} + TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-127.0.0.1/32} OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317} OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_API:-api-backend} + secrets: *api-secrets volumes: - type: bind source: ${PG_CA_HOST_PATH} @@ -30,32 +61,48 @@ x-api-runtime: &api-runtime read_only: true networks: [backend, observability, egress] security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} logging: driver: json-file options: {max-size: "50m", max-file: "5"} +x-sms-api-environment: &sms-api-environment + HAN_SECRET_VARS: >- + SMS_DATABASE_URL SMS_SERVICE_TOKEN IDGTL_SMS_CALLBACK_USERNAME + IDGTL_SMS_CALLBACK_PASSWORD + SMS_DATABASE_URL_FILE: /run/secrets/sms_database_url + SMS_SERVICE_TOKEN_FILE: /run/secrets/sms_service_token + IDGTL_SMS_CALLBACK_USERNAME_FILE: /run/secrets/idgtl_sms_callback_username + IDGTL_SMS_CALLBACK_PASSWORD_FILE: /run/secrets/idgtl_sms_callback_password + IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru} + IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL} + LOG_LEVEL: ${LOG_LEVEL:-INFO} + APP_ENV: ${APP_ENV:-production-like} + RELEASE_VERSION: ${RELEASE_VERSION:-unknown} + OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317} + OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_API:-sms-service} + +x-sms-api-secrets: &sms-api-secrets + - sms_database_url + - sms_service_token + - idgtl_sms_callback_username + - idgtl_sms_callback_password + x-sms-runtime: &sms-runtime build: context: ../../sms-service image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local} - environment: - SMS_DATABASE_URL: ${SMS_DATABASE_URL} - SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN} - IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru} - IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL} - IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME} - IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD} - LOG_LEVEL: ${LOG_LEVEL:-INFO} - APP_ENV: ${APP_ENV:-production-like} - RELEASE_VERSION: ${RELEASE_VERSION:-unknown} - OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317} - OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_API:-sms-service} + environment: *sms-api-environment + secrets: *sms-api-secrets volumes: - type: bind source: ${PG_CA_HOST_PATH} target: /run/secrets/pg-ca.pem read_only: true security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} logging: driver: json-file options: {max-size: "50m", max-file: "5"} @@ -80,21 +127,31 @@ services: - /tmp:size=8m,mode=1777 cap_drop: ["ALL"] security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} keycloak: build: context: ../../keycloak image: ${KEYCLOAK_IMAGE:-han-chat-keycloak:local} - env_file: - - path: ../../.env - required: false environment: - <<: *no-sms-secrets + HAN_SECRET_VARS: >- + KC_DB_PASSWORD KC_BOOTSTRAP_ADMIN_PASSWORD + KEYCLOAK_OTP_MOCK_CODE + KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY + KEYCLOAK_OTP_HMAC_KEY KEYCLOAK_SETTINGS_BRIDGE_TOKEN + KEYCLOAK_SMS_SERVICE_TOKEN + KC_DB_PASSWORD_FILE: /run/secrets/keycloak_db_password + KC_BOOTSTRAP_ADMIN_PASSWORD_FILE: /run/secrets/keycloak_admin_password + KEYCLOAK_OTP_MOCK_CODE_FILE: /run/secrets/keycloak_otp_mock_code + KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: /run/secrets/keycloak_yandex_captcha_server_key + KEYCLOAK_OTP_HMAC_KEY_FILE: /run/secrets/keycloak_otp_hmac_key + KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token + KEYCLOAK_SMS_SERVICE_TOKEN_FILE: /run/secrets/keycloak_sms_service_token KC_DB: postgres KC_DB_URL: ${KEYCLOAK_DB_URL} - KC_DB_SCHEMA: ${KEYCLOAK_DB_SCHEMA:-keycloak} KC_DB_USERNAME: ${KEYCLOAK_DB_USERNAME} - KC_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD} + KC_DB_SCHEMA: ${KEYCLOAK_DB_SCHEMA:-keycloak} KC_PROXY_HEADERS: xforwarded KC_HTTP_ENABLED: "true" KC_HTTP_RELATIVE_PATH: /auth @@ -102,18 +159,20 @@ services: KC_METRICS_ENABLED: "true" KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL} KC_BOOTSTRAP_ADMIN_USERNAME: ${KEYCLOAK_ADMIN} - KC_BOOTSTRAP_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD} KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-false} - KEYCLOAK_OTP_MOCK_CODE: ${KEYCLOAK_OTP_MOCK_CODE:-} KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false} KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-} - KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY:-} - KEYCLOAK_OTP_HMAC_KEY: ${KEYCLOAK_OTP_HMAC_KEY:?KEYCLOAK_OTP_HMAC_KEY is required} KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300} KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp} - KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN:?KEYCLOAK_SETTINGS_BRIDGE_TOKEN is required} KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080} - KEYCLOAK_SMS_SERVICE_TOKEN: ${KEYCLOAK_SMS_SERVICE_TOKEN:?KEYCLOAK_SMS_SERVICE_TOKEN is required} + secrets: + - keycloak_db_password + - keycloak_admin_password + - keycloak_otp_mock_code + - keycloak_yandex_captcha_server_key + - keycloak_otp_hmac_key + - keycloak_settings_bridge_token + - keycloak_sms_service_token command: ["start", "--optimized", "--import-realm"] expose: ["8080", "9000"] volumes: @@ -130,6 +189,8 @@ services: start_period: 60s restart: unless-stopped security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} logging: driver: json-file options: {max-size: "50m", max-file: "5"} @@ -148,22 +209,21 @@ services: sms-worker: <<: *sms-runtime - entrypoint: [] command: ["han-sms-worker"] environment: - SMS_DATABASE_URL: ${SMS_DATABASE_URL} - SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN} - IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru} - IDGTL_SMS_API_KEY: ${IDGTL_SMS_API_KEY:?IDGTL_SMS_API_KEY is required} - IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL} - IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME} - IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD} - LOG_LEVEL: ${LOG_LEVEL:-INFO} - APP_ENV: ${APP_ENV:-production-like} - RELEASE_VERSION: ${RELEASE_VERSION:-unknown} - OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317} + <<: *sms-api-environment + HAN_SECRET_VARS: >- + SMS_DATABASE_URL SMS_SERVICE_TOKEN IDGTL_SMS_API_KEY + IDGTL_SMS_CALLBACK_USERNAME IDGTL_SMS_CALLBACK_PASSWORD + IDGTL_SMS_API_KEY_FILE: /run/secrets/idgtl_sms_api_key OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_WORKER:-sms-worker} SMS_METRICS_PORT: ${SMS_METRICS_PORT:-9464} + secrets: + - sms_database_url + - sms_service_token + - idgtl_sms_api_key + - idgtl_sms_callback_username + - idgtl_sms_callback_password expose: ["9464"] networks: [backend, observability, egress] depends_on: @@ -180,17 +240,17 @@ services: build: context: ../../message-safety image: ${MESSAGE_SAFETY_IMAGE:-han-chat-message-safety:local} - env_file: - - path: ../../.env - required: false environment: - <<: *no-sms-secrets + HAN_SECRET_VARS: MESSAGE_SAFETY_REDIS_URL MESSAGE_SAFETY_SERVICE_TOKEN + MESSAGE_SAFETY_REDIS_URL_FILE: /run/secrets/message_safety_redis_url + MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token + APP_ENV: ${APP_ENV:-production-like} + MESSAGE_SAFETY_RULES_VERSION: ${MESSAGE_SAFETY_RULES_VERSION:-2026-01-01} + MESSAGE_SAFETY_TASK_TTL_SEC: ${MESSAGE_SAFETY_TASK_TTL_SEC:-900} + secrets: + - message_safety_redis_url + - message_safety_service_token expose: ["8080"] - volumes: - - type: bind - source: ${PG_CA_HOST_PATH} - target: /run/secrets/pg-ca.pem - read_only: true networks: [backend, observability, egress] depends_on: redis: {condition: service_healthy} @@ -202,6 +262,8 @@ services: start_period: 30s restart: unless-stopped security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} logging: driver: json-file options: {max-size: "50m", max-file: "5"} @@ -223,7 +285,6 @@ services: delivery-worker: <<: *api-runtime - entrypoint: [] command: ["han-delivery-worker"] depends_on: api-backend: {condition: service_healthy} @@ -238,7 +299,6 @@ services: safety-recovery-worker: <<: *api-runtime - entrypoint: [] command: ["han-safety-worker"] depends_on: api-backend: {condition: service_healthy} @@ -253,7 +313,6 @@ services: cleanup-worker: <<: *api-runtime - entrypoint: [] command: ["han-cleanup-worker"] depends_on: api-backend: {condition: service_healthy} @@ -267,7 +326,6 @@ services: notification-expire-worker: <<: *api-runtime - entrypoint: [] command: ["han-notification-expire-worker"] depends_on: api-backend: {condition: service_healthy} @@ -281,7 +339,6 @@ services: notification-draft-cleanup-worker: <<: *api-runtime - entrypoint: [] command: ["han-notification-draft-cleanup-worker"] depends_on: api-backend: {condition: service_healthy} @@ -298,11 +355,33 @@ services: build: context: ../../bitrix-local-app image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local} - env_file: - - path: ../../.env - required: false environment: - <<: *no-sms-secrets + HAN_SECRET_VARS: >- + BITRIX_DATABASE_URL BITRIX_CLIENT_SECRET + BITRIX_APPLICATION_TOKEN BITRIX_INTERNAL_API_TOKEN + BITRIX_API_FORWARD_TOKEN BITRIX_TOKEN_ENCRYPTION_KEY + BITRIX_DATABASE_URL_FILE: /run/secrets/bitrix_database_url + BITRIX_CLIENT_SECRET_FILE: /run/secrets/bitrix_client_secret + BITRIX_APPLICATION_TOKEN_FILE: /run/secrets/bitrix_application_token + BITRIX_INTERNAL_API_TOKEN_FILE: /run/secrets/bitrix_internal_api_token + BITRIX_API_FORWARD_TOKEN_FILE: /run/secrets/bitrix_api_forward_token + BITRIX_TOKEN_ENCRYPTION_KEY_FILE: /run/secrets/bitrix_token_encryption_key + APP_ENV: ${APP_ENV:-production-like} + BITRIX_CLIENT_ID: ${BITRIX_CLIENT_ID} + BITRIX_CONNECTOR_ID: ${BITRIX_CONNECTOR_ID:-han_mobile_app} + BITRIX_CONNECTOR_NAME: ${BITRIX_CONNECTOR_NAME:-HAN Mobile App} + BITRIX_OPEN_LINE_ID: ${BITRIX_OPEN_LINE_ID} + BITRIX_EXPECTED_DOMAIN: ${BITRIX_EXPECTED_DOMAIN} + BITRIX_PUBLIC_BASE_URL: ${BITRIX_PUBLIC_BASE_URL} + BITRIX_API_FORWARD_URL: ${BITRIX_API_FORWARD_URL:-http://api-backend:8000/internal/openlines/v1/inbox} + BITRIX_HTTP_TIMEOUT_SEC: ${BITRIX_HTTP_TIMEOUT_SEC:-10} + secrets: + - bitrix_database_url + - bitrix_client_secret + - bitrix_application_token + - bitrix_internal_api_token + - bitrix_api_forward_token + - bitrix_token_encryption_key expose: ["8080"] volumes: - type: bind @@ -320,6 +399,8 @@ services: start_period: 30s restart: unless-stopped security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} logging: driver: json-file options: {max-size: "50m", max-file: "5"} @@ -328,11 +409,15 @@ services: build: context: ../../bitrix-sync image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local} - env_file: - - path: ../../.env - required: false environment: - <<: *no-sms-secrets + HAN_SECRET_VARS: BITRIX_SYNC_DATABASE_URL BITRIX_SYNC_SERVICE_TOKEN + BITRIX_SYNC_DATABASE_URL_FILE: /run/secrets/bitrix_sync_database_url + BITRIX_SYNC_SERVICE_TOKEN_FILE: /run/secrets/bitrix_sync_service_token + APP_ENV: ${APP_ENV:-production-like} + BITRIX_SYNC_ENABLED: ${BITRIX_SYNC_ENABLED:-false} + secrets: + - bitrix_sync_database_url + - bitrix_sync_service_token expose: ["8080"] volumes: - type: bind @@ -353,3 +438,65 @@ services: logging: driver: json-file options: {max-size: "50m", max-file: "5"} + +secrets: + api_database_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL + api_redis_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_URL + api_redis_realtime_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_REALTIME_URL + message_safety_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_SERVICE_TOKEN + bitrix_local_app_internal_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_LOCAL_APP_INTERNAL_TOKEN + bitrix_api_inbox_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_INBOX_TOKEN + keycloak_settings_bridge_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN + selectel_s3_access_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_ACCESS_KEY + selectel_s3_secret_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_SECRET_KEY + cursor_hmac_secret: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/CURSOR_HMAC_SECRET + sms_database_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SMS_DATABASE_URL + sms_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SMS_SERVICE_TOKEN + idgtl_sms_api_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_API_KEY + idgtl_sms_callback_username: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_CALLBACK_USERNAME + idgtl_sms_callback_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_CALLBACK_PASSWORD + message_safety_redis_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_REDIS_URL + bitrix_database_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_DATABASE_URL + bitrix_client_secret: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_CLIENT_SECRET + bitrix_application_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_APPLICATION_TOKEN + bitrix_internal_api_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_INTERNAL_API_TOKEN + bitrix_api_forward_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_FORWARD_TOKEN + bitrix_token_encryption_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_TOKEN_ENCRYPTION_KEY + bitrix_sync_database_url: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_SYNC_DATABASE_URL + bitrix_sync_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_SYNC_SERVICE_TOKEN + keycloak_db_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_DB_PASSWORD + keycloak_admin_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_ADMIN_PASSWORD + keycloak_otp_mock_code: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_MOCK_CODE + keycloak_yandex_captcha_server_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY + keycloak_otp_hmac_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_HMAC_KEY + keycloak_sms_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SMS_SERVICE_TOKEN diff --git a/codebase/backend/keycloak/Dockerfile b/codebase/backend/keycloak/Dockerfile index 744c367..a6c18b8 100644 --- a/codebase/backend/keycloak/Dockerfile +++ b/codebase/backend/keycloak/Dockerfile @@ -21,7 +21,8 @@ RUN /opt/keycloak/bin/kc.sh build FROM quay.io/keycloak/keycloak:26.1.4 COPY --from=keycloak-build --chown=keycloak:keycloak /opt/keycloak/ /opt/keycloak/ COPY --chown=keycloak:keycloak realm/han-chat-realm.json /opt/keycloak/data/import/han-chat-realm.json +COPY --chown=keycloak:keycloak --chmod=0555 container-entrypoint.sh /opt/keycloak/bin/han-container-entrypoint USER 1000 EXPOSE 8080 9000 -ENTRYPOINT ["/opt/keycloak/bin/kc.sh"] +ENTRYPOINT ["/opt/keycloak/bin/han-container-entrypoint"] CMD ["start", "--optimized", "--import-realm"] diff --git a/codebase/backend/keycloak/container-entrypoint.sh b/codebase/backend/keycloak/container-entrypoint.sh new file mode 100644 index 0000000..a90a214 --- /dev/null +++ b/codebase/backend/keycloak/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec /opt/keycloak/bin/kc.sh "$@" diff --git a/codebase/backend/keycloak/docker-compose.yml b/codebase/backend/keycloak/docker-compose.yml index a6c42df..6fef7bd 100644 --- a/codebase/backend/keycloak/docker-compose.yml +++ b/codebase/backend/keycloak/docker-compose.yml @@ -6,8 +6,20 @@ services: image: han-chat/keycloak:26.1.4-otp-1.0.0 command: ["start", "--optimized", "--import-realm"] environment: + HAN_SECRET_VARS: >- + KC_DB_PASSWORD KC_BOOTSTRAP_ADMIN_PASSWORD KEYCLOAK_OTP_MOCK_CODE + KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY KEYCLOAK_OTP_HMAC_KEY + KEYCLOAK_SETTINGS_BRIDGE_TOKEN KEYCLOAK_SMS_SERVICE_TOKEN + KC_DB_PASSWORD_FILE: /run/secrets/keycloak_db_password + KC_BOOTSTRAP_ADMIN_PASSWORD_FILE: /run/secrets/keycloak_admin_password + KEYCLOAK_OTP_MOCK_CODE_FILE: /run/secrets/keycloak_otp_mock_code + KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: /run/secrets/keycloak_yandex_captcha_server_key + KEYCLOAK_OTP_HMAC_KEY_FILE: /run/secrets/keycloak_otp_hmac_key + KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token + KEYCLOAK_SMS_SERVICE_TOKEN_FILE: /run/secrets/keycloak_sms_service_token KC_DB: postgres KC_DB_URL: ${KEYCLOAK_DB_URL:?KEYCLOAK_DB_URL is required} + KC_DB_USERNAME: ${KEYCLOAK_DB_USERNAME:?KEYCLOAK_DB_USERNAME is required} KC_DB_URL_PROPERTIES: ${KC_DB_URL_PROPERTIES:-currentSchema=keycloak} KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL:-https://tohin.ru/auth} KC_HOSTNAME_STRICT: "true" @@ -19,21 +31,23 @@ services: KC_METRICS_ENABLED: "true" KC_HTTP_MANAGEMENT_PORT: "9000" KC_BOOTSTRAP_ADMIN_USERNAME: ${KC_BOOTSTRAP_ADMIN_USERNAME:?bootstrap admin username is required} - KC_BOOTSTRAP_ADMIN_PASSWORD: ${KC_BOOTSTRAP_ADMIN_PASSWORD:?bootstrap admin password is required} KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-true} - KEYCLOAK_OTP_MOCK_CODE: ${KEYCLOAK_OTP_MOCK_CODE:-} KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false} KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-} - KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY:-} - KEYCLOAK_OTP_HMAC_KEY: ${KEYCLOAK_OTP_HMAC_KEY:?OTP HMAC key is required} KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300} KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp} - KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN:?settings bridge token is required} KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080} - KEYCLOAK_SMS_SERVICE_TOKEN: ${KEYCLOAK_SMS_SERVICE_TOKEN:-} KC_LOG_CONSOLE_OUTPUT: json KC_LOG_LEVEL: ${KEYCLOAK_LOG_LEVEL:-INFO} JAVA_OPTS_APPEND: ${KEYCLOAK_JAVA_OPTS:--XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=35} + secrets: + - keycloak_db_password + - keycloak_admin_password + - keycloak_otp_mock_code + - keycloak_yandex_captcha_server_key + - keycloak_otp_hmac_key + - keycloak_settings_bridge_token + - keycloak_sms_service_token expose: - "8080" - "9000" @@ -56,6 +70,8 @@ services: - no-new-privileges:true cap_drop: - ALL + ulimits: + core: {soft: 0, hard: 0} stop_grace_period: 30s restart: unless-stopped @@ -64,3 +80,19 @@ networks: backend: observability: egress: + +secrets: + keycloak_db_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_DB_PASSWORD + keycloak_admin_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_ADMIN_PASSWORD + keycloak_otp_mock_code: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_MOCK_CODE + keycloak_yandex_captcha_server_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY + keycloak_otp_hmac_key: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_HMAC_KEY + keycloak_settings_bridge_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN + keycloak_sms_service_token: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SMS_SERVICE_TOKEN diff --git a/codebase/backend/message-safety/Dockerfile b/codebase/backend/message-safety/Dockerfile index d23e3ec..fd08b64 100644 --- a/codebase/backend/message-safety/Dockerfile +++ b/codebase/backend/message-safety/Dockerfile @@ -5,7 +5,9 @@ WORKDIR /service COPY app ./app COPY pyproject.toml ./ RUN pip install --no-cache-dir . +COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint USER app EXPOSE 8080 HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)" +ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"] diff --git a/codebase/backend/message-safety/container-entrypoint.sh b/codebase/backend/message-safety/container-entrypoint.sh new file mode 100644 index 0000000..7e6f3ff --- /dev/null +++ b/codebase/backend/message-safety/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec "$@" diff --git a/codebase/backend/observability/docker-compose.yml b/codebase/backend/observability/docker-compose.yml index 2bc5a20..010e38d 100644 --- a/codebase/backend/observability/docker-compose.yml +++ b/codebase/backend/observability/docker-compose.yml @@ -18,12 +18,13 @@ services: APP_ENV: ${APP_ENV:-production-like} RELEASE_VERSION: ${RELEASE_VERSION:-unknown} OTEL_REMOTE_ENDPOINT: ${OTEL_REMOTE_ENDPOINT} - OTEL_REMOTE_AUTH_HEADER: ${OTEL_REMOTE_AUTH_HEADER:-} OTEL_REMOTE_TLS_INSECURE: ${OTEL_REMOTE_TLS_INSECURE:-false} expose: ["4317", "4318", "13133", "8888"] volumes: - ./otel-collector.yaml:/etc/otelcol/config.yaml:ro - otel-queue:/var/lib/otelcol/queue + secrets: + - otel_remote_auth_header networks: [observability, egress] depends_on: otel-queue-init: {condition: service_completed_successfully} @@ -38,6 +39,8 @@ services: user: "10001:10001" cap_drop: ["ALL"] security_opt: ["no-new-privileges:true"] + ulimits: + core: {soft: 0, hard: 0} mem_limit: 512m cpus: 0.5 logging: @@ -62,3 +65,7 @@ services: user: "10001:10001" cap_drop: ["ALL"] security_opt: ["no-new-privileges:true"] + +secrets: + otel_remote_auth_header: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/OTEL_REMOTE_AUTH_HEADER diff --git a/codebase/backend/observability/otel-collector.yaml b/codebase/backend/observability/otel-collector.yaml index 41a2b2c..3732d65 100644 --- a/codebase/backend/observability/otel-collector.yaml +++ b/codebase/backend/observability/otel-collector.yaml @@ -100,7 +100,7 @@ exporters: otlp/remote: endpoint: "${env:OTEL_REMOTE_ENDPOINT}" headers: - authorization: "${env:OTEL_REMOTE_AUTH_HEADER}" + authorization: ${file:/run/secrets/otel_remote_auth_header} tls: insecure: "${env:OTEL_REMOTE_TLS_INSECURE}" sending_queue: diff --git a/codebase/backend/redis/docker-compose.yml b/codebase/backend/redis/docker-compose.yml index 01fecd6..5c395bf 100644 --- a/codebase/backend/redis/docker-compose.yml +++ b/codebase/backend/redis/docker-compose.yml @@ -4,17 +4,18 @@ services: context: . image: han-chat-redis:${RELEASE_VERSION:-local} environment: - REDIS_API_PASSWORD: ${REDIS_API_PASSWORD} - REDIS_SAFETY_PASSWORD: ${REDIS_SAFETY_PASSWORD} - REDIS_HEALTH_PASSWORD: ${REDIS_HEALTH_PASSWORD} REDIS_MAXMEMORY: ${REDIS_MAXMEMORY:-384mb} REDIS_EVICTION_POLICY: ${REDIS_EVICTION_POLICY:-volatile-lru} + secrets: + - redis_api_password + - redis_safety_password + - redis_health_password expose: ["6379"] volumes: - redis-data:/data networks: [backend] healthcheck: - test: ["CMD-SHELL", "redis-cli --no-auth-warning --user ops_health --pass \"$$REDIS_HEALTH_PASSWORD\" PING | grep -qx PONG"] + test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(cat /run/secrets/redis_health_password)\" redis-cli --user ops_health PING | grep -qx PONG"] interval: 10s timeout: 3s retries: 10 @@ -28,7 +29,16 @@ services: cap_drop: ["ALL"] mem_limit: 512m ulimits: + core: {soft: 0, hard: 0} nofile: {soft: 65536, hard: 65536} logging: driver: json-file options: {max-size: "50m", max-file: "5"} + +secrets: + redis_api_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_API_PASSWORD + redis_safety_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_SAFETY_PASSWORD + redis_health_password: + file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_HEALTH_PASSWORD diff --git a/codebase/backend/redis/scripts/entrypoint.sh b/codebase/backend/redis/scripts/entrypoint.sh index 53791ba..fd8256d 100644 --- a/codebase/backend/redis/scripts/entrypoint.sh +++ b/codebase/backend/redis/scripts/entrypoint.sh @@ -1,17 +1,29 @@ #!/bin/sh set -eu -for name in REDIS_API_PASSWORD REDIS_SAFETY_PASSWORD REDIS_HEALTH_PASSWORD; do - eval "value=\${$name:-}" +for item in \ + REDIS_API_PASSWORD:redis_api_password \ + REDIS_SAFETY_PASSWORD:redis_safety_password \ + REDIS_HEALTH_PASSWORD:redis_health_password +do + name=${item%%:*} + file=/run/secrets/${item#*:} + if [ ! -r "$file" ]; then + echo "redis bootstrap: missing secret file for $name" >&2 + exit 66 + fi + value=$(cat "$file") if [ "${#value}" -lt 16 ] || printf '%s' "$value" | grep -Eq '[[:space:]]'; then echo "redis bootstrap: $name must be at least 16 characters without whitespace" >&2 exit 64 fi + export "$name=$value" done umask 077 envsubst '${REDIS_API_PASSWORD} ${REDIS_SAFETY_PASSWORD} ${REDIS_HEALTH_PASSWORD}' \ < /etc/han-redis/users.acl.template > /tmp/users.acl +unset REDIS_API_PASSWORD REDIS_SAFETY_PASSWORD REDIS_HEALTH_PASSWORD exec redis-server /etc/han-redis/redis.conf \ --aclfile /tmp/users.acl \ diff --git a/codebase/backend/scripts/validate-env b/codebase/backend/scripts/validate-env index 15a4e71..2065e6d 100644 --- a/codebase/backend/scripts/validate-env +++ b/codebase/backend/scripts/validate-env @@ -1,46 +1,56 @@ #!/usr/bin/env python3 -"""Fail-fast validation for the single HAN Chat deployment environment.""" +"""Validate public deployment config separately from runtime secrets.""" from __future__ import annotations +import os import re +import stat import sys from pathlib import Path from urllib.parse import urlparse -REQUIRED = { - "APP_ENV", "RELEASE_VERSION", "HAN_PG_HOST", "DATABASE_URL", - "BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL", - "MESSAGE_SAFETY_DATABASE_URL", "SMS_DATABASE_URL", "KEYCLOAK_DB_URL", "PUBLIC_HOST", - "PUBLIC_WEB_URL", "PUBLIC_API_URL", "PUBLIC_AUTH_URL", - "KEYCLOAK_PUBLIC_URL", "KEYCLOAK_INTERNAL_URL", "REDIS_URL", - "REDIS_REALTIME_URL", "MESSAGE_SAFETY_REDIS_URL", +REQUIRED_CONFIG = { + "SECRETS_SOURCE", "APP_ENV", "RELEASE_VERSION", "HAN_PG_HOST", "HAN_PG_PORT", + "HAN_PG_DATABASE", "PG_CA_HOST_PATH", "KEYCLOAK_DB_URL", "KEYCLOAK_DB_USERNAME", + "PUBLIC_HOST", "PUBLIC_WEB_URL", + "PUBLIC_API_URL", "PUBLIC_AUTH_URL", "KEYCLOAK_PUBLIC_URL", + "KEYCLOAK_INTERNAL_URL", "KEYCLOAK_REALM", "KEYCLOAK_SMS_SERVICE_URL", + "IDGTL_SMS_BASE_URL", "IDGTL_SMS_CALLBACK_PUBLIC_URL", +} +REQUIRED_RUNTIME = { + "DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL", + "SMS_DATABASE_URL", "KEYCLOAK_DB_PASSWORD", "REDIS_URL", + "REDIS_REALTIME_URL", "MESSAGE_SAFETY_REDIS_URL", "REDIS_API_PASSWORD", + "REDIS_SAFETY_PASSWORD", "REDIS_HEALTH_PASSWORD", "MESSAGE_SAFETY_SERVICE_TOKEN", "BITRIX_LOCAL_APP_INTERNAL_TOKEN", "BITRIX_INTERNAL_API_TOKEN", "BITRIX_API_FORWARD_TOKEN", "BITRIX_API_INBOX_TOKEN", "BITRIX_SYNC_SERVICE_TOKEN", "KEYCLOAK_SETTINGS_BRIDGE_TOKEN", "KEYCLOAK_OTP_HMAC_KEY", - "KEYCLOAK_SMS_SERVICE_URL", "KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN", - "IDGTL_SMS_BASE_URL", "IDGTL_SMS_API_KEY", "IDGTL_SMS_CALLBACK_PUBLIC_URL", - "IDGTL_SMS_CALLBACK_USERNAME", "IDGTL_SMS_CALLBACK_PASSWORD", - "KEYCLOAK_ADMIN", "KEYCLOAK_ADMIN_PASSWORD", "CURSOR_HMAC_SECRET", - "BITRIX_TOKEN_ENCRYPTION_KEY", - "SELECTEL_S3_ENDPOINT_URL", + "KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN", + "IDGTL_SMS_API_KEY", "IDGTL_SMS_CALLBACK_USERNAME", + "IDGTL_SMS_CALLBACK_PASSWORD", "KEYCLOAK_ADMIN_PASSWORD", + "CURSOR_HMAC_SECRET", "BITRIX_TOKEN_ENCRYPTION_KEY", "SELECTEL_S3_ACCESS_KEY", "SELECTEL_S3_SECRET_KEY", - "SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY", - "SELECTEL_S3_QUARANTINE_READ_SECRET_KEY", "REDIS_API_PASSWORD", - "REDIS_SAFETY_PASSWORD", "REDIS_HEALTH_PASSWORD", } -SECRET_KEYS = { - key for key in REQUIRED - if any(word in key for word in ("TOKEN", "PASSWORD", "SECRET_KEY", "ACCESS_KEY")) -} | { +OPTIONAL_SECRET_KEYS = { "BITRIX_CLIENT_SECRET", "BITRIX_APPLICATION_TOKEN", - "IDGTL_SMS_CALLBACK_USERNAME", "IDGTL_SMS_CALLBACK_PASSWORD", + "BITRIX_SYNC_CRM_WEBHOOK_URL", "BITRIX_SYNC_WEBHOOK_TOKEN", "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", + "NOTIFICATIONS_TOKEN_PRODUCER_TEST", + "OTEL_REMOTE_AUTH_HEADER", "KEYCLOAK_OTP_MOCK_CODE", +} +FORBIDDEN_CONFIG_KEYS = REQUIRED_RUNTIME | OPTIONAL_SECRET_KEYS | { + "BITRIX_SYNC_APP_DATABASE_URL", "PG_BACKUP_DSN", +} +DSN_KEYS = { + "DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_APP_DATABASE_URL", + "BITRIX_SYNC_DATABASE_URL", "MESSAGE_SAFETY_DATABASE_URL", + "SMS_DATABASE_URL", "PG_BACKUP_DSN", } PLACEHOLDER = re.compile(r"(change-me|example\.(com|ru|invalid)|<[^>]+>)", re.I) -def load(path: Path) -> tuple[dict[str, str], list[str]]: +def load_env(path: Path) -> tuple[dict[str, str], list[str]]: values: dict[str, str] = {} errors: list[str] = [] for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1): @@ -58,33 +68,107 @@ def load(path: Path) -> tuple[dict[str, str], list[str]]: return values, errors -def main() -> int: - path = Path(sys.argv[1] if len(sys.argv) > 1 else ".env") - if not path.is_file(): - print(f"ERROR: файл не найден: {path}", file=sys.stderr) - return 2 - env, errors = load(path) - for key in sorted(REQUIRED): - if not env.get(key): - errors.append(f"{key}: обязательное значение отсутствует") - for key in sorted(SECRET_KEYS): - value = env.get(key, "") - if value and (len(value) < 16 or PLACEHOLDER.search(value)): - errors.append(f"{key}: секрет должен быть непустым, уникальным и длиной >=16") - for key in ("SMS_SERVICE_TOKEN", "KEYCLOAK_SMS_SERVICE_TOKEN"): - if env.get(key) and len(env[key]) < 32: - errors.append(f"{key}: service token должен иметь длину >=32") - captcha_enabled = env.get("KEYCLOAK_YANDEX_CAPTCHA_ENABLED", "false").lower() - if captcha_enabled not in {"true", "false"}: - errors.append("KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ожидается true или false") - if captcha_enabled == "true": - for key in ( - "KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY", - "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", - ): - if not env.get(key): - errors.append(f"{key}: обязательное значение при включённой CAPTCHA отсутствует") +def load_manifest(path: Path) -> tuple[dict[str, str], list[str]]: + files, errors = load_env(path) + values: dict[str, str] = {} + for key, raw_path in files.items(): + secret_path = Path(raw_path) + if not secret_path.is_absolute(): + errors.append(f"{key}: manifest должен содержать абсолютный путь") + continue + try: + metadata = secret_path.lstat() + mode = stat.S_IMODE(metadata.st_mode) + parent_mode = stat.S_IMODE(secret_path.parent.stat().st_mode) + if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode): + errors.append(f"{key}: secret file должен быть regular file, не symlink") + continue + if os.name != "nt" and (mode & 0o022 or parent_mode & 0o077): + errors.append( + f"{key}: secret file или runtime directory имеют небезопасные права" + ) + continue + values[key] = secret_path.read_text(encoding="utf-8").rstrip("\r\n") + except OSError: + errors.append(f"{key}: secret file недоступен") + return values, errors + +def runtime_from_environment() -> dict[str, str]: + keys = REQUIRED_RUNTIME | OPTIONAL_SECRET_KEYS + return {key: os.environ[key] for key in keys if key in os.environ} + + +def validate_config(config: dict[str, str], errors: list[str]) -> None: + for key in sorted(REQUIRED_CONFIG): + if not config.get(key): + errors.append(f"{key}: обязательный config отсутствует") + if config.get("SECRETS_SOURCE") not in {"selectel", "file"}: + errors.append("SECRETS_SOURCE: ожидается selectel или file") + for key in sorted(FORBIDDEN_CONFIG_KEYS & config.keys()): + errors.append(f"{key}: секрет или credential-bearing DSN запрещён в обычном .env") + for key, value in config.items(): + if key.endswith("_FILE") and value: + errors.append(f"{key}: secret file path задаётся launcher, не обычным .env") + if key.endswith("_URL"): + parsed = urlparse(value.replace("jdbc:", "", 1)) + if parsed.username or parsed.password: + errors.append(f"{key}: URL с credentials запрещён в обычном .env") + + +def validate_runtime(config: dict[str, str], runtime: dict[str, str], errors: list[str]) -> None: + required = set(REQUIRED_RUNTIME) + if config.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "true": + required.add("KEYCLOAK_OTP_MOCK_CODE") + if config.get("KEYCLOAK_YANDEX_CAPTCHA_ENABLED", "false").lower() == "true": + required.add("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY") + if not config.get("KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY"): + errors.append( + "KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: обязательный public config отсутствует" + ) + for key in sorted(required): + value = runtime.get(key, "") + if not value: + errors.append(f"{key}: обязательный runtime secret отсутствует") + elif key == "KEYCLOAK_OTP_MOCK_CODE": + if not re.fullmatch(r"\d{6,10}", value) or value == "1234": + errors.append( + f"{key}: требуется нестандартный цифровой код длиной 6–10 цифр" + ) + elif key not in DSN_KEYS and (len(value) < 16 or PLACEHOLDER.search(value)): + errors.append(f"{key}: runtime secret слишком короткий или является placeholder") + + for key in DSN_KEYS & runtime.keys(): + value = runtime[key] + if "sslmode=verify-full" not in value or "sslrootcert=" not in value: + errors.append(f"{key}: требуется sslmode=verify-full и sslrootcert") + if re.search(r"(?:[?&](?:options|currentSchema)=)", value, re.I): + errors.append(f"{key}: options/currentSchema запрещены через PgBouncer") + + redis_contract = ( + ("REDIS_URL", "api_backend", "REDIS_API_PASSWORD", "/0"), + ("REDIS_REALTIME_URL", "api_backend", "REDIS_API_PASSWORD", "/1"), + ("MESSAGE_SAFETY_REDIS_URL", "message_safety", "REDIS_SAFETY_PASSWORD", "/2"), + ) + for key, username, password_key, database in redis_contract: + parsed = urlparse(runtime.get(key, "")) + if ( + parsed.scheme != "redis" or parsed.hostname != "redis" + or parsed.username != username or parsed.password != runtime.get(password_key) + or parsed.path != database + ): + errors.append(f"{key}: ACL user/password/host/DB не согласованы с {password_key}") + + for left, right in ( + ("BITRIX_LOCAL_APP_INTERNAL_TOKEN", "BITRIX_INTERNAL_API_TOKEN"), + ("BITRIX_API_FORWARD_TOKEN", "BITRIX_API_INBOX_TOKEN"), + ("KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN"), + ): + if runtime.get(left) != runtime.get(right): + errors.append(f"{left} должен совпадать с {right}") + + +def validate_shared(env: dict[str, str], errors: list[str]) -> None: production = env.get("APP_ENV") in {"production-like", "production"} if production and env.get("FRONTEND_DEV_PROXY_ENABLED", "").lower() != "false": errors.append("FRONTEND_DEV_PROXY_ENABLED: production-like/production требует false") @@ -105,41 +189,6 @@ def main() -> int: errors.append(f"{key}: ожидается http URL с Docker DNS service name") if not env.get("KEYCLOAK_INTERNAL_URL", "").rstrip("/").endswith("/auth"): errors.append("KEYCLOAK_INTERNAL_URL: внутренний URL должен заканчиваться на /auth") - redis_contract = ( - ("REDIS_URL", "api_backend", "REDIS_API_PASSWORD", "/0"), - ("REDIS_REALTIME_URL", "api_backend", "REDIS_API_PASSWORD", "/1"), - ("MESSAGE_SAFETY_REDIS_URL", "message_safety", "REDIS_SAFETY_PASSWORD", "/2"), - ) - for key, username, password_key, database in redis_contract: - parsed = urlparse(env.get(key, "")) - if ( - parsed.scheme != "redis" - or parsed.hostname != "redis" - or parsed.username != username - or parsed.password != env.get(password_key) - or parsed.path != database - ): - errors.append(f"{key}: ACL user/password/host/DB не согласованы с {password_key}") - for key in ( - "DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL", - "MESSAGE_SAFETY_DATABASE_URL", "SMS_DATABASE_URL", "KEYCLOAK_DB_URL", - ): - value = env.get(key, "") - if "sslmode=verify-full" not in value or "sslrootcert=" not in value: - errors.append(f"{key}: требуется sslmode=verify-full и sslrootcert") - if re.search(r"(?:[?&](?:options|currentSchema)=)", value, re.I): - errors.append( - f"{key}: options/currentSchema запрещены через PgBouncer; " - "используйте database-level search_path роли" - ) - pairs = ( - ("BITRIX_LOCAL_APP_INTERNAL_TOKEN", "BITRIX_INTERNAL_API_TOKEN"), - ("BITRIX_API_FORWARD_TOKEN", "BITRIX_API_INBOX_TOKEN"), - ("KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN"), - ) - for left, right in pairs: - if env.get(left) != env.get(right): - errors.append(f"{left} должен совпадать с {right}") try: poll = int(env.get("MESSAGE_SAFETY_TASK_POLL_MAX_SEC", "0")) nginx = int(env.get("NGINX_MESSAGE_READ_TIMEOUT_SEC", "0")) @@ -153,24 +202,55 @@ def main() -> int: if production and env.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "true": if env.get("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED", "").lower() != "true": errors.append("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true обязателен для mock OTP") - real_sms = env.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "false" - if production and real_sms and ( - env.get("IDGTL_SMS_BASE_URL", "").rstrip("/") != "https://direct.i-dgtl.ru" - ): - errors.append("IDGTL_SMS_BASE_URL: production contract требует https://direct.i-dgtl.ru") - expected_callback = f"https://{env.get('PUBLIC_HOST', '')}/callbacks/idgtl/sms" - if production and env.get("IDGTL_SMS_CALLBACK_PUBLIC_URL") != expected_callback: - errors.append( - "IDGTL_SMS_CALLBACK_PUBLIC_URL должен совпадать с публичным host и callback path" - ) + callback = f"https://{env.get('PUBLIC_HOST', '')}/callbacks/idgtl/sms" + if production and env.get("IDGTL_SMS_CALLBACK_PUBLIC_URL") != callback: + errors.append("IDGTL_SMS_CALLBACK_PUBLIC_URL должен совпадать с PUBLIC_HOST") if env.get("HAN_PG_HOST") in {"localhost", "127.0.0.1", "postgres", "db"}: errors.append("HAN_PG_HOST: PostgreSQL должен быть внешним managed endpoint") + keycloak_dsn = env.get("KEYCLOAK_DB_URL", "") + if "sslmode=verify-full" not in keycloak_dsn or "sslrootcert=" not in keycloak_dsn: + errors.append("KEYCLOAK_DB_URL: требуется sslmode=verify-full и sslrootcert") + + +def main() -> int: + args = sys.argv[1:] + config_path = Path(args.pop(0) if args and not args[0].startswith("--") else ".env") + manifest: Path | None = None + use_runtime_env = False + while args: + option = args.pop(0) + if option == "--runtime-manifest" and args: + manifest = Path(args.pop(0)) + elif option == "--runtime-env": + use_runtime_env = True + else: + print(f"ERROR: неизвестный параметр {option}", file=sys.stderr) + return 2 + if not config_path.is_file(): + print(f"ERROR: config файл не найден: {config_path}", file=sys.stderr) + return 2 + + config, errors = load_env(config_path) + validate_config(config, errors) + runtime: dict[str, str] = {} + if manifest: + if not manifest.is_file(): + errors.append("runtime manifest недоступен") + else: + runtime, manifest_errors = load_manifest(manifest) + errors.extend(manifest_errors) + elif use_runtime_env: + runtime = runtime_from_environment() + if manifest or use_runtime_env: + validate_runtime(config, runtime, errors) + validate_shared({**config, **runtime}, errors) if errors: for error in errors: print(f"ERROR: {error}", file=sys.stderr) return 1 - print(f"OK: {path} прошёл проверку ({len(env)} переменных)") + scope = "config + runtime secrets" if manifest or use_runtime_env else "non-secret config" + print(f"OK: {config_path} прошёл проверку ({scope})") return 0 diff --git a/codebase/backend/sms-service/Dockerfile b/codebase/backend/sms-service/Dockerfile index 3aade60..1843d28 100644 --- a/codebase/backend/sms-service/Dockerfile +++ b/codebase/backend/sms-service/Dockerfile @@ -13,6 +13,8 @@ COPY --from=builder /build/dist/*.whl /tmp/ RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl COPY alembic.ini ./ COPY migrations ./migrations +COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint USER 10001:10001 EXPOSE 8080 +ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--no-proxy-headers"] diff --git a/codebase/backend/sms-service/app/logging_security.py b/codebase/backend/sms-service/app/logging_security.py new file mode 100644 index 0000000..db055fb --- /dev/null +++ b/codebase/backend/sms-service/app/logging_security.py @@ -0,0 +1,47 @@ +from __future__ import annotations + +import re +from collections.abc import Mapping +from typing import Any + +REDACTED = "[REDACTED]" +_SENSITIVE_KEY = re.compile( + r"(authorization|cookie|password|passwd|secret|token|api[_-]?key|" + r"database[_-]?url|redis[_-]?url|dsn|callback[_-]?url)", + re.IGNORECASE, +) +_URI_USERINFO = re.compile(r"(?P[a-z][a-z0-9+.-]*://)[^/@\s]+@", re.IGNORECASE) +_QUERY_SECRET = re.compile( + r"(?P[?&](?:token|access_token|api_key|key|secret|password)=)[^&#\s]+", + re.IGNORECASE, +) +_AUTH_VALUE = re.compile(r"\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+", re.IGNORECASE) + + +def sanitize_text(value: str) -> str: + value = _URI_USERINFO.sub(r"\g[REDACTED]@", value) + value = _QUERY_SECRET.sub(r"\g[REDACTED]", value) + return _AUTH_VALUE.sub(r"\1 [REDACTED]", value) + + +def sanitize_value(value: Any) -> Any: + if isinstance(value, str): + return sanitize_text(value) + if isinstance(value, Mapping): + return { + str(key): REDACTED if _SENSITIVE_KEY.search(str(key)) else sanitize_value(item) + for key, item in value.items() + } + if isinstance(value, list): + return [sanitize_value(item) for item in value] + if isinstance(value, tuple): + return tuple(sanitize_value(item) for item in value) + return value + + +def redact_event( + _logger: Any, + _method_name: str, + event_dict: dict[str, Any], +) -> dict[str, Any]: + return sanitize_value(event_dict) diff --git a/codebase/backend/sms-service/app/main.py b/codebase/backend/sms-service/app/main.py index b2411ac..b070b5d 100644 --- a/codebase/backend/sms-service/app/main.py +++ b/codebase/backend/sms-service/app/main.py @@ -22,6 +22,7 @@ from starlette.exceptions import HTTPException as StarletteHTTPException from app.db import Database, SmsTemplate from app.domain import DomainError +from app.logging_security import redact_event from app.metrics import CALLBACK_LAG, CALLBACK_TOTAL from app.schemas import CallbackItem, ErrorEnvelope, MessageResponse, SendRequest, SendResponse from app.service import ( @@ -42,6 +43,7 @@ def configure_logging(level: str) -> None: processors=[ structlog.contextvars.merge_contextvars, add_trace_context, + redact_event, structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"), structlog.stdlib.add_log_level, structlog.processors.JSONRenderer(), @@ -147,7 +149,11 @@ async def http_error(request: Request, exc: StarletteHTTPException) -> JSONRespo @app.exception_handler(Exception) async def unhandled_error(request: Request, exc: Exception) -> JSONResponse: - log.exception("request.failed", error_code="internal_error") + log.error( + "request.failed", + error_code="internal_error", + error_type=type(exc).__name__, + ) return error_response(request, "internal_error", "Internal server error", 500) diff --git a/codebase/backend/sms-service/app/worker.py b/codebase/backend/sms-service/app/worker.py index ef888cc..043030c 100644 --- a/codebase/backend/sms-service/app/worker.py +++ b/codebase/backend/sms-service/app/worker.py @@ -14,6 +14,7 @@ from prometheus_client import start_http_server from sqlalchemy import and_, func, or_, select, update from app.db import Database, SendStatus, SmsOutboundMessage +from app.logging_security import redact_event from app.metrics import ( JOURNAL_ROWS, PENDING_AGE, @@ -37,6 +38,7 @@ def configure_logging(level: str) -> None: processors=[ structlog.contextvars.merge_contextvars, add_trace_context, + redact_event, structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"), structlog.stdlib.add_log_level, structlog.processors.JSONRenderer(), @@ -217,9 +219,12 @@ async def worker_loop(stop: asyncio.Event) -> None: await save_result(db, message.id, result, message.attempt_count) except TimeoutError: continue - except Exception: + except Exception as exc: SETTINGS_VALID.set(0) - log.exception("worker.iteration_failed") + log.error( + "worker.iteration_failed", + error_type=type(exc).__name__, + ) try: await asyncio.wait_for(stop.wait(), timeout=5) except TimeoutError: diff --git a/codebase/backend/sms-service/container-entrypoint.sh b/codebase/backend/sms-service/container-entrypoint.sh new file mode 100644 index 0000000..7e6f3ff --- /dev/null +++ b/codebase/backend/sms-service/container-entrypoint.sh @@ -0,0 +1,23 @@ +#!/bin/sh +set -eu + +for name in ${HAN_SECRET_VARS:-}; do + case "$name" in + ""|[0-9]*|*[!A-Z0-9_]*) + echo "container secrets: invalid variable name" >&2 + exit 64 + ;; + *) ;; + esac + eval "file=\${${name}_FILE:-}" + if [ -z "$file" ] || [ ! -r "$file" ]; then + echo "container secrets: missing file for $name" >&2 + exit 66 + fi + value=$(cat "$file") + export "$name=$value" + unset "${name}_FILE" +done + +unset HAN_SECRET_VARS +exec "$@" diff --git a/codebase/backend/sms-service/migrations/env.py b/codebase/backend/sms-service/migrations/env.py index b06c821..a05c4b7 100644 --- a/codebase/backend/sms-service/migrations/env.py +++ b/codebase/backend/sms-service/migrations/env.py @@ -1,17 +1,17 @@ from __future__ import annotations import asyncio +import os from logging.config import fileConfig from alembic import context from app.db import Base, create_postgres_engine -from app.settings import get_settings config = context.config if config.config_file_name: fileConfig(config.config_file_name) -database_url = get_settings().database_url +database_url = os.environ["SMS_DATABASE_URL"] if database_url.startswith("postgresql://"): database_url = database_url.replace("postgresql://", "postgresql+asyncpg://", 1) config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%")) diff --git a/codebase/backend/sms-service/tests/unit/test_logging_security.py b/codebase/backend/sms-service/tests/unit/test_logging_security.py new file mode 100644 index 0000000..2488eb8 --- /dev/null +++ b/codebase/backend/sms-service/tests/unit/test_logging_security.py @@ -0,0 +1,31 @@ +from app.logging_security import REDACTED, redact_event, sanitize_text + + +def test_redacts_sensitive_fields_recursively() -> None: + event = { + "api_key": "provider-key", + "nested": { + "callback_url": "https://user:password@example.test/callback", + "status": "accepted", + }, + } + + redacted = redact_event(None, "info", event) + + assert redacted["api_key"] == REDACTED + assert redacted["nested"]["callback_url"] == REDACTED + assert redacted["nested"]["status"] == "accepted" + + +def test_redacts_credentials_embedded_in_text() -> None: + value = ( + "POST https://callback-user:callback-password@example.test/cb" + "?api_key=query-secret Authorization=Basic header-secret" + ) + + redacted = sanitize_text(value) + + assert "callback-password" not in redacted + assert "query-secret" not in redacted + assert "header-secret" not in redacted + assert redacted.count(REDACTED) == 3 diff --git a/codebase/backend/tests/test_config.py b/codebase/backend/tests/test_config.py index d4e0857..d9f81bb 100644 --- a/codebase/backend/tests/test_config.py +++ b/codebase/backend/tests/test_config.py @@ -44,11 +44,13 @@ class InfrastructureConfigTests(unittest.TestCase): application = (ROOT / "infra/compose/application.yml").read_text(encoding="utf-8") self.assertIn("networks: [backend, observability, egress]", application) self.assertIn("networks: [public, backend, observability, egress]", application) - self.assertIn('KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ""', application) + self.assertIn( + "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: " + "/run/secrets/keycloak_yandex_captcha_server_key", + application, + ) self.assertEqual( - application.count( - "IDGTL_SMS_API_KEY: ${IDGTL_SMS_API_KEY:?IDGTL_SMS_API_KEY is required}" - ), + application.count("IDGTL_SMS_API_KEY_FILE: /run/secrets/idgtl_sms_api_key"), 1, ) @@ -56,7 +58,7 @@ class InfrastructureConfigTests(unittest.TestCase): self.assertEqual(jobs.count("networks: [backend, egress]"), 5) observability = (ROOT / "observability/docker-compose.yml").read_text(encoding="utf-8") - self.assertIn("networks: [observability, backend, egress]", observability) + self.assertIn("networks: [observability, egress]", observability) redis = (ROOT / "redis/docker-compose.yml").read_text(encoding="utf-8") self.assertNotIn("egress", redis) @@ -64,6 +66,8 @@ class InfrastructureConfigTests(unittest.TestCase): def test_only_nginx_fragment_publishes_ports(self) -> None: forbidden = ( ROOT / "infra/compose/application.yml", + ROOT / "api-backend/docker-compose.yml", + ROOT / "keycloak/docker-compose.yml", ROOT / "redis/docker-compose.yml", ROOT / "observability/docker-compose.yml", ROOT / "deployment/docker-compose.jobs.yml", @@ -177,6 +181,10 @@ class InfrastructureConfigTests(unittest.TestCase): self.assertIn("storage: file_storage", config) self.assertIn("retry_on_failure:", config) self.assertIn('insecure: "${env:OTEL_REMOTE_TLS_INSECURE}"', config) + self.assertIn( + "authorization: ${file:/run/secrets/otel_remote_auth_header}", config + ) + self.assertNotIn("OTEL_REMOTE_AUTH_HEADER", config) self.assertIn("tail_sampling:", config) self.assertNotIn("probabilistic_sampler:", config) self.assertIn('targets: ["sms-service:8080"]', config) @@ -231,7 +239,7 @@ class InfrastructureConfigTests(unittest.TestCase): def test_smoke_script_does_not_source_env_as_shell(self) -> None: smoke = (ROOT / "deployment/scripts/smoke.sh").read_text(encoding="utf-8") - self.assertNotIn('. "./$ENV_FILE"', smoke) + self.assertNotIn('. "./$CONFIG_FILE"', smoke) for variable in ("PUBLIC_HOST", "PUBLIC_WEB_URL", "KEYCLOAK_REALM"): self.assertIn(f"{variable}=$(env_value {variable})", smoke) @@ -298,18 +306,64 @@ class InfrastructureConfigTests(unittest.TestCase): for variable in ( "KC_DB_SCHEMA", "KEYCLOAK_OTP_MOCK_ENABLED", - "KEYCLOAK_OTP_MOCK_CODE", "KEYCLOAK_YANDEX_CAPTCHA_ENABLED", - "KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY", - "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", - "KEYCLOAK_OTP_HMAC_KEY", "KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC", "KEYCLOAK_SETTINGS_BRIDGE_URL", - "KEYCLOAK_SETTINGS_BRIDGE_TOKEN", "KEYCLOAK_SMS_SERVICE_URL", - "KEYCLOAK_SMS_SERVICE_TOKEN", ): self.assertIn(f" {variable}:", application) + for variable in ( + "KEYCLOAK_OTP_MOCK_CODE", + "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", + "KEYCLOAK_OTP_HMAC_KEY", + "KEYCLOAK_SETTINGS_BRIDGE_TOKEN", + "KEYCLOAK_SMS_SERVICE_TOKEN", + ): + self.assertIn(f" {variable}_FILE:", application) + self.assertIn( + " KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: " + "${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}", + application, + ) + + def test_compose_secrets_do_not_enter_config_environment(self) -> None: + compose_paths = ( + ROOT / "infra/compose/application.yml", + ROOT / "redis/docker-compose.yml", + ROOT / "observability/docker-compose.yml", + ROOT / "deployment/docker-compose.jobs.yml", + ) + combined = "\n".join(path.read_text(encoding="utf-8") for path in compose_paths) + self.assertNotIn("env_file:", combined) + for path in compose_paths: + self.assertIn( + "core: {soft: 0, hard: 0}", + path.read_text(encoding="utf-8"), + path, + ) + for variable in ( + "DATABASE_URL", + "REDIS_URL", + "SMS_SERVICE_TOKEN", + "KEYCLOAK_DB_PASSWORD", + "OTEL_REMOTE_AUTH_HEADER", + ): + self.assertNotIn(f"${{{variable}", combined) + self.assertIn( + "${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL", combined + ) + self.assertNotIn("entrypoint: []", combined) + + for service in ( + "api-backend", + "sms-service", + "message-safety", + "bitrix-local-app", + "bitrix-sync", + "keycloak", + ): + dockerfile = (ROOT / service / "Dockerfile").read_text(encoding="utf-8") + self.assertIn("han-container-entrypoint", dockerfile, service) def test_env_validator_accepts_materialized_example(self) -> None: example = (ROOT / ".env.example").read_text(encoding="utf-8") @@ -317,15 +371,10 @@ class InfrastructureConfigTests(unittest.TestCase): self.assertNotIn("currentSchema=", example) self.assertIn("KEYCLOAK_DB_SCHEMA=keycloak", example) self.assertIn("HAN_PG_PORT=5433", example) - for required in ( - "CURSOR_HMAC_SECRET=", - "BITRIX_TOKEN_ENCRYPTION_KEY=", - "KEYCLOAK_INTERNAL_URL=http://keycloak:8080/auth", - "KEYCLOAK_SMS_SERVICE_URL=http://sms-service:8080", - "IDGTL_SMS_CALLBACK_PUBLIC_URL=https://chat.example.ru/callbacks/idgtl/sms", - ): - self.assertIn(required, example) - materialized = example.replace("change-me", "0123456789abcdef0123456789abcdef") + self.assertIn("SECRETS_SOURCE=file", example) + self.assertNotIn("CURSOR_HMAC_SECRET=", example) + self.assertNotIn("BITRIX_TOKEN_ENCRYPTION_KEY=", example) + materialized = example materialized = materialized.replace( "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false", "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true", @@ -341,11 +390,9 @@ class InfrastructureConfigTests(unittest.TestCase): ) self.assertEqual(result.returncode, 0, result.stderr) - def test_env_validator_requires_captcha_keys_only_when_enabled(self) -> None: + def test_env_validator_keeps_captcha_server_key_out_of_config(self) -> None: example = (ROOT / ".env.example").read_text(encoding="utf-8") materialized = example.replace( - "change-me", "0123456789abcdef0123456789abcdef" - ).replace( "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false", "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true", ) @@ -353,33 +400,17 @@ class InfrastructureConfigTests(unittest.TestCase): "KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false", "KEYCLOAK_YANDEX_CAPTCHA_ENABLED=true", ) - enabled_with_keys = enabled_without_keys.replace( - "KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=", - "KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=client-key", - ).replace( - "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=", - "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=server-key-0123456789", - ) with tempfile.TemporaryDirectory() as directory: env_file = Path(directory) / ".env" env_file.write_text(enabled_without_keys, encoding="utf-8") - missing = subprocess.run( + result = subprocess.run( [sys.executable, str(ROOT / "scripts/validate-env"), str(env_file)], text=True, capture_output=True, check=False, ) - env_file.write_text(enabled_with_keys, encoding="utf-8") - configured = subprocess.run( - [sys.executable, str(ROOT / "scripts/validate-env"), str(env_file)], - text=True, - capture_output=True, - check=False, - ) - self.assertNotEqual(missing.returncode, 0) - self.assertIn("KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY", missing.stderr) - self.assertIn("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", missing.stderr) - self.assertEqual(configured.returncode, 0, configured.stderr) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertNotIn("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=", example) if __name__ == "__main__": diff --git a/codebase/backend/tests/test_secret_hygiene.py b/codebase/backend/tests/test_secret_hygiene.py new file mode 100644 index 0000000..1efa0c3 --- /dev/null +++ b/codebase/backend/tests/test_secret_hygiene.py @@ -0,0 +1,126 @@ +from __future__ import annotations + +import os +import runpy +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +VALIDATOR = ROOT / "scripts/validate-env" + + +def runtime_values() -> dict[str, str]: + symbols = runpy.run_path(str(VALIDATOR)) + values = {key: "a" * 32 for key in symbols["REQUIRED_RUNTIME"]} + pg_tail = "?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem" + values.update( + { + "DATABASE_URL": f"postgresql+asyncpg://han_app:password@pg:5433/han_chat{pg_tail}", + "BITRIX_DATABASE_URL": f"postgresql://bitrix:password@pg:5433/han_chat{pg_tail}", + "BITRIX_SYNC_DATABASE_URL": f"postgresql://sync:password@pg:5433/han_chat{pg_tail}", + "MESSAGE_SAFETY_DATABASE_URL": f"postgresql://safety:password@pg:5433/han_chat{pg_tail}", + "SMS_DATABASE_URL": f"postgresql+asyncpg://sms:password@pg:5433/han_chat{pg_tail}", + "KEYCLOAK_DB_URL": f"jdbc:postgresql://pg:5433/han_chat{pg_tail}", + "REDIS_API_PASSWORD": "r" * 32, + "REDIS_SAFETY_PASSWORD": "s" * 32, + "REDIS_HEALTH_PASSWORD": "h" * 32, + "REDIS_URL": f"redis://api_backend:{'r' * 32}@redis:6379/0", + "REDIS_REALTIME_URL": f"redis://api_backend:{'r' * 32}@redis:6379/1", + "MESSAGE_SAFETY_REDIS_URL": f"redis://message_safety:{'s' * 32}@redis:6379/2", + "BITRIX_LOCAL_APP_INTERNAL_TOKEN": "b" * 32, + "BITRIX_INTERNAL_API_TOKEN": "b" * 32, + "BITRIX_API_FORWARD_TOKEN": "f" * 32, + "BITRIX_API_INBOX_TOKEN": "f" * 32, + "KEYCLOAK_SMS_SERVICE_TOKEN": "k" * 32, + "SMS_SERVICE_TOKEN": "k" * 32, + "KEYCLOAK_OTP_MOCK_CODE": "846271", + } + ) + return values + + +class SecretHygieneTests(unittest.TestCase): + def run_validator( + self, config: Path, *args: str, environment: dict[str, str] | None = None + ) -> subprocess.CompletedProcess[str]: + return subprocess.run( + [sys.executable, str(VALIDATOR), str(config), *args], + text=True, + capture_output=True, + check=False, + env=environment, + ) + + def test_example_is_non_secret_config(self) -> None: + example = (ROOT / ".env.example").read_text(encoding="utf-8") + symbols = runpy.run_path(str(VALIDATOR)) + for key in symbols["FORBIDDEN_CONFIG_KEYS"]: + self.assertNotRegex(example, rf"(?m)^{key}=") + self.assertIn("SECRETS_SOURCE=file", example) + + def test_config_rejects_secret_key_and_credential_url(self) -> None: + example = (ROOT / ".env.example").read_text(encoding="utf-8") + with tempfile.TemporaryDirectory() as directory: + config = Path(directory) / ".env" + config.write_text( + example + + "\nSMS_SERVICE_TOKEN=not-for-config\n" + + "EXTERNAL_URL=https://user:password@example.net/path\n", + encoding="utf-8", + ) + result = self.run_validator(config) + self.assertNotEqual(result.returncode, 0) + self.assertIn("SMS_SERVICE_TOKEN", result.stderr) + self.assertIn("URL с credentials", result.stderr) + self.assertNotIn("not-for-config", result.stderr) + + def test_runtime_environment_is_validated_without_value_disclosure(self) -> None: + values = runtime_values() + secret_canary = "canary-never-print-" + "x" * 20 + values["CURSOR_HMAC_SECRET"] = secret_canary + environment = {**os.environ, **values} + with tempfile.TemporaryDirectory() as directory: + config = Path(directory) / ".env" + config.write_text( + (ROOT / ".env.example").read_text(encoding="utf-8").replace( + "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false", + "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true", + ), + encoding="utf-8", + ) + result = self.run_validator(config, "--runtime-env", environment=environment) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertNotIn(secret_canary, result.stdout + result.stderr) + + def test_runtime_manifest_reads_protected_files_without_value_disclosure(self) -> None: + values = runtime_values() + secret_canary = "manifest-canary-" + "z" * 20 + values["CURSOR_HMAC_SECRET"] = secret_canary + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + config = root / ".env" + config.write_text( + (ROOT / ".env.example").read_text(encoding="utf-8").replace( + "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false", + "KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true", + ), + encoding="utf-8", + ) + entries = [] + for index, (key, value) in enumerate(sorted(values.items())): + path = root / f"secret-{index}" + path.write_text(value + "\n", encoding="utf-8") + path.chmod(0o600) + entries.append(f"{key}={path.resolve()}") + manifest = root / "manifest" + manifest.write_text("\n".join(entries) + "\n", encoding="utf-8") + result = self.run_validator(config, "--runtime-manifest", str(manifest)) + self.assertEqual(result.returncode, 0, result.stderr) + self.assertNotIn(secret_canary, result.stdout + result.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/codebase/backend/tests/test_secrets_loader.py b/codebase/backend/tests/test_secrets_loader.py new file mode 100644 index 0000000..fb70bcd --- /dev/null +++ b/codebase/backend/tests/test_secrets_loader.py @@ -0,0 +1,471 @@ +from __future__ import annotations + +import base64 +import importlib.util +import io +import json +import os +import stat +import subprocess +import sys +import tempfile +import unittest +import urllib.error +from pathlib import Path +from typing import Any + +ROOT = Path(__file__).resolve().parents[1] +LOADER_PATH = ROOT / "deployment" / "secrets" / "secrets_loader.py" +LAUNCHER_PATH = ROOT / "deployment" / "secrets" / "han-secrets" +SPEC = importlib.util.spec_from_file_location("han_secrets_loader", LOADER_PATH) +assert SPEC and SPEC.loader +loader = importlib.util.module_from_spec(SPEC) +sys.modules[SPEC.name] = loader +SPEC.loader.exec_module(loader) + + +class FakeResponse: + def __init__( + self, status: int, body: bytes, headers: dict[str, str] | None = None + ) -> None: + self.status = status + self._body = body + self.headers = headers or {} + + def __enter__(self) -> "FakeResponse": + return self + + def __exit__(self, *args: Any) -> None: + return None + + def read(self, amount: int) -> bytes: + return self._body[:amount] + + +class FakeOpener: + def __init__(self, outcomes: list[Any]) -> None: + self.outcomes = outcomes + self.requests: list[Any] = [] + + def open(self, request: Any, timeout: float) -> FakeResponse: + self.requests.append((request, timeout)) + outcome = self.outcomes.pop(0) + if isinstance(outcome, BaseException): + raise outcome + return outcome + + +def identity_response(*, project_scoped: bool = True) -> FakeResponse: + token: dict[str, Any] = { + "catalog": [ + { + "type": "secrets-manager", + "endpoints": [ + { + "region": "ru-test", + "interface": "public", + "url": "https://secrets.example", + } + ], + } + ] + } + if project_scoped: + token["project"] = {"id": "project-id"} + return FakeResponse( + 201, + json.dumps({"token": token}).encode(), + {"X-Subject-Token": "iam-token-do-not-log"}, + ) + + +def secret_response(value: bytes) -> FakeResponse: + return FakeResponse( + 200, json.dumps({"value": base64.b64encode(value).decode()}).encode() + ) + + +def selectel_config(runtime_dir: Path) -> dict[str, Any]: + return { + "version": 1, + "mode": "selectel", + "runtime_dir": str(runtime_dir), + "http": {"timeout_seconds": 2, "retries": 1, "max_response_bytes": 4096}, + "selectel": { + "account_id": "123456", + "username": "reader", + "project_name": "production", + "region": "ru-test", + "password_file": "selectel-password", + }, + "secrets": { + "DATABASE_URL": { + "remote": "han/database-url", + "consumers": ["api", "migration"], + "max_bytes": 512, + }, + "SHARED_TOKEN": { + "remote": "han/shared-token", + "consumers": ["api"], + "max_bytes": 128, + }, + }, + } + + +def write_json(path: Path, value: dict[str, Any]) -> None: + path.write_text(json.dumps(value), encoding="utf-8") + + +def make_private(path: Path) -> None: + os.chmod(path, 0o600) + + +class SecretsLoaderTests(unittest.TestCase): + def test_decodes_current_secret_from_nested_selectel_version(self) -> None: + encoded = base64.b64encode(b"current-secret").decode() + + value = loader.decode_secret( + {"name": "DATABASE_URL", "version": {"version_id": 1, "value": encoded}}, + "DATABASE_URL", + 1024, + ) + + self.assertEqual(value, b"current-secret") + + def test_empty_literal_never_requires_provider_or_fallback_value(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + fallback = root / "fallback.env" + fallback.write_text("REQUIRED_TOKEN=required-value\n", encoding="utf-8") + make_private(fallback) + config = { + "version": 1, + "mode": "file", + "runtime_dir": str(root / "run"), + "file": {"path": str(fallback)}, + "secrets": { + "REQUIRED_TOKEN": { + "remote": "provider-token", + "consumers": ["service"], + }, + "OPTIONAL_HEADER": { + "literal": "", + "consumers": ["service"], + }, + }, + } + config_path = root / "file.json" + write_json(config_path, config) + + loader.run(config_path, environ={}) + + self.assertEqual((root / "run" / "OPTIONAL_HEADER").read_bytes(), b"") + self.assertIn( + 'OPTIONAL_HEADER=""', + (root / "run" / "service.env").read_text(encoding="utf-8"), + ) + + def test_launcher_uses_public_source_switch_without_exporting_values(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + runtime = root / "runtime" + fallback = root / "fallback.env" + fallback.write_text("TEST_SECRET=canary-secret-value\n", encoding="utf-8") + make_private(fallback) + public = root / ".env" + public.write_text( + "SECRETS_SOURCE=file\nAPP_ENV=production\n", encoding="utf-8" + ) + loader_config = root / "production.file.json" + write_json( + loader_config, + { + "version": 1, + "mode": "file", + "runtime_dir": str(runtime), + "file": {"path": str(fallback)}, + "secrets": { + "TEST_SECRET": { + "remote": "unused-in-file-mode", + "consumers": ["test-service"], + } + }, + }, + ) + probe = ( + "import json,os,pathlib;" + "p=os.environ['TEST_SECRET_FILE'];" + "print(json.dumps({'raw':os.environ.get('TEST_SECRET')," + "'value':pathlib.Path(p).read_text()}))" + ) + result = subprocess.run( + [ + sys.executable, + str(LAUNCHER_PATH), + "run", + "--config", + str(public), + "--loader-config", + str(loader_config), + "--", + sys.executable, + "-c", + probe, + ], + text=True, + capture_output=True, + check=False, + ) + + self.assertEqual(result.returncode, 0, result.stderr) + payload = json.loads(result.stdout) + self.assertIsNone(payload["raw"]) + self.assertEqual(payload["value"], "canary-secret-value") + self.assertEqual( + (runtime / "manifest").read_text(encoding="utf-8").strip(), + f"TEST_SECRET={(runtime / 'TEST_SECRET').resolve()}", + ) + + def test_selectel_flow_uses_project_scope_catalog_and_per_service_files(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + credentials = root / "credentials" + credentials.mkdir() + (credentials / "selectel-password").write_text( + "service-user-password\n", encoding="utf-8" + ) + make_private(credentials / "selectel-password") + runtime = root / "run" + config_path = root / "config.json" + write_json(config_path, selectel_config(runtime)) + opener = FakeOpener( + [ + identity_response(), + secret_response(b"postgresql://user:password@db/app"), + secret_response(b'token with spaces and "quotes"'), + ] + ) + + def factory(**kwargs: Any) -> Any: + return loader.HTTPClient( + **kwargs, opener=opener, sleeper=lambda _: None, jitter=lambda: 0 + ) + + consumers = loader.run( + config_path, + environ={"CREDENTIALS_DIRECTORY": str(credentials)}, + client_factory=factory, + ) + + self.assertEqual(consumers, ["api", "migration"]) + auth_request = opener.requests[0][0] + auth_payload = json.loads(auth_request.data) + self.assertEqual( + auth_payload["auth"]["scope"]["project"]["name"], "production" + ) + self.assertEqual( + auth_payload["auth"]["identity"]["password"]["user"]["domain"]["name"], + "123456", + ) + self.assertEqual( + opener.requests[1][0].get_header("X-auth-token"), + "iam-token-do-not-log", + ) + self.assertEqual( + opener.requests[1][0].full_url, + "https://secrets.example/v1/han%2Fdatabase-url", + ) + api_text = (runtime / "api.env").read_text(encoding="utf-8") + migration_text = (runtime / "migration.env").read_text(encoding="utf-8") + self.assertIn('DATABASE_URL="postgresql://user:password@db/app"', api_text) + self.assertIn( + 'SHARED_TOKEN="token with spaces and \\"quotes\\""', api_text + ) + self.assertNotIn("SHARED_TOKEN", migration_text) + if os.name != "nt": + self.assertEqual(stat.S_IMODE((runtime / "api.env").stat().st_mode), 0o600) + self.assertEqual(stat.S_IMODE(runtime.stat().st_mode), 0o700) + + def test_retry_uses_jitter_and_then_succeeds(self) -> None: + opener = FakeOpener( + [urllib.error.URLError("temporary"), FakeResponse(200, b"{}")] + ) + sleeps: list[float] = [] + client = loader.HTTPClient( + timeout=1, + retries=1, + max_response_bytes=1024, + opener=opener, + sleeper=sleeps.append, + jitter=lambda: 0.25, + ) + result = client.request( + "GET", "https://provider.example/v1/key", expected=frozenset({200}) + ) + self.assertEqual(result.status, 200) + self.assertEqual(sleeps, [0.1875]) + + def test_paired_canonical_names_fetch_provider_version_once(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + credentials = root / "credentials" + credentials.mkdir() + credential = credentials / "selectel-password" + credential.write_text("password", encoding="utf-8") + make_private(credential) + config = selectel_config(root / "run") + config["secrets"]["SHARED_TOKEN"]["remote"] = "han/database-url" + config_path = root / "config.json" + write_json(config_path, config) + opener = FakeOpener([identity_response(), secret_response(b"same-value")]) + + def factory(**kwargs: Any) -> Any: + return loader.HTTPClient(**kwargs, opener=opener) + + loader.run( + config_path, + environ={"CREDENTIALS_DIRECTORY": str(credentials)}, + client_factory=factory, + ) + self.assertEqual(len(opener.requests), 2) + output = (root / "run" / "api.env").read_text(encoding="utf-8") + self.assertIn('DATABASE_URL="same-value"', output) + self.assertIn('SHARED_TOKEN="same-value"', output) + + def test_http_error_is_redacted_and_body_is_not_read(self) -> None: + leaked = b"postgresql://admin:secret@db/app iam-token response-body" + + class ExplodingBody(io.BytesIO): + def read(self, *args: Any, **kwargs: Any) -> bytes: + raise AssertionError("HTTP error body must not be read") + + error = urllib.error.HTTPError( + "https://provider.example/v1/key", + 403, + "body contains a secret", + {}, + ExplodingBody(leaked), + ) + client = loader.HTTPClient( + timeout=1, + retries=0, + max_response_bytes=1024, + opener=FakeOpener([error]), + ) + with self.assertRaises(loader.LoaderError) as caught: + client.request( + "GET", + "https://provider.example/v1/key", + headers={"X-Auth-Token": "iam-token"}, + expected=frozenset({200}), + ) + message = str(caught.exception) + self.assertEqual(message, "provider request failed with HTTP 403") + for forbidden in ("secret", "iam-token", "response-body", "postgresql://"): + self.assertNotIn(forbidden, message) + + def test_unscoped_token_fails_without_replacing_existing_output(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + credentials = root / "credentials" + credentials.mkdir() + (credentials / "selectel-password").write_text("password", encoding="utf-8") + make_private(credentials / "selectel-password") + runtime = root / "run" + runtime.mkdir() + existing = runtime / "api.env" + existing.write_text('DATABASE_URL="old-value"\n', encoding="utf-8") + config_path = root / "config.json" + write_json(config_path, selectel_config(runtime)) + opener = FakeOpener([identity_response(project_scoped=False)]) + + def factory(**kwargs: Any) -> Any: + return loader.HTTPClient(**kwargs, opener=opener) + + with self.assertRaisesRegex(loader.LoaderError, "not project-scoped"): + loader.run( + config_path, + environ={"CREDENTIALS_DIRECTORY": str(credentials)}, + client_factory=factory, + ) + self.assertEqual( + existing.read_text(encoding="utf-8"), + 'DATABASE_URL="old-value"\n', + ) + + def test_invalid_base64_and_size_limit_fail_closed(self) -> None: + with self.assertRaisesRegex(loader.LoaderError, "invalid base64"): + loader.decode_secret({"value": "not-base64!"}, "TOKEN", 128) + with self.assertRaisesRegex(loader.LoaderError, "configured limit"): + loader.decode_secret( + {"value": base64.b64encode(b"too-long").decode()}, "TOKEN", 3 + ) + with self.assertRaisesRegex(loader.LoaderError, "dotenv"): + loader.decode_secret( + {"value": base64.b64encode(b"line1\nline2").decode()}, "TOKEN", 128 + ) + + def test_file_mode_is_explicit_strict_and_narrow(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + source = root / "fallback.env" + source.write_text( + "# exact recovery set\n" + 'DATABASE_URL="postgresql://user:pass@db/app"\n' + "SHARED_TOKEN='literal value'\n", + encoding="utf-8", + ) + make_private(source) + runtime = root / "run" + config = selectel_config(runtime) + config["mode"] = "file" + config.pop("selectel") + config.pop("http") + config["file"] = {"path": str(source)} + config_path = root / "config.json" + write_json(config_path, config) + loader.run(config_path, environ={}) + output = (runtime / "api.env").read_text(encoding="utf-8") + self.assertIn("literal value", output) + self.assertNotIn("selectel", output) + + source.write_text( + "DATABASE_URL=ok\nSHARED_TOKEN=ok\nUNDECLARED=leak\n", + encoding="utf-8", + ) + with self.assertRaisesRegex(loader.LoaderError, "undeclared key"): + loader.run(config_path, environ={}) + + def test_selectel_failure_never_falls_back_to_file_section(self) -> None: + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + config = selectel_config(root / "run") + config["file"] = {"path": str(root / "fallback.env")} + config_path = root / "config.json" + write_json(config_path, config) + with self.assertRaisesRegex(loader.LoaderError, "forbidden"): + loader.run(config_path, environ={}) + + def test_relative_credential_requires_systemd_directory(self) -> None: + selectel = {"password_file": "credential"} + with self.assertRaisesRegex(loader.LoaderError, "CREDENTIALS_DIRECTORY"): + loader.credential_value(selectel, {}) + + def test_response_content_length_limit_is_enforced(self) -> None: + client = loader.HTTPClient( + timeout=1, + retries=0, + max_response_bytes=10, + opener=FakeOpener( + [FakeResponse(200, b"{}", {"Content-Length": "100"})] + ), + ) + with self.assertRaisesRegex(loader.LoaderError, "exceeds"): + client.request( + "GET", "https://provider.example/v1/key", expected=frozenset({200}) + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/modules/module-10-deployment-runbook.md b/modules/module-10-deployment-runbook.md index 3439b6e..49bf57a 100644 --- a/modules/module-10-deployment-runbook.md +++ b/modules/module-10-deployment-runbook.md @@ -344,7 +344,7 @@ git status --short - [ ] SBOM/vulnerability scan без unresolved critical/high. - [ ] Root Compose — единственный production entrypoint. -## 9. Stage 6 — `.env` и secrets +## 9. Stage 6 — несекретный `.env` и runtime secrets ### 9.1. Создание @@ -355,19 +355,24 @@ cp .env.example .env chmod 600 .env ``` -Генерировать минимум 256-bit: +`.env` содержит только несекретный config и `SECRETS_SOURCE=file|selectel`. +Пароли, токены, access keys, credential-bearing DSN и `*_FILE` пути в нём +запрещены. Секреты выдаёт единый интерфейс: ```bash -openssl rand -hex 32 +deployment/secrets/han-secrets run --config .env -- ``` -Не выполнять `export SECRET=...` в shared shell history. Использовать editor с restricted permissions или secret manager/secret files. +Launcher устанавливает `HAN_SECRETS_ACTIVE=1`, не выводит значения и может +передать paths-only manifest через `HAN_RUNTIME_SECRET_MANIFEST`. Файлы manifest +должны быть абсолютными, недоступными group/other. Генерацию выполняет secret +manager; не выполнять `export SECRET=...` и не вставлять значения в history. ### 9.2. Обязательные группы - `APP_ENV`, release/version, log level; -- private PG host/port/database, TLS CA и runtime DSN; -- Redis ACL credentials/URLs DB0/1/2; +- private PG host/port/database и TLS CA в config; runtime DSN в secret backend; +- Redis ACL credentials/URLs DB0/1/2 только в secret backend; - public web/API/auth URLs; - Keycloak realm/audience/hostname/bootstrap/provider technical secrets; - SMS DB URL, парные Keycloak↔SMS tokens, Direct `TOKEN_1`, callback URL и отдельные callback credentials; @@ -390,9 +395,10 @@ Service token и webhook token — разные secrets. ### 9.3. Validation -Добавить/запустить `scripts/validate-env`: +Запустить `scripts/validate-env` в двух независимых режимах: -- mandatory not empty; +- config: mandatory not empty, `SECRETS_SOURCE`, запрет secret keys/DSN credentials; +- runtime: manifest/files или child environment без печати значений; - нет `change-me`, example IP/domain, default OTP; - URLs have correct schemes; - public URLs HTTPS, internal URLs service DNS; @@ -409,14 +415,18 @@ Service token и webhook token — разные secrets. ```bash cd ./scripts/validate-env .env -docker compose config --quiet +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet ``` `docker compose config` может раскрыть resolved secrets; не сохранять/публиковать его stdout. ### Gate 6 -- [ ] `.env` mode 0600, отсутствует в git. +- [ ] `.env` отсутствует в git и содержит только несекретный config. +- [ ] Secret launcher и permissions manifest/files проверены. - [ ] Все placeholder/default secrets отклонены. - [ ] Paired tokens совпадают. - [ ] DSN private/TLS; URLs/issuer согласованы. @@ -1000,10 +1010,13 @@ DB backup включает realm/users/signing keys/provider data. Secret-free r 4. остановить новые claims/send при возможности; 5. переключить image tags на previous digests; 6. не выполнять Alembic downgrade; -7. `docker compose up -d`; +7. `SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true deployment/scripts/rollback.sh `; 8. health/smoke/idempotency; 9. проверить outbox/inbox/SMS pending/uncertain/recovery. +Используется текущий runtime secret set и текущий несекретный config. Snapshot +старого `.env` не создаётся и не применяется. + ### После backward-incompatible migration Обычный rollback запрещён. Выбор: diff --git a/ops-monitoring/diagnose-han-chat.sh b/ops-monitoring/diagnose-han-chat.sh index c42b72c..abde598 100644 --- a/ops-monitoring/diagnose-han-chat.sh +++ b/ops-monitoring/diagnose-han-chat.sh @@ -2,13 +2,23 @@ # diagnose-han-chat.sh — запускать из /opt/han-chat/backend set -euo pipefail cd /opt/han-chat/backend +CONFIG_FILE="${CONFIG_FILE:-.env}" +SECRETS_LAUNCHER="${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}" +if [[ "${HAN_SECRETS_ACTIVE:-0}" != 1 ]]; then + [[ -x "$SECRETS_LAUNCHER" ]] || { + echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2 + exit 66 + } + exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@" +fi +compose() { docker compose --env-file "$CONFIG_FILE" "$@"; } echo "=== 1. Статус сервисов ===" -docker compose --env-file .env ps api-backend keycloak redis message-safety +compose ps api-backend keycloak redis message-safety echo echo "=== 2. Healthcheck api-backend (что именно падает) ===" -docker compose --env-file .env exec -T api-backend \ +compose exec -T api-backend \ python -c " import json, urllib.request try: @@ -22,13 +32,13 @@ except urllib.error.HTTPError as e: echo echo "=== 3. Keycloak health (management port 9000) ===" -docker compose --env-file .env exec -T keycloak bash -c \ +compose exec -T keycloak bash -c \ "exec 3<>/dev/tcp/127.0.0.1/9000 && printf 'GET /auth/health/ready HTTP/1.0\r\n\r\n' >&3 && cat <&3" \ 2>/dev/null | head -20 || echo "Keycloak health FAILED" echo echo "=== 4. api-backend -> Keycloak (JWKS) ===" -docker compose --env-file .env exec -T api-backend \ +compose exec -T api-backend \ python -c " import os, urllib.request, json base = os.environ.get('KEYCLOAK_INTERNAL_URL','http://keycloak:8080/auth').rstrip('/') diff --git a/ops-monitoring/send_sms.md b/ops-monitoring/send_sms.md index 18c2673..8d015ee 100644 --- a/ops-monitoring/send_sms.md +++ b/ops-monitoring/send_sms.md @@ -5,41 +5,25 @@ cd /opt/han-chat/backend umask 077 -read -r -p "Тестовый номер в E.164 (+79...): " TEST_PHONE - -CHALLENGE_ID=$(python3 -c 'import uuid; print(uuid.uuid4())') -OTP_CODE=$(python3 -c 'import secrets; print(f"{secrets.randbelow(1000000):06d}")') - -SMS_TOKEN=$(python3 - <<'PY' -from pathlib import Path - -for line in Path(".env").read_text().splitlines(): - if line.startswith("SMS_SERVICE_TOKEN="): - print(line.split("=", 1)[1].strip().strip("\"'")) - break -else: - raise SystemExit("SMS_SERVICE_TOKEN отсутствует") -PY -) - -export TEST_PHONE CHALLENGE_ID OTP_CODE SMS_TOKEN REQUEST_FILE=$(mktemp) - -python3 - "$REQUEST_FILE" <<'PY' +read -r -p "Тестовый номер в E.164 (+79...): " TEST_PHONE +python3 - "$REQUEST_FILE" "$TEST_PHONE" <<'PY' import json -import os +import secrets import sys +import uuid +challenge_id = str(uuid.uuid4()) payload = { - "idempotency_key": f"ops:smoke:{os.environ['CHALLENGE_ID']}", + "idempotency_key": f"ops:smoke:{challenge_id}", "template_code": "auth_otp", "locale": "ru", - "phone_e164": os.environ["TEST_PHONE"], + "phone_e164": sys.argv[2], "substitutions": { - "code": os.environ["OTP_CODE"], + "code": f"{secrets.randbelow(1000000):06d}", "ttl_min": "1", }, - "customer_ref": os.environ["CHALLENGE_ID"], + "customer_ref": challenge_id, "message_ttl_sec": 60, } @@ -51,16 +35,20 @@ PY ## Создайте функцию отправки: send_sms_smoke() { + deployment/secrets/han-secrets run --config .env -- \ docker compose --env-file .env --profile ops run --rm --no-deps \ --user 0:0 \ --entrypoint sh \ - -e SMS_TOKEN \ + -v /run/han-chat/secrets/SMS_SERVICE_TOKEN:/run/secrets/sms_service_token:ro \ -v "$REQUEST_FILE:/tmp/sms-request.json:ro" \ toolbox -ec ' + umask 077 + printf "Authorization: Bearer %s\n" \ + "$(cat /run/secrets/sms_service_token)" > /tmp/auth-header curl -sS \ -w "\nHTTP %{http_code}\n" \ -X POST \ - -H "Authorization: Bearer $SMS_TOKEN" \ + -H @/tmp/auth-header \ -H "Content-Type: application/json" \ -H "X-Request-ID: ops-sms-smoke" \ --data-binary @/tmp/sms-request.json \ @@ -90,7 +78,7 @@ LIMIT 5; ## После проверки удалите секретные данные: shred -u "$REQUEST_FILE" 2>/dev/null || rm -f "$REQUEST_FILE" -unset SMS_TOKEN OTP_CODE TEST_PHONE CHALLENGE_ID REQUEST_FILE +unset TEST_PHONE REQUEST_FILE # Тесты @@ -138,33 +126,12 @@ HTTP 403 ### 3. Проверить Basic auth внутри Docker-сети -Получите credentials из `.env`: - -```bash -CB_USER=$(python3 - <<'PY' -from pathlib import Path -for line in Path(".env").read_text().splitlines(): - if line.startswith("IDGTL_SMS_CALLBACK_USERNAME="): - print(line.split("=", 1)[1].strip().strip("\"'")) - break -PY -) - -CB_PASS=$(python3 - <<'PY' -from pathlib import Path -for line in Path(".env").read_text().splitlines(): - if line.startswith("IDGTL_SMS_CALLBACK_PASSWORD="): - print(line.split("=", 1)[1].strip().strip("\"'")) - break -PY -) - -export CB_USER CB_PASS -``` +Credentials получает только дочерний процесс через secret launcher. Неверные credentials: ```bash +deployment/secrets/han-secrets run --config .env -- \ docker compose --env-file .env --profile ops run --rm --no-deps \ --entrypoint sh toolbox -ec ' curl -sS -o /dev/null -w "HTTP %{http_code}\n" \ @@ -180,10 +147,20 @@ docker compose --env-file .env --profile ops run --rm --no-deps \ Правильные credentials: ```bash +deployment/secrets/han-secrets run --config .env -- \ docker compose --env-file .env --profile ops run --rm --no-deps \ - --entrypoint sh -e CB_USER -e CB_PASS toolbox -ec ' + --entrypoint sh \ + -v /run/han-chat/secrets/IDGTL_SMS_CALLBACK_USERNAME:/run/secrets/callback_user:ro \ + -v /run/han-chat/secrets/IDGTL_SMS_CALLBACK_PASSWORD:/run/secrets/callback_password:ro \ + toolbox -ec ' + umask 077 + { + printf "user = \"%s:%s\"\n" \ + "$(cat /run/secrets/callback_user)" \ + "$(cat /run/secrets/callback_password)" + } > /tmp/curl-auth.conf curl -sS -o /dev/null -w "HTTP %{http_code}\n" \ - -u "$CB_USER:$CB_PASS" \ + --config /tmp/curl-auth.conf \ -H "Content-Type: application/json" \ --data "[]" \ http://sms-service:8080/callbacks/idgtl/sms @@ -192,12 +169,6 @@ docker compose --env-file .env --profile ops run --rm --no-deps \ Ожидается `HTTP 422`: авторизация прошла, но пустой callback-массив невалиден. -После проверки: - -```bash -unset CB_USER CB_PASS -``` - ### 4. Проверить реальный callback Direct После тестовой SMS: diff --git a/releases/#0 deploy-steps.md b/releases/#0 deploy-steps.md index 9a52c91..0dab391 100644 --- a/releases/#0 deploy-steps.md +++ b/releases/#0 deploy-steps.md @@ -1,555 +1,117 @@ -# usefull commands - - Убрать переносы строк: sed -i 's/\r$//' name-file.sh - - find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} + - - docker compose stop # мягко останавливает контейнеры (не удаляет) - - sudo shutdown -h now # выглючить ВМ - - Генерация паролей - - (со спец.символами) openssl rand -base64 32 - - (без спец.символов) openssl rand -hex 16 - - openssl rand -base64 16 | xclip -selection clipboard # Linux +# Release #0: безопасный порядок развертывания -Посмотреть состояние контейнеров -docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}" +Этот файл больше не является журналом реальных адресов, SSH-ключей и локальных +путей. Фактические значения инфраструктуры хранятся в защищённой CMDB/ops wiki, +а не в Git. -Туннель до БД: ssh -i C:\Users\MI\.ssh\hansel -L 5433:192.168.0.211:5432 root@135.106.164.58 -N +## 1. Подготовка ВМ -#Обновление проекта -mkdir -p ~/.ssh -cp /mnt/c/Users/MI/.ssh/hansel ~/.ssh/hansel -chmod 600 ~/.ssh/hansel +Разрешите извне только 80/443 и SSH из административной сети. PostgreSQL +доступен ВМ только через приватную сеть. -1. Скопировать и автоматически почистить артефакты -Когда вы убедитесь, что вывод предыдущей команды вас устраивает, просто уберите флаг -n: -rsync -rltD --no-perms --no-owner --no-group -ivc --delete \ +```sh +scp -i deployment/scripts/setup-vm.sh \ + @:/tmp/setup-vm.sh +ssh -i @ +sudo chmod 0755 /tmp/setup-vm.sh +sudo /tmp/setup-vm.sh +``` + +После создания пользователя `deploy` проверьте отдельную SSH-сессию до +отключения root login. Не копируйте приватный ключ на ВМ. + +## 2. Доставка приложения + +Предпочтительно использовать Git либо архив без runtime-данных: + +```sh +tar -C HAN_chat_specification/codebase/backend \ --exclude='.env' \ - --exclude='*.crt' \ - --exclude='*.pem' \ - --exclude='*.key' \ - --exclude='secrets/' \ - --exclude='node_modules/' \ - --exclude='dist/' \ - --exclude='.expo/' \ - --exclude='playwright-report/' \ - --exclude='test-results/' \ - --exclude='*.log' \ - -e "ssh -i ~/.ssh/hansel" \ - /mnt/c/Users/MI/Documents/Assistent/HAN_chat_specification/codebase/backend/ \ - root@135.106.164.58:/opt/han-chat/backend/ + --exclude='secrets' \ + --exclude='backups' \ + -czf han-chat-backend.tar.gz . +scp -i han-chat-backend.tar.gz deploy@:/tmp/ +``` --r — рекурсивно. --l — сохранять символические ссылки. --t — сохранять время модификации (важно для будущих проверок). --D — сохранять устройства (на всякий случай, как в -a). ---no-perms --no-owner --no-group — главное исправление: не пытаться копировать права, владельца и группу с Windows на Linux. Это избавит от ложных срабатываний. --i — покажет только реально измененные файлы (можно заменить на -v, если хотите просто список). --a (archive) — сохраняет права, время и рекурсивно копирует. --v (verbose) — выводит список файлов. --n (dry-run) — главный флаг, показывает, что бы произошло, но не делает этого. ---delete — решение вашей проблемы. Говорит rsync удалять на приемнике (ВМ) файлы, которых нет в источнике (локально). -Важно: не забудьте поставить слэш / в конце пути к локальному проекту, иначе rsync скопирует саму папку внутрь папки на ВМ. - -2. Копирование env (опционально) -scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/backend +На ВМ: +```sh +sudo install -d -o deploy -g deploy -m 0755 /opt/han-chat/backend +sudo -u deploy tar -C /opt/han-chat/backend \ + -xzf /tmp/han-chat-backend.tar.gz cd /opt/han-chat/backend -find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} + -chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh - -docker compose --env-file .env build frontend-static keycloak -docker compose --env-file .env up -d \ - --no-deps \ - --force-recreate frontend-static keycloak - -Посмотреть состояние контейнеров -docker compose ps --format "table {{.Service}}\t{{.Status}}" -docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}" - -# Архивный способ копирования: - -cd /tmp -rm han-chat-backend.tar.gz -cd /opt/han-chat/backend - -#команда складывает архив в ту папку, из которой запускается команда -cd C:\Users\MI\Documents\Assistent\ -rm C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz -tar -C C:\Users\MI\Documents\Assistent\HAN_chat_specification\codebase\backend -czf han-chat-backend.tar.gz . -scp -i C:\Users\MI\.ssh\hansel C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz root@135.106.164.58:/tmp/han-chat-backend.tar.gz - -tar -xzf /tmp/han-chat-backend.tar.gz -find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} + -chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh - -docker compose --env-file .env build frontend-static keycloak -docker compose --env-file .env up -d \ - --no-deps \ - --force-recreate frontend-static keycloak - -# Разворачиваем инфраструктуру в Селектел ч1 - -## Создание сети - -## Создание групп безопасности для сети и для БД - Для сети открываем порты 80, 22, 443 - Для БД открываем порты 5432, 5433 но только из CIDR (192.168.0.211\24) - -## Создаем ВМ, подключаем к новой сети (через порт) и перезагружаем (либо руками вводить новые сетевые настройки - надо выбрать) - -# Подключение к ВМ и первичный скрипт -++ ssh -i C:\Users\MI\.ssh\hansel root@135.106.164.58 -++ scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\Assistent\HAN_chat_specification\codebase\backend\deployment\scripts\setup-vm.sh" root@135.106.164.58:/tmp/setup-vm.sh - chmod +x /tmp/setup-vm.sh - sudo /tmp/setup-vm.sh - -# Скрипт создал деплой пользователя. Нужно Перенести туда публичный ключ и дать права на файл и папку - -## Вставляем публичный ключ новой строкой Далее даем права -sudo nano /home/deploy/.ssh/authorized_keys -sudo chmod 700 /home/deploy/.ssh -sudo chmod 600 /home/deploy/.ssh/authorized_keys -sudo chown -R deploy:deploy /home/deploy/.ssh - -## Пароль деплою для использования sudo -sudo passwd deploy - -## После этого можно подключаться под деплой пользователем -ssh -i C:\Users\MI\.ssh\han_chat_deploy deploy@135.106.164.58 - -## GIT не работал. Проект пришлось копировать. -tar -C HAN_chat_specification/codebase/backend -czf han-chat-backend.tar.gz . -scp -i C:\Users\MI\.ssh\han_chat_deploy C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz deploy@135.106.164.58:/tmp/ - -scp -i C:\Users\MI\.ssh\hansel C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz root@135.106.164.58:/tmp/ - - - - -## Дальше распаковка -cd /opt/han-chat/backend -tar -xzf /tmp/han-chat-backend.tar.gz - -## Правим переносы строк и даем права на исполнение -find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} + -chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh - -# Разворачиваем инфраструктуру в Селектел ч2 - -## Создаем БД -В базе подключаем pgcrypto -Чтобы корректно работал PgBouncer выбираем session pooling - -### Копируем СА сертификат - -Вариант Деплоя: -mkdir -p /opt/han-chat/backend/secrets/pg -scp -i C:\Users\MI\.ssh\han_chat_deploy -r C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\sertificates\CA.pem deploy@135.106.164.58:/opt/han-chat/backend/secrets/pg/ca.pem -chmod 644 /opt/han-chat/backend/secrets/pg/ca.pem - -Вариант Селектела: -mkdir -p ~/.postgresql/ -wget https://storage.dbaas.selcloud.ru/CA.pem -O ~/.postgresql/root.crt -chmod 0600 ~/.postgresql/root.crt - -### Правки DNS-маршрутизации (в прошлый раз, какие-то трабблы были с базовой маршрутизацией) - скрипт ниже создаст -sudo tee /etc/netplan/99-han-dns.yaml <<'EOF' -network: - version: 2 - ethernets: - eth0: - nameservers: - addresses: - - 8.8.8.8 - - 1.1.1.1 - search: - - selcloud.ru - eth1: - dhcp4-overrides: - use-dns: false - use-domains: false -EOF - -sudo chmod 600 /etc/netplan/99-han-dns.yaml -sudo netplan apply -resolvectl flush-caches - -### Проверка подключения к Postgers в ВМ -nc -zv 192.168.0.211 5433 - -sudo apt update -sudo apt install postgresql-client - -psql "host=master.ef54e3e4-ad3d-4b80-a6af-d63269e0895a.c.dbaas.selcloud.ru \ - port=5432 \ - dbname=han_chat \ - user=dbAdmin \ - sslmode=verify-ca" - -(psql "host=master.ef54e3e4-ad3d-4b80-a6af-d63269e0895a.c.dbaas.selcloud.ru port=5432 dbname=han_chat user=dbAdmin sslmode=verify-ca") - -### Заводим пользователей и схемы - -Пользователей создаем через интерфейс селектела. -Под админом даем права на создание в БД: - GRANT CREATE ON DATABASE han_chat TO han_app; - GRANT CREATE ON DATABASE han_chat TO bitrix_local_app; - GRANT CREATE ON DATABASE han_chat TO bitrix_sync_user; - GRANT CREATE ON DATABASE han_chat TO message_safety_app; - GRANT CREATE ON DATABASE han_chat TO keycloak_user; - GRANT CREATE ON DATABASE han_chat TO sms_user - - Если создаем пользователей после того как отозвали права from public, надо давать гранты на коннект: - GRANT CONNECT ON DATABASE han_chat TO sms_user - -Схемы создаем от лица пользователей, заходя каждым из них в БД. -+ Запрещаем всем посторонним входить в схему han_app и др. -CREATE SCHEMA IF NOT EXISTS han_app; -REVOKE ALL ON SCHEMA han_app FROM PUBLIC; -ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO han_app; -SHOW search_path; --чтобы заработало надо переподключиться (должно быть han_app) - -CREATE SCHEMA IF NOT EXISTS bitrix_local; -REVOKE ALL ON SCHEMA bitrix_local FROM PUBLIC; -ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO bitrix_local; -SHOW search_path; --чтобы заработало надо переподключиться - -CREATE SCHEMA IF NOT EXISTS bitrix_sync; -REVOKE ALL ON SCHEMA bitrix_sync FROM PUBLIC; -ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO bitrix_sync; -SHOW search_path; --чтобы заработало надо переподключиться - -CREATE SCHEMA IF NOT EXISTS message_safety; -REVOKE ALL ON SCHEMA message_safety FROM PUBLIC; -ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO message_safety; -SHOW search_path; --чтобы заработало надо переподключиться - -CREATE SCHEMA IF NOT EXISTS keycloak; -REVOKE ALL ON SCHEMA keycloak FROM PUBLIC; -ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO keycloak; -SHOW search_path; --чтобы заработало надо переподключиться - -CREATE SCHEMA IF NOT EXISTS sms AUTHORIZATION sms_user; -REVOKE ALL ON SCHEMA sms FROM PUBLIC; -ALTER ROLE sms_user IN DATABASE han_chat SET search_path TO sms, public; -SHOW search_path; - - -Проверка search_path -SELECT r.rolname, d.datname, s.setconfig -FROM pg_db_role_setting s -JOIN pg_roles r ON r.oid = s.setrole -JOIN pg_database d ON d.oid = s.setdatabase -WHERE r.rolname in ('han_app', - 'bitrix_local_app', - 'bitrix_sync_user', - 'message_safety_app', - 'keycloak_user'); - - - -После реализации bitrix_sync (проверить, вероятно не на все таблицы права нужны): --- Даем право на чтение (SELECT) всех СУЩЕСТВУЮЩИХ таблиц в схеме -GRANT SELECT, INSERT ON ALL TABLES IN SCHEMA han_app TO bitrix_sync_user; --- Настраиваем права по умолчанию для новых таблиц -ALTER DEFAULT PRIVILEGES IN SCHEMA han_app -GRANT SELECT ON TABLES TO bitrix_sync_user; --- Если нужно дать право на чтение и для новых последовательностей (sequences): -ALTER DEFAULT PRIVILEGES IN SCHEMA han_app -GRANT USAGE, SELECT ON SEQUENCES TO bitrix_sync_user; - -## Заводим S3 хранилища: - -Создаем 2 сервисных пользователя и заводим им ключи: - - API backend SELECTEL_S3_ACCESS_KEY / SELECTEL_S3_SECRET_KEY - - Message Safety SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY / SELECTEL_S3_QUARANTINE_READ_SECRET_KEY - -Создаем 3 приватных бакета - - han-chat-quarantine (политика доступа API backend RW, Message Safety R) - - han-chat-attachments (политика доступа API backend RW) - - han-chat-documents (политика доступа API backend RW) - -RW: - ListBucket - GetBucketLocation - PutObject - GetObject - DeleteObject - ListBucketMultipartUploads - ListMultipartUploadParts - AbortMultipartUpload - -R: - ListBucket - GetBucketLocation - GetObject - ListBucketMultipartUploads - ListMultipartUploadParts - -Политика CORS (бакет han-chat-quarantine, vHosted обязателен): - Allowed origin: https://chat.han0107.ru - Methods: POST, PUT, GET, HEAD - Headers: * (или явно content-type; wildcard x-amz-* в Selectel не работает) - Expose headers: ETag - -# Тест инфраструктуры - -mkdir -p /opt/han-chat/infratest -cd /opt/han-chat/infratest - -scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\Assistent\HAN_chat_specification\infratest\*" root@135.106.164.58:/opt/han-chat/infratest - ---настраиваю env на локальной машине и копирую на диск -scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/infratest/ - - -Создаем изолированное окружение: -python3 -m venv .venv -. .venv/bin/activate -python3 -m pip install --upgrade pip -python3 -m pip install -r requirements.txt - -Если не работает, то надо DNS переправить на eth0 -resolvectl dns eth0 1.1.1.1 8.8.8.8 -resolvectl domain eth0 '~.' -resolvectl flush-caches -resolvectl query pypi.org - -# Подготовка к запуску - -Сертификат рекомендуется скопировать в папку с секретами -cp /root/.postgresql/root.crt /opt/han-chat/backend/secrets/pg/root.crt -chmod 644 /opt/han-chat/backend/secrets/pg/root.crt - -В битрикс регистрируем локальное приложение: -Установка: https://chat.example.ru/bitrix/install -Обработчик: https://chat.example.ru/bitrix/handler -??? (не понадобилось) Placement: https://chat.example.ru/bitrix/placement - ---настраиваю env на локальной машине и копирую на диск -scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/backend - ---Проверка .env -cd /opt/han-chat/backend -chmod 600 .env +sudo find . -type f \( -name '*.sh' -o -name 'validate-env' \ + -o -name 'han-secrets' -o -name 'han-compose' \) \ + -exec dos2unix {} + +sudo chmod 0755 scripts/validate-env deployment/scripts/*.sh \ + deployment/secrets/han-secrets deployment/secrets/han-compose \ + redis/scripts/*.sh nginx/scripts/*.sh +sudo deployment/scripts/setup-vm.sh +``` + +## 3. Несекретная конфигурация + +`.env` создаётся на самой ВМ из `.env.example` и содержит только URL, resource +names, feature flags и `SECRETS_SOURCE`. Его разрешено передавать как обычный +config, но запрещено добавлять credential-bearing DSN, password, token и key. + +```sh +cp .env.example .env +chmod 0600 .env +nano .env ./scripts/validate-env .env -docker compose --env-file .env config --quiet -docker compose --env-file .env config --services +``` + +## 4. Selectel Secrets Manager + +Выполните `deployment/secrets/SELECTEL_RUNBOOK.ru.md`: + +1. отдельный проект и service user; +2. provider secrets и audit alerts; +3. root-only JSON-карта; +4. encrypted systemd credential; +5. `SECRETS_SOURCE=selectel`. + +Не передавайте значение секрета аргументом команды, через `export`, тикет или +shell history. Старые значения из прежнего `.env` после cutover ротируются. + +## 5. Проверка и запуск + +```sh +sudo systemctl daemon-reload +sudo systemctl enable han-secrets@production.service +sudo systemctl restart han-secrets@production.service +sudo ./scripts/validate-env .env \ + --runtime-manifest /run/han-chat/secrets/manifest +sudo deployment/secrets/han-compose config --quiet python3 -m unittest discover -s tests -v - ---Проверка портов (Публиковаться должны только 80 и 443 у nginx) -docker compose --env-file .env config | grep -n 'published:' - ---Собираем локальные образы: -docker compose --env-file .env build --pull - -docker compose --env-file .env images - -## Миграции БД и начальные настройки -cd /opt/han-chat/backend -PITR_MARKER_CONFIRMED=true deployment/scripts/migrate.sh -deployment/scripts/seed.sh - -## Запуск внутренних сервисов - -Сначала запустите Redis: - -```sh -docker compose --env-file .env up -d redis -docker compose --env-file .env ps redis +sudo deployment/secrets/han-compose build --pull +sudo deployment/scripts/migrate.sh +sudo deployment/secrets/han-compose up -d --wait +sudo deployment/scripts/smoke.sh ``` -Затем Keycloak и OpenTelemetry: +Для диагностики используйте `han-compose ps` и ограниченные logs. Не выводите +resolved Compose config, `docker inspect` environment, полный `env` или secret +files. + +## 6. Откат + +Откат приложения использует immutable image/release ID и подтверждённую +совместимость схемы: ```sh -docker compose --env-file .env up -d keycloak otel-collector -docker compose --env-file .env ps keycloak otel-collector +sudo SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \ + deployment/scripts/rollback.sh ``` -Первый запуск Keycloak может занять несколько минут: он создаст свои таблицы и -импортирует realm `han-chat`. +Откат секрета выполняется активацией предыдущей версии в Selectel, повторным +sync и пересозданием только затронутых сервисов. Snapshot старого `.env` не +создаётся. -После готовности Keycloak: +## 7. Break-glass -```sh -docker compose --env-file .env up -d message-safety -docker compose --env-file .env up -d api-backend -docker compose --env-file .env up -d bitrix-local-app bitrix-sync -docker compose --env-file .env up -d \ - delivery-worker safety-recovery-worker cleanup-worker -docker compose --env-file .env ps -``` - -Если сервис не становится healthy: - -```sh -docker compose --env-file .env logs --tail=200 -docker inspect "$(docker compose --env-file .env ps -q )" -``` - -## Первоначальный выпуск TLS-сертификата - -Для ACME требуется работающий nginx по HTTP. В `.env` оставьте -`NGINX_TLS_ENABLED=true`, но первый nginx запустите с временным переопределением: - -```sh -NGINX_TLS_ENABLED=false \ - docker compose --env-file .env up -d frontend-static nginx -``` - -Проверьте HTTP: - -```sh -curl -I http://chat.han0107.ru/ -``` - -Сначала рекомендуется проверить Certbot через staging: - -```sh -docker compose --env-file .env --profile certbot run --rm certbot certonly \ - --staging \ - --webroot -w /var/www/certbot \ - -d chat.han0107.ru \ - --cert-name chat.han0107.ru-staging \ - --email ap@han.ru \ - --agree-tos --no-eff-email --non-interactive -``` - -После успешного staging-теста выпустите рабочий сертификат с основным cert-name -без `--staging`: - -```sh -docker compose --env-file .env --profile certbot run --rm certbot certonly \ - --webroot -w /var/www/certbot \ - -d chat.han0107.ru \ - --cert-name chat.han0107.ru \ - --email ap@han.ru \ - --agree-tos --no-eff-email --non-interactive -``` - -Пересоздайте nginx уже с TLS: - -```sh -docker compose --env-file .env up -d --force-recreate nginx -docker compose --env-file .env exec -T nginx nginx -t -c /tmp/nginx.conf -curl -I https://chat.han0107.ru/ -``` - -Повторно запустите VM setup, чтобы он обнаружил проект и установил systemd-таймер -продления сертификата: - -```sh -sudo /opt/han-chat/backend/deployment/scripts/setup-vm.sh -systemctl status han-chat-ssl-renew.timer - -## Запуск всего контура - -Теперь можно привести весь проект к состоянию, описанному Compose: - -```sh -cd /opt/han-chat/backend -docker compose --env-file .env up -d -docker compose --env-file .env up -d --wait \ - api-backend keycloak sms-service bitrix-local-app - -# Nginx разрешает имена upstream при загрузке конфигурации. Если upstream -# был пересоздан и получил новый Docker IP, без reload edge продолжит ходить -# на старый адрес и вернёт 502. -docker compose --env-file .env exec -T nginx nginx -t -c /tmp/nginx.conf -docker compose --env-file .env kill -s HUP nginx - -docker compose --env-file .env ps -``` - -Проверьте, что контейнеры не перезапускаются: - -```sh -docker compose --env-file .env ps -docker compose --env-file .env logs --since=10m -``` - - -## Публичная проверка - -Запустите smoke-тест: - -```sh -cd /opt/han-chat/backend -deployment/scripts/smoke.sh -``` - -Также вручную проверьте: - -```sh -curl -fsS https://chat.han0107.ru/api/v1/public/app-config | jq -curl -fsS https://chat.han0107.ru/api/v1/public/content | jq -curl -fsS \ - https://chat.han0107.ru/auth/realms/han-chat/.well-known/openid-configuration | jq -``` - -Если после `up`, `build`, `pull`, rollback или `--force-recreate` менялись -`api-backend`, `keycloak`, `sms-service` либо `bitrix-local-app`, всегда -повторяйте проверку `/tmp/nginx.conf` и `docker compose kill -s HUP nginx`. -Команды `nginx -t`/`nginx -s reload` без `-c /tmp/nginx.conf` здесь неверны: -контейнер read-only и рабочий PID расположен в `/tmp/nginx.pid`. - -Внутренний API не должен быть опубликован: - -```sh -curl -i https://chat.han0107.ru/internal/safety/v1/messages/check -``` - -Ожидаемый статус — `404`. - -Откройте в браузере: - -```text -https://chat.example.ru/ -``` - -Для тестовой авторизации используйте значение `KEYCLOAK_OTP_MOCK_CODE` из `.env`. - - -Настройки keykcloack: - - - - -Для битрикса код установки приложения: -docker compose --env-file .env exec -T api-backend python - <<'PY' -import os -import urllib.request - -base = os.environ["BITRIX_LOCAL_APP_BASE_URL"] -token = os.environ["BITRIX_LOCAL_APP_INTERNAL_TOKEN"] - -request = urllib.request.Request( - base + "/internal/openlines/v1/setup/retry", - method="POST", - headers={"Authorization": "Bearer " + token}, -) -print(urllib.request.urlopen(request).read().decode()) -PY - -## План включения реальной SMS-авторизации - -Этот раздел — чек-лист будущего release из `modules/module-11-idgtl-sms.md`, а не подтверждение готовности текущего Compose. Пока отсутствуют реализованные `sms-service`/worker, migrations, callback route и env validation, оставлять `KEYCLOAK_OTP_MOCK_ENABLED=true`. - -Prerequisites без placeholders: - -- согласованные i-Digital sender и active approved template `auth_otp` с placeholders `code`, `ttl_min`; -- выданный Direct `TOKEN_1` (`IDGTL_SMS_API_KEY`, без повторного Base64); -- отдельные random callback username/password и публичный HTTPS URL; -- повторно подтверждённый source IP callback Direct; -- фактический статический egress IP, измеренный из `sms-worker`, записанный в inventory и переданный Direct для allowlist; при динамическом IP сначала настроить NAT/static IP. - -Rollout: - -1. Seed новых `otp.phone.*` в App DB. -2. Создать schema/role `sms`, применить versioned migrations и seed template/settings. -3. Проверить Keycloak→`sms-service`→локальный mock Direct в test environment. -4. Развернуть production `sms-service`/worker и nginx callback route, не выключая mock. -5. Применить Keycloak expand migration/SPI; прежние незавершённые challenges истечь по module-11. -6. Выполнить provider smoke на контролируемом номере; проверить `sms_message_id`, journal, callback и redaction. -7. Переключить `KEYCLOAK_OTP_MOCK_ENABLED=false`. -8. Проверить resend→`superseded`, expiry snapshot, limits и то, что Direct reject/timeout после durable order не меняет verify. - -Rollback: вернуть Keycloak в mock mode; не удалять schema/journal. Остановить новые real orders, дать worker завершить либо зафиксировать in-flight/`uncertain`. Schema downgrade только при доказанной backward compatibility, иначе forward-fix. \ No newline at end of file +Только при подтверждённом инциденте доставьте root-only recovery file из +защищённой офлайн-копии, установите `SECRETS_SOURCE=file`, выполните +sync/validate/recreate и зафиксируйте событие. Автоматический fallback запрещён. +После восстановления Selectel верните штатный режим и удалите recovery file.