Перенесены секреты из .env в SM
This commit is contained in:
@@ -49,6 +49,15 @@
|
|||||||
21. Спрятать сеть за балансировщиком нагрузки
|
21. Спрятать сеть за балансировщиком нагрузки
|
||||||
22. Автопродление TLS падает при перезагрузке nginx; сертификат действует до 14.10.2026. (Исправить reload внутри контейнера и проверить systemctl start an-chat-ssl-renew.service до успешного завершения.)
|
22. Автопродление TLS падает при перезагрузке nginx; сертификат действует до 14.10.2026. (Исправить reload внутри контейнера и проверить systemctl start an-chat-ssl-renew.service до успешного завершения.)
|
||||||
23. WireGuard-only SSH.
|
23. WireGuard-only SSH.
|
||||||
|
25. Перенести секреты из .env в KMS Selectel.
|
||||||
|
26. Запрет входа под root: В /etc/ssh/sshd_config установите PermitRootLogin no. Заходите под обычным пользователем (например, deploy) и используйте sudo для админских задач.
|
||||||
|
27. Удалите все ненужные пакеты, компиляторы (gcc, make) и сервисы. Чем меньше программ на сервере, тем меньше потенциальных уязвимостей.
|
||||||
|
28. Монтирование с флагами безопасности: Разделы диска (особенно /tmp и /var/tmp) следует монтировать с флагами noexec (запрет запуска исполняемых файлов) и nosuid (игнорирование битов setuid).
|
||||||
|
29. Systemd-ограничения: используйте директивы в юните
|
||||||
|
NoNewPrivileges=yes # Запрещает повышение привилегий через setuid
|
||||||
|
ProtectSystem=strict # Делает всю ОС доступной только для чтения
|
||||||
|
PrivateTmp=yes # Дает процессу свой изолированный /tmp
|
||||||
|
ProtectHome=yes # Скрывает домашние директории пользователей
|
||||||
|
|
||||||
24. Добавить логи (Для Python-сервисов добавить OTLP Log Exporter: api-backend; sms-service; sms-worker. Подключить LoggerProvider, BatchLogRecordProcessor и bounded queue. Передавать resource attributes: service.name; service.version; deployment.environment; service.namespace=han-chat.) Экспортировать структурированные поля request_id, trace_id, span_id, severity и event name. Оставить stdout как аварийный локальный журнал. Добавить canary-тесты, запрещающие экспорт токенов, cookie, телефонов, email, текстов сообщений, SQL и object keys.).
|
24. Добавить логи (Для Python-сервисов добавить OTLP Log Exporter: api-backend; sms-service; sms-worker. Подключить LoggerProvider, BatchLogRecordProcessor и bounded queue. Передавать resource attributes: service.name; service.version; deployment.environment; service.namespace=han-chat.) Экспортировать структурированные поля request_id, trace_id, span_id, severity и event name. Оставить stdout как аварийный локальный журнал. Добавить canary-тесты, запрещающие экспорт токенов, cookie, телефонов, email, текстов сообщений, SQL и object keys.).
|
||||||
25. Nginx metrics/tracing в signoz
|
25. Nginx metrics/tracing в signoz
|
||||||
|
|||||||
@@ -1,4 +1,7 @@
|
|||||||
# Copy to .env, replace every placeholder, then run: ./scripts/validate-env .env
|
# Non-secret deployment configuration only.
|
||||||
|
# Copy to .env, replace placeholders, then run: ./scripts/validate-env .env
|
||||||
|
# Secrets are supplied at runtime by deployment/secrets/han-secrets.
|
||||||
|
SECRETS_SOURCE=file
|
||||||
APP_ENV=production-like
|
APP_ENV=production-like
|
||||||
RELEASE_VERSION=change-me-release
|
RELEASE_VERSION=change-me-release
|
||||||
LOG_LEVEL=INFO
|
LOG_LEVEL=INFO
|
||||||
@@ -12,22 +15,15 @@ BITRIX_SYNC_IMAGE=han-chat-bitrix-sync:local
|
|||||||
KEYCLOAK_IMAGE=han-chat-keycloak:local
|
KEYCLOAK_IMAGE=han-chat-keycloak:local
|
||||||
SMS_SERVICE_IMAGE=han-chat-sms-service:local
|
SMS_SERVICE_IMAGE=han-chat-sms-service:local
|
||||||
|
|
||||||
# Managed PostgreSQL is external to Compose. All production DSNs must verify TLS.
|
# Managed PostgreSQL is external to Compose. Credential-bearing DSNs are secrets.
|
||||||
HAN_PG_HOST=managed-pg.private.example
|
HAN_PG_HOST=managed-pg.private.example
|
||||||
HAN_PG_PORT=5433
|
HAN_PG_PORT=5433
|
||||||
HAN_PG_DATABASE=han_chat
|
HAN_PG_DATABASE=han_chat
|
||||||
PG_CA_HOST_PATH=/opt/han-chat/secrets/pg/ca.pem
|
PG_CA_HOST_PATH=/opt/han-chat/secrets/pg/ca.pem
|
||||||
# PgBouncer session mode uses each role's database-level search_path; do not pass libpq options.
|
# PgBouncer session mode uses database-level search_path; runtime DSNs must not pass options/currentSchema.
|
||||||
DATABASE_URL=postgresql+asyncpg://han_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
BITRIX_DATABASE_URL=postgresql://bitrix_local_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
BITRIX_SYNC_APP_DATABASE_URL=postgresql://bitrix_sync_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
BITRIX_SYNC_DATABASE_URL=postgresql://bitrix_sync_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
MESSAGE_SAFETY_DATABASE_URL=postgresql://message_safety_app:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
SMS_DATABASE_URL=postgresql+asyncpg://sms_user:change-me@managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
KEYCLOAK_DB_URL=jdbc:postgresql://managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
KEYCLOAK_DB_URL=jdbc:postgresql://managed-pg.private.example:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
||||||
KEYCLOAK_DB_SCHEMA=keycloak
|
KEYCLOAK_DB_SCHEMA=keycloak
|
||||||
KEYCLOAK_DB_USERNAME=keycloak_user
|
KEYCLOAK_DB_USERNAME=keycloak_user
|
||||||
KEYCLOAK_DB_PASSWORD=change-me
|
|
||||||
|
|
||||||
PUBLIC_HOST=chat.example.ru
|
PUBLIC_HOST=chat.example.ru
|
||||||
PUBLIC_WEB_URL=https://chat.example.ru
|
PUBLIC_WEB_URL=https://chat.example.ru
|
||||||
@@ -70,61 +66,21 @@ KEYCLOAK_REALM=han-chat
|
|||||||
KEYCLOAK_AUDIENCE=han-chat-api
|
KEYCLOAK_AUDIENCE=han-chat-api
|
||||||
# На мок среде true,true. На продакшн false,false.
|
# На мок среде true,true. На продакшн false,false.
|
||||||
KEYCLOAK_OTP_MOCK_ENABLED=true
|
KEYCLOAK_OTP_MOCK_ENABLED=true
|
||||||
KEYCLOAK_OTP_MOCK_CODE=change-me
|
|
||||||
KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false
|
KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false
|
KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false
|
||||||
# Обязательны только при KEYCLOAK_YANDEX_CAPTCHA_ENABLED=true.
|
# Public site key; the server key stays in Secrets Manager when CAPTCHA is enabled.
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=
|
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=
|
|
||||||
# (openssl rand -hex 32)
|
|
||||||
KEYCLOAK_OTP_HMAC_KEY=change-me
|
|
||||||
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC=300
|
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC=300
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_URL=http://api-backend:8000/internal/settings/v1/otp
|
KEYCLOAK_SETTINGS_BRIDGE_URL=http://api-backend:8000/internal/settings/v1/otp
|
||||||
KEYCLOAK_SMS_SERVICE_URL=http://sms-service:8080
|
KEYCLOAK_SMS_SERVICE_URL=http://sms-service:8080
|
||||||
# Должен совпадать с SMS_SERVICE_TOKEN.
|
|
||||||
KEYCLOAK_SMS_SERVICE_TOKEN=change-me
|
|
||||||
KEYCLOAK_ADMIN=bootstrap-admin
|
KEYCLOAK_ADMIN=bootstrap-admin
|
||||||
# (openssl rand -hex 32)
|
|
||||||
KEYCLOAK_ADMIN_PASSWORD=change-me
|
|
||||||
# (openssl rand -hex 32)
|
|
||||||
CURSOR_HMAC_SECRET=change-me
|
|
||||||
|
|
||||||
# Redis пароль 1 (openssl rand -hex 32)
|
|
||||||
REDIS_API_PASSWORD=change-me
|
|
||||||
REDIS_URL=redis://api_backend:change-me@redis:6379/0
|
|
||||||
REDIS_REALTIME_URL=redis://api_backend:change-me@redis:6379/1
|
|
||||||
# Redis пароль 2 (openssl rand -hex 32)
|
|
||||||
REDIS_SAFETY_PASSWORD=change-me
|
|
||||||
MESSAGE_SAFETY_REDIS_URL=redis://message_safety:change-me@redis:6379/2
|
|
||||||
# Redis пароль 3 (openssl rand -hex 32)
|
|
||||||
REDIS_HEALTH_PASSWORD=change-me
|
|
||||||
|
|
||||||
REDIS_MAXMEMORY=384mb
|
REDIS_MAXMEMORY=384mb
|
||||||
REDIS_EVICTION_POLICY=volatile-lru
|
REDIS_EVICTION_POLICY=volatile-lru
|
||||||
|
|
||||||
#token1 (openssl rand -hex 32)
|
# i-Digital Direct non-secret configuration.
|
||||||
MESSAGE_SAFETY_SERVICE_TOKEN=change-me
|
|
||||||
#token2 (openssl rand -hex 32)
|
|
||||||
BITRIX_LOCAL_APP_INTERNAL_TOKEN=change-me
|
|
||||||
BITRIX_INTERNAL_API_TOKEN=change-me
|
|
||||||
#token3 (openssl rand -hex 32)
|
|
||||||
BITRIX_API_FORWARD_TOKEN=change-me
|
|
||||||
BITRIX_API_INBOX_TOKEN=change-me
|
|
||||||
#token4 (openssl rand -hex 32)
|
|
||||||
BITRIX_SYNC_SERVICE_TOKEN=change-me
|
|
||||||
#token5 (openssl rand -hex 32)
|
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_TOKEN=change-me
|
|
||||||
#token6 (openssl rand -hex 32), должен совпадать с KEYCLOAK_SMS_SERVICE_TOKEN
|
|
||||||
SMS_SERVICE_TOKEN=change-me
|
|
||||||
# Тестовый продюсер Internal Notifications API. В БД хранится только hash.
|
|
||||||
NOTIFICATIONS_TOKEN_PRODUCER_TEST=change-me
|
|
||||||
|
|
||||||
# i-Digital Direct. Перед production заменить placeholders согласованными значениями.
|
|
||||||
IDGTL_SMS_BASE_URL=https://direct.i-dgtl.ru
|
IDGTL_SMS_BASE_URL=https://direct.i-dgtl.ru
|
||||||
IDGTL_SMS_API_KEY=change-me
|
|
||||||
IDGTL_SMS_CALLBACK_PUBLIC_URL=https://chat.example.ru/callbacks/idgtl/sms
|
IDGTL_SMS_CALLBACK_PUBLIC_URL=https://chat.example.ru/callbacks/idgtl/sms
|
||||||
IDGTL_SMS_CALLBACK_USERNAME=change-me
|
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD=change-me
|
|
||||||
|
|
||||||
BITRIX_LOCAL_APP_BASE_URL=http://bitrix-local-app:8080
|
BITRIX_LOCAL_APP_BASE_URL=http://bitrix-local-app:8080
|
||||||
BITRIX_API_INBOX_PATH=/internal/openlines/v1/inbox
|
BITRIX_API_INBOX_PATH=/internal/openlines/v1/inbox
|
||||||
@@ -139,28 +95,17 @@ MESSAGE_SAFETY_RULES_VERSION=2026-01-01
|
|||||||
#Отключение синхронизации (при отключенной синхронизации параметры ниже не работают)
|
#Отключение синхронизации (при отключенной синхронизации параметры ниже не работают)
|
||||||
BITRIX_SYNC_ENABLED=false
|
BITRIX_SYNC_ENABLED=false
|
||||||
BITRIX_SYNC_CRM_BASE_URL=https://example.bitrix24.ru
|
BITRIX_SYNC_CRM_BASE_URL=https://example.bitrix24.ru
|
||||||
BITRIX_SYNC_CRM_WEBHOOK_URL=change-me
|
|
||||||
BITRIX_SYNC_CONTACT_MAP_INTERVAL_SEC=60
|
BITRIX_SYNC_CONTACT_MAP_INTERVAL_SEC=60
|
||||||
BITRIX_SYNC_CONTACT_UPDATE_INTERVAL_SEC=30
|
BITRIX_SYNC_CONTACT_UPDATE_INTERVAL_SEC=30
|
||||||
BITRIX_SYNC_CRM_MAX_CONCURRENCY=2
|
BITRIX_SYNC_CRM_MAX_CONCURRENCY=2
|
||||||
BITRIX_SYNC_CONTACT_LIST_BATCH_SIZE=50
|
BITRIX_SYNC_CONTACT_LIST_BATCH_SIZE=50
|
||||||
# (openssl rand -hex 32)
|
BITRIX_CLIENT_ID=change-me-client-id
|
||||||
BITRIX_SYNC_WEBHOOK_TOKEN=change-me
|
|
||||||
|
|
||||||
# Сначала регистрируется приложение в Битрикс24, потом тут прописываются секреты
|
|
||||||
BITRIX_CLIENT_ID=change-me
|
|
||||||
BITRIX_CLIENT_SECRET=change-me
|
|
||||||
BITRIX_APPLICATION_TOKEN=change-me
|
|
||||||
|
|
||||||
BITRIX_CONNECTOR_ID=han_mobile_app
|
BITRIX_CONNECTOR_ID=han_mobile_app
|
||||||
BITRIX_CONNECTOR_NAME=HAN Mobile App
|
BITRIX_CONNECTOR_NAME=HAN Mobile App
|
||||||
BITRIX_OPEN_LINE_ID=<N>
|
BITRIX_OPEN_LINE_ID=<N>
|
||||||
BITRIX_EXPECTED_DOMAIN=<ваш-портал>.bitrix24.ru
|
BITRIX_EXPECTED_DOMAIN=<ваш-портал>.bitrix24.ru
|
||||||
BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix
|
BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix
|
||||||
|
|
||||||
# ENCRYPTION_KEY обязательно через base64
|
|
||||||
# Generate with: python -c "import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())"
|
|
||||||
BITRIX_TOKEN_ENCRYPTION_KEY=change-me
|
|
||||||
BITRIX_HTTP_TIMEOUT_SEC=10
|
BITRIX_HTTP_TIMEOUT_SEC=10
|
||||||
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC=20
|
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC=20
|
||||||
|
|
||||||
@@ -168,10 +113,6 @@ SELECTEL_S3_ENDPOINT_URL=https://s3.ru-7.storage.selcloud.ru
|
|||||||
SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents
|
SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents
|
||||||
SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments
|
SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments
|
||||||
SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine
|
SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine
|
||||||
SELECTEL_S3_ACCESS_KEY=change-me
|
|
||||||
SELECTEL_S3_SECRET_KEY=change-me
|
|
||||||
SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY=change-me
|
|
||||||
SELECTEL_S3_QUARANTINE_READ_SECRET_KEY=change-me
|
|
||||||
|
|
||||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
|
OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
|
||||||
OTEL_SERVICE_NAME_API=api-backend
|
OTEL_SERVICE_NAME_API=api-backend
|
||||||
@@ -181,7 +122,6 @@ SMS_METRICS_PORT=9464
|
|||||||
#Если есть внешний OTLP-сервис, замените (Точный формат авторизации зависит от провайдера):
|
#Если есть внешний OTLP-сервис, замените (Точный формат авторизации зависит от провайдера):
|
||||||
#Если внешнего OTLP-сервиса пока нет, otlp.example.invalid:4317 можно временно оставить, но Collector будет постоянно пытаться подключиться, писать предупреждения и накапливать очередь.
|
#Если внешнего OTLP-сервиса пока нет, otlp.example.invalid:4317 можно временно оставить, но Collector будет постоянно пытаться подключиться, писать предупреждения и накапливать очередь.
|
||||||
OTEL_REMOTE_ENDPOINT=otlp.example.invalid:4317
|
OTEL_REMOTE_ENDPOINT=otlp.example.invalid:4317
|
||||||
OTEL_REMOTE_AUTH_HEADER=change-me
|
|
||||||
# true только для plaintext OTLP внутри доверенной приватной сети (например self-hosted SigNoz).
|
# true только для plaintext OTLP внутри доверенной приватной сети (например self-hosted SigNoz).
|
||||||
OTEL_REMOTE_TLS_INSECURE=false
|
OTEL_REMOTE_TLS_INSECURE=false
|
||||||
# SDK отправляет все spans локальному Collector; решение о хранении принимает tail_sampling.
|
# SDK отправляет все spans локальному Collector; решение о хранении принимает tail_sampling.
|
||||||
|
|||||||
@@ -13,7 +13,8 @@ COPY --from=builder /build/dist/*.whl /tmp/
|
|||||||
RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl
|
RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl
|
||||||
COPY alembic.ini ./
|
COPY alembic.ini ./
|
||||||
COPY alembic ./alembic
|
COPY alembic ./alembic
|
||||||
|
COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint
|
||||||
USER 10001:10001
|
USER 10001:10001
|
||||||
EXPOSE 8000
|
EXPOSE 8000
|
||||||
ENTRYPOINT ["uvicorn"]
|
ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"]
|
||||||
CMD ["app.main:app", "--host", "0.0.0.0", "--port", "8000", "--no-proxy-headers"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--no-proxy-headers"]
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import asyncio
|
import asyncio
|
||||||
|
import os
|
||||||
from logging.config import fileConfig
|
from logging.config import fileConfig
|
||||||
|
|
||||||
from sqlalchemy import pool
|
from sqlalchemy import pool
|
||||||
@@ -6,12 +7,11 @@ from sqlalchemy import pool
|
|||||||
from alembic import context
|
from alembic import context
|
||||||
from app.db import Base
|
from app.db import Base
|
||||||
from app.postgres import create_postgres_engine
|
from app.postgres import create_postgres_engine
|
||||||
from app.settings import get_settings
|
|
||||||
|
|
||||||
config = context.config
|
config = context.config
|
||||||
if config.config_file_name:
|
if config.config_file_name:
|
||||||
fileConfig(config.config_file_name)
|
fileConfig(config.config_file_name)
|
||||||
database_url = get_settings().database_url
|
database_url = os.environ["DATABASE_URL"]
|
||||||
config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
|
config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
|
||||||
target_metadata = Base.metadata
|
target_metadata = Base.metadata
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
REDACTED = "[REDACTED]"
|
||||||
|
_SENSITIVE_KEY = re.compile(
|
||||||
|
r"(authorization|cookie|password|passwd|secret|token|api[_-]?key|"
|
||||||
|
r"database[_-]?url|redis[_-]?url|dsn|callback[_-]?url)",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_URI_USERINFO = re.compile(r"(?P<scheme>[a-z][a-z0-9+.-]*://)[^/@\s]+@", re.IGNORECASE)
|
||||||
|
_QUERY_SECRET = re.compile(
|
||||||
|
r"(?P<prefix>[?&](?:token|access_token|api_key|key|secret|password)=)[^&#\s]+",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_AUTH_VALUE = re.compile(r"\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_text(value: str) -> str:
|
||||||
|
value = _URI_USERINFO.sub(r"\g<scheme>[REDACTED]@", value)
|
||||||
|
value = _QUERY_SECRET.sub(r"\g<prefix>[REDACTED]", value)
|
||||||
|
return _AUTH_VALUE.sub(r"\1 [REDACTED]", value)
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_value(value: Any) -> Any:
|
||||||
|
if isinstance(value, str):
|
||||||
|
return sanitize_text(value)
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {
|
||||||
|
str(key): REDACTED if _SENSITIVE_KEY.search(str(key)) else sanitize_value(item)
|
||||||
|
for key, item in value.items()
|
||||||
|
}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [sanitize_value(item) for item in value]
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
return tuple(sanitize_value(item) for item in value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def redact_event(
|
||||||
|
_logger: Any,
|
||||||
|
_method_name: str,
|
||||||
|
event_dict: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
return sanitize_value(event_dict)
|
||||||
@@ -50,6 +50,7 @@ from app.integrations import (
|
|||||||
S3Client,
|
S3Client,
|
||||||
SafetyClient,
|
SafetyClient,
|
||||||
)
|
)
|
||||||
|
from app.logging_security import redact_event
|
||||||
from app.metrics import AUTH_BOOTSTRAP, HTTP_DURATION, HTTP_REQUESTS, RATE_LIMIT_DECISIONS
|
from app.metrics import AUTH_BOOTSTRAP, HTTP_DURATION, HTTP_REQUESTS, RATE_LIMIT_DECISIONS
|
||||||
from app.notification_routes import router as notification_router
|
from app.notification_routes import router as notification_router
|
||||||
from app.notification_service import synchronize_source_tokens
|
from app.notification_service import synchronize_source_tokens
|
||||||
@@ -96,6 +97,7 @@ def configure_logging(level: str) -> None:
|
|||||||
processors=[
|
processors=[
|
||||||
structlog.contextvars.merge_contextvars,
|
structlog.contextvars.merge_contextvars,
|
||||||
add_trace_context,
|
add_trace_context,
|
||||||
|
redact_event,
|
||||||
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
||||||
structlog.stdlib.add_log_level,
|
structlog.stdlib.add_log_level,
|
||||||
structlog.processors.JSONRenderer(),
|
structlog.processors.JSONRenderer(),
|
||||||
@@ -342,7 +344,11 @@ async def http_error(request: Request, exc: StarletteHTTPException):
|
|||||||
|
|
||||||
@app.exception_handler(Exception)
|
@app.exception_handler(Exception)
|
||||||
async def unhandled_error(request: Request, exc: Exception):
|
async def unhandled_error(request: Request, exc: Exception):
|
||||||
log.exception("request.failed", error_code="internal_error")
|
log.error(
|
||||||
|
"request.failed",
|
||||||
|
error_code="internal_error",
|
||||||
|
error_type=type(exc).__name__,
|
||||||
|
)
|
||||||
return error_response(request, "internal_error", "Internal server error", 500)
|
return error_response(request, "internal_error", "Internal server error", 500)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec "$@"
|
||||||
@@ -5,39 +5,54 @@ services:
|
|||||||
expose:
|
expose:
|
||||||
- "8000"
|
- "8000"
|
||||||
environment:
|
environment:
|
||||||
|
HAN_SECRET_VARS: >-
|
||||||
|
DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY
|
||||||
|
CURSOR_HMAC_SECRET
|
||||||
|
DATABASE_URL_FILE: /run/secrets/api_database_url
|
||||||
|
REDIS_URL_FILE: /run/secrets/api_redis_url
|
||||||
|
REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url
|
||||||
|
MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token
|
||||||
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token
|
||||||
|
BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token
|
||||||
|
SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key
|
||||||
|
SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key
|
||||||
|
CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret
|
||||||
APP_ENV: ${APP_ENV}
|
APP_ENV: ${APP_ENV}
|
||||||
API_PORT: ${API_PORT:-8000}
|
API_PORT: ${API_PORT:-8000}
|
||||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
DATABASE_URL: ${DATABASE_URL}
|
|
||||||
REDIS_URL: ${REDIS_URL}
|
|
||||||
REDIS_REALTIME_URL: ${REDIS_REALTIME_URL}
|
|
||||||
KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL}
|
KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL}
|
||||||
KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL}
|
KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL}
|
||||||
KEYCLOAK_REALM: ${KEYCLOAK_REALM}
|
KEYCLOAK_REALM: ${KEYCLOAK_REALM}
|
||||||
KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE}
|
KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE}
|
||||||
MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL}
|
MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL}
|
||||||
MESSAGE_SAFETY_SERVICE_TOKEN: ${MESSAGE_SAFETY_SERVICE_TOKEN}
|
|
||||||
MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5}
|
MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5}
|
||||||
MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2}
|
MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2}
|
||||||
MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300}
|
MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300}
|
||||||
MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD: ${MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD:-5}
|
MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD: ${MESSAGE_SAFETY_CIRCUIT_FAILURE_THRESHOLD:-5}
|
||||||
MESSAGE_SAFETY_CIRCUIT_OPEN_SEC: ${MESSAGE_SAFETY_CIRCUIT_OPEN_SEC:-30}
|
MESSAGE_SAFETY_CIRCUIT_OPEN_SEC: ${MESSAGE_SAFETY_CIRCUIT_OPEN_SEC:-30}
|
||||||
BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL}
|
BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL}
|
||||||
BITRIX_LOCAL_APP_INTERNAL_TOKEN: ${BITRIX_LOCAL_APP_INTERNAL_TOKEN}
|
|
||||||
BITRIX_API_INBOX_TOKEN: ${BITRIX_API_INBOX_TOKEN}
|
|
||||||
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20}
|
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20}
|
||||||
BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD: ${BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD:-5}
|
BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD: ${BITRIX_LOCAL_APP_CIRCUIT_FAILURE_THRESHOLD:-5}
|
||||||
BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC: ${BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC:-30}
|
BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC: ${BITRIX_LOCAL_APP_CIRCUIT_OPEN_SEC:-30}
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN}
|
|
||||||
SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL}
|
SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL}
|
||||||
SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS}
|
SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS}
|
||||||
SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS}
|
SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS}
|
||||||
SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE}
|
SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE}
|
||||||
SELECTEL_S3_ACCESS_KEY: ${SELECTEL_S3_ACCESS_KEY}
|
|
||||||
SELECTEL_S3_SECRET_KEY: ${SELECTEL_S3_SECRET_KEY}
|
|
||||||
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT}
|
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT}
|
||||||
CURSOR_HMAC_SECRET: ${CURSOR_HMAC_SECRET}
|
secrets:
|
||||||
NOTIFICATIONS_TOKEN_PRODUCER_TEST: ${NOTIFICATIONS_TOKEN_PRODUCER_TEST}
|
- api_database_url
|
||||||
|
- api_redis_url
|
||||||
|
- api_redis_realtime_url
|
||||||
|
- message_safety_service_token
|
||||||
|
- bitrix_local_app_internal_token
|
||||||
|
- bitrix_api_inbox_token
|
||||||
|
- keycloak_settings_bridge_token
|
||||||
|
- selectel_s3_access_key
|
||||||
|
- selectel_s3_secret_key
|
||||||
|
- cursor_hmac_secret
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test:
|
test:
|
||||||
- CMD
|
- CMD
|
||||||
@@ -48,6 +63,8 @@ services:
|
|||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 5
|
retries: 5
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
networks:
|
networks:
|
||||||
- backend
|
- backend
|
||||||
- observability
|
- observability
|
||||||
@@ -55,3 +72,25 @@ services:
|
|||||||
networks:
|
networks:
|
||||||
backend:
|
backend:
|
||||||
observability:
|
observability:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
api_database_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL
|
||||||
|
api_redis_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_URL
|
||||||
|
api_redis_realtime_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_REALTIME_URL
|
||||||
|
message_safety_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
|
bitrix_local_app_internal_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_LOCAL_APP_INTERNAL_TOKEN
|
||||||
|
bitrix_api_inbox_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_INBOX_TOKEN
|
||||||
|
keycloak_settings_bridge_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN
|
||||||
|
selectel_s3_access_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_ACCESS_KEY
|
||||||
|
selectel_s3_secret_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_SECRET_KEY
|
||||||
|
cursor_hmac_secret:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/CURSOR_HMAC_SECRET
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from app.logging_security import REDACTED, redact_event, sanitize_text
|
||||||
|
|
||||||
|
|
||||||
|
def test_redacts_sensitive_fields_recursively() -> None:
|
||||||
|
event = {
|
||||||
|
"authorization": "Bearer top-secret",
|
||||||
|
"nested": {
|
||||||
|
"database_url": "postgresql://user:password@db/app",
|
||||||
|
"safe": "kept",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
redacted = redact_event(None, "info", event)
|
||||||
|
|
||||||
|
assert redacted["authorization"] == REDACTED
|
||||||
|
assert redacted["nested"]["database_url"] == REDACTED
|
||||||
|
assert redacted["nested"]["safe"] == "kept"
|
||||||
|
|
||||||
|
|
||||||
|
def test_redacts_credentials_embedded_in_text() -> None:
|
||||||
|
value = (
|
||||||
|
"POST https://callback-user:callback-password@example.test/cb"
|
||||||
|
"?token=query-secret Authorization=Bearer header-secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
redacted = sanitize_text(value)
|
||||||
|
|
||||||
|
assert "callback-password" not in redacted
|
||||||
|
assert "query-secret" not in redacted
|
||||||
|
assert "header-secret" not in redacted
|
||||||
|
assert redacted.count(REDACTED) == 3
|
||||||
@@ -6,7 +6,9 @@ COPY app ./app
|
|||||||
COPY alembic ./alembic
|
COPY alembic ./alembic
|
||||||
COPY alembic.ini pyproject.toml ./
|
COPY alembic.ini pyproject.toml ./
|
||||||
RUN pip install --no-cache-dir .
|
RUN pip install --no-cache-dir .
|
||||||
|
COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint
|
||||||
USER app
|
USER app
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
||||||
|
ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"]
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
||||||
|
|||||||
@@ -766,7 +766,7 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
|||||||
timeout=cfg.bitrix_http_timeout_sec,
|
timeout=cfg.bitrix_http_timeout_sec,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.exception(
|
logger.error(
|
||||||
"outbound delivery failed",
|
"outbound delivery failed",
|
||||||
extra={
|
extra={
|
||||||
"request_id": request.state.request_id,
|
"request_id": request.state.request_id,
|
||||||
@@ -1153,8 +1153,11 @@ async def worker_loop(app: FastAPI, kind: str) -> None:
|
|||||||
await process_outbound(app)
|
await process_outbound(app)
|
||||||
else:
|
else:
|
||||||
await process_setup(app)
|
await process_setup(app)
|
||||||
except Exception:
|
except Exception as exc:
|
||||||
logger.exception("worker iteration failed", extra={"worker_kind": kind})
|
logger.error(
|
||||||
|
"worker iteration failed",
|
||||||
|
extra={"worker_kind": kind, "error_type": type(exc).__name__},
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
await asyncio.wait_for(app.state.stop.wait(), app.state.settings.bitrix_worker_poll_sec)
|
await asyncio.wait_for(app.state.stop.wait(), app.state.settings.bitrix_worker_poll_sec)
|
||||||
except TimeoutError:
|
except TimeoutError:
|
||||||
@@ -1197,7 +1200,7 @@ async def process_inbox(app: FastAPI) -> None:
|
|||||||
)
|
)
|
||||||
await session.commit()
|
await session.commit()
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.exception(
|
logger.error(
|
||||||
"inbound forward failed",
|
"inbound forward failed",
|
||||||
extra={
|
extra={
|
||||||
"inbox_id": str(row.id),
|
"inbox_id": str(row.id),
|
||||||
@@ -1275,7 +1278,7 @@ async def process_outbound(app: FastAPI) -> None:
|
|||||||
timeout=app.state.settings.bitrix_http_timeout_sec,
|
timeout=app.state.settings.bitrix_http_timeout_sec,
|
||||||
)
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.exception(
|
logger.error(
|
||||||
"outbound retry failed",
|
"outbound retry failed",
|
||||||
extra={
|
extra={
|
||||||
"outbound_id": str(row.id),
|
"outbound_id": str(row.id),
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec "$@"
|
||||||
@@ -6,7 +6,9 @@ COPY app ./app
|
|||||||
COPY alembic ./alembic
|
COPY alembic ./alembic
|
||||||
COPY alembic.ini pyproject.toml ./
|
COPY alembic.ini pyproject.toml ./
|
||||||
RUN pip install --no-cache-dir .
|
RUN pip install --no-cache-dir .
|
||||||
|
COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint
|
||||||
USER app
|
USER app
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
||||||
|
ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"]
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec "$@"
|
||||||
@@ -262,7 +262,7 @@ han-chat-documents
|
|||||||
|
|
||||||
## 8. Создание файла окружения
|
## 8. Создание файла окружения
|
||||||
|
|
||||||
На VM:
|
`.env` содержит только несекретную конфигурацию. На VM:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cd /opt/han-chat/backend
|
cd /opt/han-chat/backend
|
||||||
@@ -272,7 +272,18 @@ chmod 600 .env
|
|||||||
nano .env
|
nano .env
|
||||||
```
|
```
|
||||||
|
|
||||||
Замените все `change-me` и адреса `example.*`.
|
Замените несекретные адреса `example.*`. Не добавляйте в `.env` пароли, токены,
|
||||||
|
ключи, credential-bearing DSN или пути `*_FILE`. Установите отдельно проверенный
|
||||||
|
launcher `deployment/secrets/han-secrets` из ops-пакета. Его интерфейс:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
deployment/secrets/han-secrets run --config .env -- <command>
|
||||||
|
```
|
||||||
|
|
||||||
|
Launcher читает `SECRETS_SOURCE=file|selectel`, устанавливает
|
||||||
|
`HAN_SECRETS_ACTIVE=1`, выдаёт значения только дочернему процессу и не печатает
|
||||||
|
их. Рекомендуемый `HAN_RUNTIME_SECRET_MANIFEST` содержит только пары
|
||||||
|
`SECRET_KEY=/absolute/protected/path`; файлы имеют mode `0400`/`0600`.
|
||||||
|
|
||||||
### 8.1. Основные адреса
|
### 8.1. Основные адреса
|
||||||
|
|
||||||
@@ -295,7 +306,7 @@ KEYCLOAK_AUDIENCE=han-chat-api
|
|||||||
|
|
||||||
### 8.2. PostgreSQL
|
### 8.2. PostgreSQL
|
||||||
|
|
||||||
Укажите фактические host, port, database, пользователей и пароли:
|
В `.env` укажите только host, port, database и путь к публичному CA:
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
HAN_PG_HOST=<PG_HOST>
|
HAN_PG_HOST=<PG_HOST>
|
||||||
@@ -303,17 +314,13 @@ HAN_PG_PORT=6432
|
|||||||
HAN_PG_DATABASE=han_chat
|
HAN_PG_DATABASE=han_chat
|
||||||
PG_CA_HOST_PATH=/opt/han-chat/backend/secrets/pg/ca.pem
|
PG_CA_HOST_PATH=/opt/han-chat/backend/secrets/pg/ca.pem
|
||||||
|
|
||||||
DATABASE_URL=postgresql+asyncpg://han_app:<PASSWORD>@<PG_HOST>:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
BITRIX_DATABASE_URL=postgresql://bitrix_local_app:<PASSWORD>@<PG_HOST>:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
BITRIX_SYNC_DATABASE_URL=postgresql://bitrix_sync_user:<PASSWORD>@<PG_HOST>:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
MESSAGE_SAFETY_DATABASE_URL=postgresql://message_safety_app:<PASSWORD>@<PG_HOST>:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
|
|
||||||
KEYCLOAK_DB_URL=jdbc:postgresql://<PG_HOST>:5433/han_chat?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem
|
|
||||||
KEYCLOAK_DB_SCHEMA=keycloak
|
KEYCLOAK_DB_SCHEMA=keycloak
|
||||||
KEYCLOAK_DB_USERNAME=keycloak_user
|
|
||||||
KEYCLOAK_DB_PASSWORD=<PASSWORD>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Все service DSN, включая JDBC и backup DSN, формирует secret backend. Runtime
|
||||||
|
validator проверяет `verify-full`, `sslrootcert` и запрет `options/currentSchema`
|
||||||
|
без вывода строк подключения.
|
||||||
|
|
||||||
Порт `5433` используется с PgBouncer в режиме `session`. Не добавляйте
|
Порт `5433` используется с PgBouncer в режиме `session`. Не добавляйте
|
||||||
`options=-csearch_path...` или JDBC-параметр `currentSchema`: они передают
|
`options=-csearch_path...` или JDBC-параметр `currentSchema`: они передают
|
||||||
startup parameter `search_path`, который Selectel PgBouncer отклоняет. Для
|
startup parameter `search_path`, который Selectel PgBouncer отклоняет. Для
|
||||||
@@ -323,9 +330,12 @@ Keycloak схема задаётся отдельно через `KEYCLOAK_DB_SC
|
|||||||
Если пароль содержит `@`, `:`, `/`, `?`, `#` или `%`, его необходимо
|
Если пароль содержит `@`, `:`, `/`, `?`, `#` или `%`, его необходимо
|
||||||
URL-кодировать внутри PostgreSQL URL.
|
URL-кодировать внутри PostgreSQL URL.
|
||||||
|
|
||||||
### 8.3. Генерация секретов
|
### 8.3. Подготовка runtime-секретов
|
||||||
|
|
||||||
Для обычных токенов используйте:
|
Генерируйте секреты вне shell history средствами secret manager. Не выполняйте
|
||||||
|
`export TOKEN=...` и не вставляйте значения в команды. Имена обязательных
|
||||||
|
runtime-переменных определены в `scripts/validate-env`; парные токены связываются
|
||||||
|
в secret backend.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
openssl rand -hex 32
|
openssl rand -hex 32
|
||||||
@@ -337,7 +347,8 @@ openssl rand -hex 32
|
|||||||
python3 -c 'import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())'
|
python3 -c 'import base64,secrets; print(base64.urlsafe_b64encode(secrets.token_bytes(32)).decode())'
|
||||||
```
|
```
|
||||||
|
|
||||||
Заполните все секреты. Две пары должны совпадать:
|
Ни один из этих секретов не добавляется в `.env`. Пары проверяются runtime
|
||||||
|
validator без вывода значений.
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
BITRIX_LOCAL_APP_INTERNAL_TOKEN=<TOKEN_A>
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN=<TOKEN_A>
|
||||||
@@ -361,7 +372,8 @@ KEYCLOAK_ADMIN_PASSWORD=<UNIQUE_ADMIN_PASSWORD>
|
|||||||
|
|
||||||
### 8.4. Redis
|
### 8.4. Redis
|
||||||
|
|
||||||
Создайте три разных пароля и продублируйте их в URL:
|
Создайте три разных пароля в secret backend; там же сформируйте Redis URL.
|
||||||
|
Следующий блок описывает логический контракт и не является содержимым `.env`:
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
REDIS_API_PASSWORD=<REDIS_API_PASSWORD>
|
REDIS_API_PASSWORD=<REDIS_API_PASSWORD>
|
||||||
@@ -380,11 +392,10 @@ MESSAGE_SAFETY_REDIS_URL=redis://message_safety:<REDIS_SAFETY_PASSWORD>@redis:63
|
|||||||
```dotenv
|
```dotenv
|
||||||
KEYCLOAK_OTP_MOCK_ENABLED=true
|
KEYCLOAK_OTP_MOCK_ENABLED=true
|
||||||
KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true
|
KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true
|
||||||
KEYCLOAK_OTP_MOCK_CODE=<ТЕСТОВЫЙ_КОД_НЕ_КОРОЧЕ_16_СИМВОЛОВ>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Этот код будет вводиться пользователем при тестовой авторизации. Не используйте
|
`KEYCLOAK_OTP_MOCK_CODE` хранится только в secret backend. Не используйте mock
|
||||||
его как production-механизм доставки OTP.
|
как production-механизм доставки OTP.
|
||||||
|
|
||||||
Целевой real mode задаёт `modules/module-11-idgtl-sms.md`: Keycloak генерирует и локально проверяет OTP, `sms-service` надёжно записывает заказ/журнал, worker вызывает i-Digital Direct, callback обновляет только delivery journal. Нельзя просто установить `KEYCLOAK_OTP_MOCK_ENABLED=false`.
|
Целевой real mode задаёт `modules/module-11-idgtl-sms.md`: Keycloak генерирует и локально проверяет OTP, `sms-service` надёжно записывает заказ/журнал, worker вызывает i-Digital Direct, callback обновляет только delivery journal. Нельзя просто установить `KEYCLOAK_OTP_MOCK_ENABLED=false`.
|
||||||
|
|
||||||
@@ -397,13 +408,10 @@ SELECTEL_S3_ENDPOINT_URL=https://s3.storage.selcloud.ru
|
|||||||
SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine
|
SELECTEL_S3_BUCKET_QUARANTINE=han-chat-quarantine
|
||||||
SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments
|
SELECTEL_S3_BUCKET_ATTACHMENTS=han-chat-attachments
|
||||||
SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents
|
SELECTEL_S3_BUCKET_DOCUMENTS=han-chat-documents
|
||||||
|
|
||||||
SELECTEL_S3_ACCESS_KEY=<API_ACCESS_KEY>
|
|
||||||
SELECTEL_S3_SECRET_KEY=<API_SECRET_KEY>
|
|
||||||
SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY=<SAFETY_READ_ACCESS_KEY>
|
|
||||||
SELECTEL_S3_QUARANTINE_READ_SECRET_KEY=<SAFETY_READ_SECRET_KEY>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Обе пары S3 credentials хранятся только в secret backend.
|
||||||
|
|
||||||
API backend принудительно использует virtual-hosted addressing:
|
API backend принудительно использует virtual-hosted addressing:
|
||||||
`https://<bucket>.s3.storage.selcloud.ru/<object-key>`. Это обязательно для
|
`https://<bucket>.s3.storage.selcloud.ru/<object-key>`. Это обязательно для
|
||||||
браузерных presigned PUT и CORS в Selectel; path-style URL для этого сценария не
|
браузерных presigned PUT и CORS в Selectel; path-style URL для этого сценария не
|
||||||
@@ -411,20 +419,17 @@ API backend принудительно использует virtual-hosted addre
|
|||||||
|
|
||||||
### 8.7. Bitrix24
|
### 8.7. Bitrix24
|
||||||
|
|
||||||
До установки локального приложения заполните:
|
До установки локального приложения загрузите credentials в secret backend.
|
||||||
|
В `.env` остаются только несекретные connector/public URL параметры:
|
||||||
|
|
||||||
```dotenv
|
```dotenv
|
||||||
BITRIX_CLIENT_ID=<BITRIX_CLIENT_ID>
|
|
||||||
BITRIX_CLIENT_SECRET=<BITRIX_CLIENT_SECRET>
|
|
||||||
BITRIX_CONNECTOR_ID=han_mobile_app
|
BITRIX_CONNECTOR_ID=han_mobile_app
|
||||||
BITRIX_OPEN_LINE_ID=8
|
BITRIX_OPEN_LINE_ID=8
|
||||||
BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix
|
BITRIX_PUBLIC_BASE_URL=https://chat.example.ru/bitrix
|
||||||
BITRIX_TOKEN_ENCRYPTION_KEY=<URLSAFE_BASE64_KEY>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`BITRIX_APPLICATION_TOKEN` можно окончательно задать после создания приложения в
|
`BITRIX_APPLICATION_TOKEN` сохраняется в secret backend после создания
|
||||||
Bitrix24. До этого используйте отдельное случайное значение, проходящее
|
приложения и никогда не помещается в `.env`.
|
||||||
валидацию окружения.
|
|
||||||
|
|
||||||
### 8.8. TLS и наблюдаемость
|
### 8.8. TLS и наблюдаемость
|
||||||
|
|
||||||
@@ -448,8 +453,11 @@ ACME_EMAIL=<ADMIN_EMAIL>
|
|||||||
```sh
|
```sh
|
||||||
cd /opt/han-chat/backend
|
cd /opt/han-chat/backend
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
docker compose --env-file .env config --quiet
|
sudo systemctl restart han-secrets@production.service
|
||||||
docker compose --env-file .env config --services
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
|
sudo deployment/secrets/han-compose config --services
|
||||||
python3 -m unittest discover -s tests -v
|
python3 -m unittest discover -s tests -v
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -670,7 +678,8 @@ curl -i https://chat.example.ru/internal/safety/v1/messages/check
|
|||||||
https://chat.example.ru/
|
https://chat.example.ru/
|
||||||
```
|
```
|
||||||
|
|
||||||
Для тестовой авторизации используйте значение `KEYCLOAK_OTP_MOCK_CODE` из `.env`.
|
Для тестовой авторизации получите mock code утверждённым защищённым способом,
|
||||||
|
не читая его из `.env` и не помещая в shell history.
|
||||||
|
|
||||||
## 16. Подключение Bitrix24
|
## 16. Подключение Bitrix24
|
||||||
|
|
||||||
@@ -685,7 +694,7 @@ Placement: https://chat.example.ru/bitrix/placement
|
|||||||
После установки:
|
После установки:
|
||||||
|
|
||||||
1. Получите и сохраните application token.
|
1. Получите и сохраните application token.
|
||||||
2. Запишите его в `BITRIX_APPLICATION_TOKEN` файла `.env`.
|
2. Сохраните его как `BITRIX_APPLICATION_TOKEN` в secret backend.
|
||||||
3. Пересоздайте сервис:
|
3. Пересоздайте сервис:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
@@ -735,7 +744,7 @@ docker compose --env-file .env up -d --force-recreate <SERVICE_NAME>
|
|||||||
1. Создайте backup/PITR marker PostgreSQL.
|
1. Создайте backup/PITR marker PostgreSQL.
|
||||||
2. Сохраните текущие image digests.
|
2. Сохраните текущие image digests.
|
||||||
3. Получите новый код.
|
3. Получите новый код.
|
||||||
4. Проверьте `.env`.
|
4. Проверьте несекретный `.env` и runtime secret set.
|
||||||
5. Пересоберите образы.
|
5. Пересоберите образы.
|
||||||
6. Примените миграции и seed.
|
6. Примените миграции и seed.
|
||||||
7. Пересоздайте сервисы.
|
7. Пересоздайте сервисы.
|
||||||
|
|||||||
@@ -60,16 +60,20 @@ verify key-based deploy access in a separate SSH session.
|
|||||||
## Gate 6 — environment and secrets
|
## Gate 6 — environment and secrets
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
umask 077
|
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
chmod 600 .env
|
# Replace non-secret configuration placeholders only.
|
||||||
# Replace placeholders using a protected editor/secret manager.
|
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
docker compose --env-file .env config --quiet
|
sudo systemctl restart han-secrets@production.service
|
||||||
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] Token pairs match, PG verifies TLS, public URLs are HTTPS.
|
- [ ] `SECRETS_SOURCE=file|selectel`; `.env` contains no secret keys or credential-bearing DSNs.
|
||||||
- [ ] Mock OTP risk is accepted and all secrets are unique >=128-bit values.
|
- [ ] `deployment/secrets/han-secrets` sets `HAN_SECRETS_ACTIVE=1`, does not log values,
|
||||||
|
and optionally exposes a paths-only `HAN_RUNTIME_SECRET_MANIFEST`.
|
||||||
|
- [ ] Runtime token pairs match, PG verifies TLS, public URLs are HTTPS.
|
||||||
|
- [ ] Mock OTP risk is accepted and runtime secrets are unique >=128-bit values.
|
||||||
- [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` is unique and supplied only through secret/env; the `producer_test` source seed stores only its hash.
|
- [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` is unique and supplied only through secret/env; the `producer_test` source seed stores only its hash.
|
||||||
- [ ] `FRONTEND_DEV_PROXY_ENABLED=false` and Safety/nginx timeout budgets match.
|
- [ ] `FRONTEND_DEV_PROXY_ENABLED=false` and Safety/nginx timeout budgets match.
|
||||||
|
|
||||||
@@ -211,7 +215,6 @@ For local acceptance only, start the redacted debug collector with:
|
|||||||
Provider backup/PITR is authoritative. A supplemental verified logical dump:
|
Provider backup/PITR is authoritative. A supplemental verified logical dump:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
PG_BACKUP_DSN='postgresql://...?...sslmode=verify-full&sslrootcert=...' \
|
|
||||||
deployment/scripts/backup.sh /opt/han-chat/backups
|
deployment/scripts/backup.sh /opt/han-chat/backups
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -225,10 +228,13 @@ Only roll back to images compatible with the current schema:
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \
|
SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \
|
||||||
deployment/scripts/rollback.sh /secure/path/previous-release.env
|
deployment/scripts/rollback.sh <PREVIOUS_IMMUTABLE_RELEASE>
|
||||||
ENV_FILE=/secure/path/previous-release.env deployment/scripts/smoke.sh
|
deployment/scripts/smoke.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Rollback reuses the current runtime secret set and non-secret config. Do not
|
||||||
|
create or restore an environment snapshot.
|
||||||
|
|
||||||
## Real SMS rollout addendum
|
## Real SMS rollout addendum
|
||||||
|
|
||||||
This runbook remains mock-only until module-11 artifacts exist. An SMS release requires schema/role `sms`, versioned migrations and an active approved `auth_otp` seed, `sms-service`/worker, the exact callback route, paired service tokens, Direct `TOKEN_1`, approved sender/template, separate callback credentials, a reconfirmed callback source IP, and a static worker egress IP.
|
This runbook remains mock-only until module-11 artifacts exist. An SMS release requires schema/role `sms`, versioned migrations and an active approved `auth_otp` seed, `sms-service`/worker, the exact callback route, paired service tokens, Direct `TOKEN_1`, approved sender/template, separate callback credentials, a reconfirmed callback source IP, and a static worker egress IP.
|
||||||
|
|||||||
@@ -63,16 +63,21 @@ sudo deployment/scripts/setup-vm.sh
|
|||||||
## Этап 6 — окружение и секреты
|
## Этап 6 — окружение и секреты
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
umask 077
|
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
chmod 600 .env
|
# Замените только несекретные placeholders.
|
||||||
# Замените заглушки через защищенный редактор или менеджер секретов.
|
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
docker compose --env-file .env config --quiet
|
sudo systemctl restart han-secrets@production.service
|
||||||
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] Парные токены совпадают, PostgreSQL проверяет TLS, публичные URL используют HTTPS.
|
- [ ] `SECRETS_SOURCE=file|selectel`; `.env` не содержит secret keys и DSN с credentials.
|
||||||
- [ ] Риск mock OTP принят; все секреты уникальны и содержат не менее 128 бит энтропии.
|
- [ ] `deployment/secrets/han-secrets` устанавливает `HAN_SECRETS_ACTIVE=1`,
|
||||||
|
не пишет значения в лог и при возможности передаёт paths-only manifest
|
||||||
|
через `HAN_RUNTIME_SECRET_MANIFEST`.
|
||||||
|
- [ ] Runtime-пары токенов совпадают, PostgreSQL проверяет TLS.
|
||||||
|
- [ ] Риск mock OTP принят; runtime-секреты уникальны и содержат не менее 128 бит энтропии.
|
||||||
- [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` сгенерирован отдельно, передан только через secret/env; seed `notification_sources.code='producer_test'` содержит только его hash.
|
- [ ] `NOTIFICATIONS_TOKEN_PRODUCER_TEST` сгенерирован отдельно, передан только через secret/env; seed `notification_sources.code='producer_test'` содержит только его hash.
|
||||||
- [ ] Установлено `FRONTEND_DEV_PROXY_ENABLED=false`; таймауты Safety и nginx согласованы.
|
- [ ] Установлено `FRONTEND_DEV_PROXY_ENABLED=false`; таймауты Safety и nginx согласованы.
|
||||||
|
|
||||||
@@ -233,7 +238,6 @@ deployment/scripts/smoke.sh
|
|||||||
Основным механизмом являются backup/PITR провайдера. Дополнительный проверенный логический дамп:
|
Основным механизмом являются backup/PITR провайдера. Дополнительный проверенный логический дамп:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
PG_BACKUP_DSN='postgresql://...?...sslmode=verify-full&sslrootcert=...' \
|
|
||||||
deployment/scripts/backup.sh /opt/han-chat/backups
|
deployment/scripts/backup.sh /opt/han-chat/backups
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -248,10 +252,13 @@ smoke-тесты и фиксируйте фактические RPO/RTO. Redis
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \
|
SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \
|
||||||
deployment/scripts/rollback.sh /secure/path/previous-release.env
|
deployment/scripts/rollback.sh <PREVIOUS_IMMUTABLE_RELEASE>
|
||||||
ENV_FILE=/secure/path/previous-release.env deployment/scripts/smoke.sh
|
deployment/scripts/smoke.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Откат использует текущие runtime-секреты и текущий несекретный config. Snapshot
|
||||||
|
старого `.env` не создаётся и не восстанавливается.
|
||||||
|
|
||||||
## Дополнение: rollout реальной SMS-авторизации
|
## Дополнение: rollout реальной SMS-авторизации
|
||||||
|
|
||||||
Текущий runbook остаётся mock-only, пока артефакты module-11 не реализованы. Для SMS release обязательны: schema/role `sms`, migrations/seed active approved `auth_otp`, `sms-service`/worker, exact callback route, парные service tokens, Direct `TOKEN_1`, согласованные sender/template, отдельные callback credentials, подтверждённый callback source IP и статический egress IP worker.
|
Текущий runbook остаётся mock-only, пока артефакты module-11 не реализованы. Для SMS release обязательны: schema/role `sms`, migrations/seed active approved `auth_otp`, `sms-service`/worker, exact callback route, парные service tokens, Direct `TOKEN_1`, согласованные sender/template, отдельные callback credentials, подтверждённый callback source IP и статический egress IP worker.
|
||||||
|
|||||||
@@ -1,83 +1,123 @@
|
|||||||
x-no-sms-secrets: &no-sms-secrets
|
x-api-job-secrets: &api-job-secrets
|
||||||
SMS_DATABASE_URL: ""
|
- api_database_url
|
||||||
SMS_SERVICE_TOKEN: ""
|
- api_redis_url
|
||||||
KEYCLOAK_SMS_SERVICE_TOKEN: ""
|
- api_redis_realtime_url
|
||||||
IDGTL_SMS_API_KEY: ""
|
- message_safety_service_token
|
||||||
IDGTL_SMS_CALLBACK_USERNAME: ""
|
- bitrix_local_app_internal_token
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD: ""
|
- bitrix_api_inbox_token
|
||||||
|
- keycloak_settings_bridge_token
|
||||||
|
- selectel_s3_access_key
|
||||||
|
- selectel_s3_secret_key
|
||||||
|
- cursor_hmac_secret
|
||||||
|
|
||||||
|
x-api-job-environment: &api-job-environment
|
||||||
|
HAN_SECRET_VARS: >-
|
||||||
|
DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY
|
||||||
|
CURSOR_HMAC_SECRET
|
||||||
|
DATABASE_URL_FILE: /run/secrets/api_database_url
|
||||||
|
REDIS_URL_FILE: /run/secrets/api_redis_url
|
||||||
|
REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url
|
||||||
|
MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token
|
||||||
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token
|
||||||
|
BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token
|
||||||
|
SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key
|
||||||
|
SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key
|
||||||
|
CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret
|
||||||
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
|
KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL}
|
||||||
|
KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL:-http://keycloak:8080/auth}
|
||||||
|
KEYCLOAK_REALM: ${KEYCLOAK_REALM:-han-chat}
|
||||||
|
KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE:-han-chat-api}
|
||||||
|
MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL:-http://message-safety:8080}
|
||||||
|
MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5}
|
||||||
|
MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2}
|
||||||
|
MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300}
|
||||||
|
BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL:-http://bitrix-local-app:8080}
|
||||||
|
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20}
|
||||||
|
SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL}
|
||||||
|
SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS}
|
||||||
|
SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS}
|
||||||
|
SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE}
|
||||||
|
TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-127.0.0.1/32}
|
||||||
|
|
||||||
services:
|
services:
|
||||||
migrate-api:
|
migrate-api:
|
||||||
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
||||||
profiles: ["ops"]
|
profiles: ["ops"]
|
||||||
env_file:
|
environment:
|
||||||
- path: ../.env
|
HAN_SECRET_VARS: DATABASE_URL
|
||||||
required: false
|
DATABASE_URL_FILE: /run/secrets/api_database_url
|
||||||
environment: *no-sms-secrets
|
secrets:
|
||||||
entrypoint: []
|
- api_database_url
|
||||||
command: ["alembic", "upgrade", "head"]
|
command: ["alembic", "upgrade", "head"]
|
||||||
volumes:
|
volumes:
|
||||||
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
||||||
networks: [backend, egress]
|
networks: [backend, egress]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
migrate-bitrix-local:
|
migrate-bitrix-local:
|
||||||
image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local}
|
image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local}
|
||||||
profiles: ["ops"]
|
profiles: ["ops"]
|
||||||
env_file:
|
environment:
|
||||||
- path: ../.env
|
HAN_SECRET_VARS: BITRIX_DATABASE_URL
|
||||||
required: false
|
BITRIX_DATABASE_URL_FILE: /run/secrets/bitrix_database_url
|
||||||
environment: *no-sms-secrets
|
secrets:
|
||||||
entrypoint: []
|
- bitrix_database_url
|
||||||
command: ["alembic", "upgrade", "head"]
|
command: ["alembic", "upgrade", "head"]
|
||||||
volumes:
|
volumes:
|
||||||
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
||||||
networks: [backend, egress]
|
networks: [backend, egress]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
migrate-bitrix-sync:
|
migrate-bitrix-sync:
|
||||||
image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local}
|
image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local}
|
||||||
profiles: ["ops"]
|
profiles: ["ops"]
|
||||||
env_file:
|
environment:
|
||||||
- path: ../.env
|
HAN_SECRET_VARS: BITRIX_SYNC_DATABASE_URL
|
||||||
required: false
|
BITRIX_SYNC_DATABASE_URL_FILE: /run/secrets/bitrix_sync_database_url
|
||||||
environment: *no-sms-secrets
|
secrets:
|
||||||
entrypoint: []
|
- bitrix_sync_database_url
|
||||||
command: ["alembic", "upgrade", "head"]
|
command: ["alembic", "upgrade", "head"]
|
||||||
volumes:
|
volumes:
|
||||||
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
||||||
networks: [backend, egress]
|
networks: [backend, egress]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
migrate-sms:
|
migrate-sms:
|
||||||
image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local}
|
image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local}
|
||||||
profiles: ["ops"]
|
profiles: ["ops"]
|
||||||
environment:
|
environment:
|
||||||
SMS_DATABASE_URL: ${SMS_DATABASE_URL}
|
HAN_SECRET_VARS: SMS_DATABASE_URL
|
||||||
SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN}
|
SMS_DATABASE_URL_FILE: /run/secrets/sms_database_url
|
||||||
IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru}
|
secrets:
|
||||||
IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL}
|
- sms_database_url
|
||||||
IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME}
|
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD}
|
|
||||||
entrypoint: []
|
|
||||||
command: ["alembic", "upgrade", "head"]
|
command: ["alembic", "upgrade", "head"]
|
||||||
volumes:
|
volumes:
|
||||||
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
- ${PG_CA_HOST_PATH}:/run/secrets/pg-ca.pem:ro
|
||||||
networks: [backend, egress]
|
networks: [backend, egress]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
seed-settings:
|
seed-settings:
|
||||||
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
||||||
profiles: ["ops"]
|
profiles: ["ops"]
|
||||||
env_file:
|
environment: *api-job-environment
|
||||||
- path: ../.env
|
secrets: *api-job-secrets
|
||||||
required: false
|
|
||||||
environment: *no-sms-secrets
|
|
||||||
entrypoint: []
|
|
||||||
command:
|
command:
|
||||||
- /bin/sh
|
- /bin/sh
|
||||||
- -ec
|
- -ec
|
||||||
@@ -91,6 +131,8 @@ services:
|
|||||||
networks: [backend, egress]
|
networks: [backend, egress]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
toolbox:
|
toolbox:
|
||||||
image: curlimages/curl:8.11.1
|
image: curlimages/curl:8.11.1
|
||||||
|
|||||||
@@ -1,10 +1,22 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
|
||||||
if [ -z "${PG_BACKUP_DSN:-}" ]; then
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
echo "PG_BACKUP_DSN is required (managed PostgreSQL TLS DSN, supplied via secure environment)." >&2
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z "${PG_BACKUP_DSN_FILE:-}" ] || [ ! -r "$PG_BACKUP_DSN_FILE" ]; then
|
||||||
|
echo "PG_BACKUP_DSN_FILE is required from the secret launcher." >&2
|
||||||
exit 64
|
exit 64
|
||||||
fi
|
fi
|
||||||
|
PG_BACKUP_DSN=$(cat "$PG_BACKUP_DSN_FILE")
|
||||||
case "$PG_BACKUP_DSN" in
|
case "$PG_BACKUP_DSN" in
|
||||||
*sslmode=verify-full*sslrootcert=*) ;;
|
*sslmode=verify-full*sslrootcert=*) ;;
|
||||||
*) echo "PG_BACKUP_DSN must enforce sslmode=verify-full and sslrootcert." >&2; exit 64 ;;
|
*) echo "PG_BACKUP_DSN must enforce sslmode=verify-full and sslrootcert." >&2; exit 64 ;;
|
||||||
@@ -16,7 +28,9 @@ mkdir -p "$output_dir"
|
|||||||
stamp=$(date -u +%Y%m%dT%H%M%SZ)
|
stamp=$(date -u +%Y%m%dT%H%M%SZ)
|
||||||
archive="$output_dir/han-chat-$stamp.dump"
|
archive="$output_dir/han-chat-$stamp.dump"
|
||||||
|
|
||||||
pg_dump --dbname="$PG_BACKUP_DSN" --format=custom --no-owner --no-privileges --file="$archive"
|
PGDATABASE="$PG_BACKUP_DSN" pg_dump \
|
||||||
|
--format=custom --no-owner --no-privileges --file="$archive"
|
||||||
|
unset PG_BACKUP_DSN
|
||||||
pg_restore --list "$archive" >/dev/null
|
pg_restore --list "$archive" >/dev/null
|
||||||
sha256sum "$archive" > "$archive.sha256"
|
sha256sum "$archive" > "$archive.sha256"
|
||||||
chmod 600 "$archive" "$archive.sha256"
|
chmod 600 "$archive" "$archive.sha256"
|
||||||
|
|||||||
@@ -1,20 +1,34 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
cd "$(dirname "$0")/../.."
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
|
||||||
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
if [ "${PITR_MARKER_CONFIRMED:-false}" != "true" ]; then
|
if [ "${PITR_MARKER_CONFIRMED:-false}" != "true" ]; then
|
||||||
echo "Refusing migration: create provider PITR marker, then set PITR_MARKER_CONFIRMED=true" >&2
|
echo "Refusing migration: create provider PITR marker, then set PITR_MARKER_CONFIRMED=true" >&2
|
||||||
exit 64
|
exit 64
|
||||||
fi
|
fi
|
||||||
|
|
||||||
./scripts/validate-env "${ENV_FILE:-.env}"
|
if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" config --quiet
|
./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST"
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-api alembic current
|
else
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-local alembic current
|
./scripts/validate-env "$CONFIG_FILE" --runtime-env
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-sync alembic current
|
fi
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-sms alembic current
|
docker compose --env-file "$CONFIG_FILE" config --quiet
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-api
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-api alembic current
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-local
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-local alembic current
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-bitrix-sync
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-sync alembic current
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm migrate-sms
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-sms alembic current
|
||||||
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-api
|
||||||
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-local
|
||||||
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-bitrix-sync
|
||||||
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm migrate-sms
|
||||||
echo "Migrations completed; record revisions in release evidence."
|
echo "Migrations completed; record revisions in release evidence."
|
||||||
|
|||||||
@@ -1,21 +1,34 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
cd "$(dirname "$0")/../.."
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
|
||||||
previous_env=${1:-}
|
previous_release=${1:-}
|
||||||
if [ -z "$previous_env" ] || [ ! -r "$previous_env" ]; then
|
if [ -z "$previous_release" ]; then
|
||||||
echo "Usage: $0 /secure/path/previous-release.env" >&2
|
echo "Usage: $0 <previous-immutable-release>" >&2
|
||||||
exit 64
|
exit 64
|
||||||
fi
|
fi
|
||||||
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
if [ "${SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED:-false}" != "true" ]; then
|
if [ "${SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED:-false}" != "true" ]; then
|
||||||
echo "Refusing rollback: set SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true after migration review." >&2
|
echo "Refusing rollback: set SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true after migration review." >&2
|
||||||
exit 64
|
exit 64
|
||||||
fi
|
fi
|
||||||
|
|
||||||
./scripts/validate-env "$previous_env"
|
if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then
|
||||||
docker compose --env-file "$previous_env" config --quiet
|
./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST"
|
||||||
docker compose --env-file "$previous_env" up -d --remove-orphans
|
else
|
||||||
docker compose --env-file "$previous_env" ps
|
./scripts/validate-env "$CONFIG_FILE" --runtime-env
|
||||||
|
fi
|
||||||
|
RELEASE_VERSION="$previous_release" docker compose --env-file "$CONFIG_FILE" config --quiet
|
||||||
|
RELEASE_VERSION="$previous_release" docker compose --env-file "$CONFIG_FILE" up -d --remove-orphans
|
||||||
|
docker compose --env-file "$CONFIG_FILE" ps
|
||||||
|
|
||||||
echo "Application images rolled back without Alembic downgrade."
|
echo "Application images rolled back without Alembic downgrade."
|
||||||
echo "Run deployment/scripts/smoke.sh with ENV_FILE=$previous_env and verify outbox/inbox idempotency."
|
echo "Run deployment/scripts/smoke.sh and verify outbox/inbox idempotency."
|
||||||
|
|||||||
@@ -1,7 +1,18 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
cd "$(dirname "$0")/../.."
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE=${ENV_FILE:-.env}
|
||||||
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
|
||||||
./scripts/validate-env "${ENV_FILE:-.env}"
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
docker compose --env-file "${ENV_FILE:-.env}" --profile ops run --rm seed-settings
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
|
./scripts/validate-env "$CONFIG_FILE" \
|
||||||
|
--runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST"
|
||||||
|
docker compose --env-file "$CONFIG_FILE" --profile ops run --rm seed-settings
|
||||||
echo "Seed and mandatory-settings validation completed."
|
echo "Seed and mandatory-settings validation completed."
|
||||||
|
|||||||
@@ -435,6 +435,46 @@ configure_application_security() {
|
|||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
|
install_secret_loader_if_possible() {
|
||||||
|
step "Загрузчик секретов"
|
||||||
|
local source_dir="${DEPLOY_DIR}/deployment/secrets"
|
||||||
|
if [[ ! -f "${source_dir}/secrets_loader.py" || ! -f "${source_dir}/han-secrets" ]]; then
|
||||||
|
log "Проект еще не скопирован: загрузчик секретов будет установлен при повторном запуске"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
chmod 0750 "${source_dir}/han-secrets" "${source_dir}/han-compose"
|
||||||
|
|
||||||
|
install -d -m 0700 -o root -g root \
|
||||||
|
/etc/han \
|
||||||
|
/etc/han/secrets \
|
||||||
|
/etc/han/credentials
|
||||||
|
install -d -m 0755 -o root -g root \
|
||||||
|
/usr/local/lib/han-secrets \
|
||||||
|
/usr/local/share/doc/han-secrets
|
||||||
|
install -m 0750 -o root -g root \
|
||||||
|
"${source_dir}/secrets_loader.py" \
|
||||||
|
/usr/local/lib/han-secrets/secrets_loader.py
|
||||||
|
install -m 0750 -o root -g root \
|
||||||
|
"${source_dir}/han-secrets" \
|
||||||
|
/usr/local/lib/han-secrets/han-secrets
|
||||||
|
install -m 0750 -o root -g root \
|
||||||
|
"${source_dir}/han-compose" \
|
||||||
|
/usr/local/bin/han-compose
|
||||||
|
install -m 0644 -o root -g root \
|
||||||
|
"${source_dir}/han-secrets@.service" \
|
||||||
|
/etc/systemd/system/han-secrets@.service
|
||||||
|
install -m 0644 -o root -g root \
|
||||||
|
"${source_dir}/SELECTEL_RUNBOOK.ru.md" \
|
||||||
|
/usr/local/share/doc/han-secrets/SELECTEL_RUNBOOK.ru.md
|
||||||
|
if [[ ! -e /etc/han/secrets/production.selectel.json.example ]]; then
|
||||||
|
install -m 0600 -o root -g root \
|
||||||
|
"${source_dir}/config.example.json" \
|
||||||
|
/etc/han/secrets/production.selectel.json.example
|
||||||
|
fi
|
||||||
|
systemctl daemon-reload
|
||||||
|
log "Загрузчик установлен, но не включен: сначала выполните SELECTEL_RUNBOOK.ru.md"
|
||||||
|
}
|
||||||
|
|
||||||
install_ssl_timer_if_possible() {
|
install_ssl_timer_if_possible() {
|
||||||
step "Таймер продления TLS"
|
step "Таймер продления TLS"
|
||||||
local renew_script="${DEPLOY_DIR}/deployment/scripts/ssl-renew.sh"
|
local renew_script="${DEPLOY_DIR}/deployment/scripts/ssl-renew.sh"
|
||||||
@@ -504,14 +544,18 @@ HSTS max-age: ${HSTS_MAX_AGE_SECONDS}
|
|||||||
${DEPLOY_DIR}
|
${DEPLOY_DIR}
|
||||||
3. Поместите CA PostgreSQL:
|
3. Поместите CA PostgreSQL:
|
||||||
${DEPLOY_DIR}/secrets/pg/ca.pem
|
${DEPLOY_DIR}/secrets/pg/ca.pem
|
||||||
4. Создайте и заполните окружение:
|
4. Создайте только несекретный config:
|
||||||
cd ${DEPLOY_DIR}
|
cd ${DEPLOY_DIR}
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
chmod 600 .env
|
chmod 600 .env
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
5. Продолжите с Gate 7 в:
|
5. Настройте Selectel, encrypted bootstrap credential и fallback map:
|
||||||
|
deployment/secrets/SELECTEL_RUNBOOK.ru.md
|
||||||
|
6. Выполняйте Compose только через:
|
||||||
|
sudo deployment/secrets/han-compose <command>
|
||||||
|
7. Продолжите с Gate 7 в:
|
||||||
deployment/RUNBOOK.ru.md
|
deployment/RUNBOOK.ru.md
|
||||||
6. После копирования проекта повторно запустите этот скрипт для установки TLS-таймера.
|
8. После копирования проекта повторно запустите этот скрипт для установки unit-файлов.
|
||||||
|
|
||||||
Важно: членство в группе docker начнет действовать после нового входа в систему.
|
Важно: членство в группе docker начнет действовать после нового входа в систему.
|
||||||
EOF
|
EOF
|
||||||
@@ -538,6 +582,7 @@ main() {
|
|||||||
configure_docker_firewall
|
configure_docker_firewall
|
||||||
configure_ssh
|
configure_ssh
|
||||||
configure_application_security
|
configure_application_security
|
||||||
|
install_secret_loader_if_possible
|
||||||
install_ssl_timer_if_possible
|
install_ssl_timer_if_possible
|
||||||
verify
|
verify
|
||||||
summary
|
summary
|
||||||
|
|||||||
@@ -2,11 +2,23 @@
|
|||||||
set -eu
|
set -eu
|
||||||
cd "$(dirname "$0")/../.."
|
cd "$(dirname "$0")/../.."
|
||||||
|
|
||||||
ENV_FILE=${ENV_FILE:-.env}
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
./scripts/validate-env "$ENV_FILE"
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
if [ -n "${HAN_RUNTIME_SECRET_MANIFEST:-}" ]; then
|
||||||
|
./scripts/validate-env "$CONFIG_FILE" --runtime-manifest "$HAN_RUNTIME_SECRET_MANIFEST"
|
||||||
|
else
|
||||||
|
./scripts/validate-env "$CONFIG_FILE" --runtime-env
|
||||||
|
fi
|
||||||
|
|
||||||
env_value() {
|
env_value() {
|
||||||
python3 - "$ENV_FILE" "$1" <<'PY'
|
python3 - "$CONFIG_FILE" "$1" <<'PY'
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,23 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
cd "$(dirname "$0")/../.."
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
|
SECRETS_LAUNCHER=${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}
|
||||||
|
|
||||||
|
if [ "${HAN_SECRETS_ACTIVE:-0}" != "1" ]; then
|
||||||
|
[ -x "$SECRETS_LAUNCHER" ] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
lock=/tmp/han-chat-cert-renew.lock
|
lock=/tmp/han-chat-cert-renew.lock
|
||||||
exec 9>"$lock"
|
exec 9>"$lock"
|
||||||
flock -n 9 || { echo '{"event":"tls.renew.skipped","reason":"lock_busy"}'; exit 0; }
|
flock -n 9 || { echo '{"event":"tls.renew.skipped","reason":"lock_busy"}'; exit 0; }
|
||||||
|
|
||||||
compose() {
|
compose() {
|
||||||
docker compose --env-file .env "$@"
|
docker compose --env-file "$CONFIG_FILE" "$@"
|
||||||
}
|
}
|
||||||
|
|
||||||
nginx_container="$(compose ps --status running --quiet nginx)"
|
nginx_container="$(compose ps --status running --quiet nginx)"
|
||||||
|
|||||||
@@ -1,5 +1,18 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -Eeuo pipefail
|
set -Eeuo pipefail
|
||||||
|
cd "$(dirname "$0")/../.."
|
||||||
|
CONFIG_FILE="${CONFIG_FILE:-.env}"
|
||||||
|
SECRETS_LAUNCHER="${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}"
|
||||||
|
|
||||||
|
if [[ "${HAN_SECRETS_ACTIVE:-0}" != 1 ]]; then
|
||||||
|
[[ -x "$SECRETS_LAUNCHER" ]] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
|
||||||
|
compose() { docker compose --env-file "$CONFIG_FILE" "$@"; }
|
||||||
|
|
||||||
NETWORK="${OBSERVABILITY_NETWORK:-han-chat-observability}"
|
NETWORK="${OBSERVABILITY_NETWORK:-han-chat-observability}"
|
||||||
COLLECTOR_SERVICE="${COLLECTOR_SERVICE:-otel-collector}"
|
COLLECTOR_SERVICE="${COLLECTOR_SERVICE:-otel-collector}"
|
||||||
@@ -8,7 +21,7 @@ errors=0
|
|||||||
ok() { printf 'OK %s\n' "$*"; }
|
ok() { printf 'OK %s\n' "$*"; }
|
||||||
fail() { printf 'FAIL %s\n' "$*" >&2; errors=$((errors + 1)); }
|
fail() { printf 'FAIL %s\n' "$*" >&2; errors=$((errors + 1)); }
|
||||||
|
|
||||||
collector_id="$(docker compose ps -q "$COLLECTOR_SERVICE" 2>/dev/null || true)"
|
collector_id="$(compose ps -q "$COLLECTOR_SERVICE" 2>/dev/null || true)"
|
||||||
if [[ -n "$collector_id" ]] &&
|
if [[ -n "$collector_id" ]] &&
|
||||||
[[ "$(docker inspect --format '{{.State.Status}}' "$collector_id")" == running ]]; then
|
[[ "$(docker inspect --format '{{.State.Status}}' "$collector_id")" == running ]]; then
|
||||||
ok "Collector service is running"
|
ok "Collector service is running"
|
||||||
@@ -17,7 +30,7 @@ else
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
for service in sms-service sms-worker; do
|
for service in sms-service sms-worker; do
|
||||||
if docker compose exec -T "$service" python - <<'PY' >/dev/null 2>&1
|
if compose exec -T "$service" python - <<'PY' >/dev/null 2>&1
|
||||||
import os
|
import os
|
||||||
import urllib.request
|
import urllib.request
|
||||||
port = os.environ.get("SMS_METRICS_PORT", "9464") if "worker" in os.environ.get("OTEL_SERVICE_NAME", "") else "8080"
|
port = os.environ.get("SMS_METRICS_PORT", "9464") if "worker" in os.environ.get("OTEL_SERVICE_NAME", "") else "8080"
|
||||||
@@ -38,7 +51,7 @@ docker run --rm --network "$NETWORK" \
|
|||||||
|| fail "telemetrygen failed"
|
|| fail "telemetrygen failed"
|
||||||
|
|
||||||
bad_logs="$(
|
bad_logs="$(
|
||||||
docker compose logs --since=10m "$COLLECTOR_SERVICE" 2>&1 |
|
compose logs --since=10m "$COLLECTOR_SERVICE" 2>&1 |
|
||||||
grep -Ei 'queue is full|connection refused|tls:|Unauthenticated|Permanent error' || true
|
grep -Ei 'queue is full|connection refused|tls:|Unauthenticated|Permanent error' || true
|
||||||
)"
|
)"
|
||||||
if [[ -z "$bad_logs" ]]; then
|
if [[ -z "$bad_logs" ]]; then
|
||||||
|
|||||||
@@ -0,0 +1,206 @@
|
|||||||
|
# Selectel Secrets Manager для HAN Chat
|
||||||
|
|
||||||
|
## Модель
|
||||||
|
|
||||||
|
`han-secrets` читает только `SECRETS_SOURCE=selectel|file` из обычного `.env`,
|
||||||
|
выбирает соответствующую root-only JSON-карту и вызывает `secrets_loader.py`.
|
||||||
|
Загрузчик получает project-scoped IAM token, читает объявленные секреты и
|
||||||
|
создаёт в `/run/han-chat/secrets`:
|
||||||
|
|
||||||
|
- отдельные файлы с каноническими именами для Compose secrets;
|
||||||
|
- узкие service dotenv-файлы для диагностики состава без вывода значений;
|
||||||
|
- `manifest` вида `NAME=/absolute/path`, используемый валидатором.
|
||||||
|
|
||||||
|
Каталог `/run` находится в tmpfs и имеет режим `0700`. Канонические файлы имеют
|
||||||
|
`0444`: локальные пользователи не могут пройти через root-only каталог, а
|
||||||
|
не-root UID контейнера может прочитать только явно смонтированный Compose
|
||||||
|
secret. Значения не передаются через Docker Config.Env, argv, общий `.env` или
|
||||||
|
логи. После полного root/docker-компромисса runtime-значения извлекаемы — это
|
||||||
|
ограничение модели, а не гарантия Secret Manager.
|
||||||
|
|
||||||
|
Сбой Selectel никогда автоматически не включает file fallback. Уже работающие
|
||||||
|
контейнеры продолжают использовать текущие значения; новый sync завершается
|
||||||
|
fail-closed.
|
||||||
|
|
||||||
|
## 1. Ресурсы Selectel
|
||||||
|
|
||||||
|
1. Создайте отдельный проект `han-chat-secrets-prod`.
|
||||||
|
2. Создайте сервисного пользователя `han-chat-secrets-reader`.
|
||||||
|
3. Назначьте ему `member` только в этом проекте. Не выдавайте account scope,
|
||||||
|
`iam.admin` и доступ к другим production-ресурсам. Если Selectel добавит
|
||||||
|
отдельную read-only роль Secrets Manager, замените `member` на неё.
|
||||||
|
4. Ограничьте обращения к `api.selectel.ru` исходящим IP ВМ, если функция
|
||||||
|
доступна в аккаунте.
|
||||||
|
5. Включите экспорт audit logs. Контролируйте события `secrets.secret*` и
|
||||||
|
`secrets.secret_version*`; alert на delete, смену current version вне окна,
|
||||||
|
массовые чтения и обращения не от штатного пользователя/IP.
|
||||||
|
6. Выполните canary и убедитесь, что provider audit содержит metadata операции,
|
||||||
|
но не value, Base64 payload, IAM token или response body.
|
||||||
|
|
||||||
|
Secrets Manager принимает project-scoped IAM token в `X-Auth-Token`. Token
|
||||||
|
живёт до 24 часов, но загрузчик использует его только в памяти одного запуска.
|
||||||
|
TLS и redirect policy отключать нельзя.
|
||||||
|
|
||||||
|
## 2. Каталог секретов
|
||||||
|
|
||||||
|
Скопируйте `config.example.json` в
|
||||||
|
`/etc/han/secrets/production-like.selectel.json` и замените account, username,
|
||||||
|
project, region и `remote`. Каноническое имя слева обязано совпадать с
|
||||||
|
Compose/validator; `remote` — неизменяемый ключ в Selectel.
|
||||||
|
|
||||||
|
Используйте консервативные provider keys с дефисами, например:
|
||||||
|
|
||||||
|
- `han-chat-prod-pg-han-app-dsn`, `han-chat-prod-pg-bitrix-local-dsn`,
|
||||||
|
`han-chat-prod-pg-bitrix-sync-dsn`, `han-chat-prod-pg-sms-dsn`,
|
||||||
|
`han-chat-prod-pg-keycloak-password`, `han-chat-prod-pg-backup-dsn`;
|
||||||
|
- `han-chat-prod-redis-api-password`, `han-chat-prod-redis-safety-password`,
|
||||||
|
`han-chat-prod-redis-health-password`, а также три credential-bearing URL;
|
||||||
|
- `han-chat-prod-message-safety-token`, `han-chat-prod-bitrix-internal-token`,
|
||||||
|
`han-chat-prod-bitrix-forward-token`, `han-chat-prod-bitrix-sync-token`,
|
||||||
|
`han-chat-prod-keycloak-settings-token`, `han-chat-prod-sms-service-token`;
|
||||||
|
- Keycloak bootstrap password, OTP HMAC и mock code только для среды, где mock
|
||||||
|
действительно включён;
|
||||||
|
- Bitrix client secret, application token и token encryption key;
|
||||||
|
- пары S3 app access/secret key;
|
||||||
|
- i-Digital API key и callback username/password.
|
||||||
|
|
||||||
|
Одинаковые пары env (`BITRIX_LOCAL_APP_INTERNAL_TOKEN` /
|
||||||
|
`BITRIX_INTERNAL_API_TOKEN`, `BITRIX_API_FORWARD_TOKEN` /
|
||||||
|
`BITRIX_API_INBOX_TOKEN`, `SMS_SERVICE_TOKEN` /
|
||||||
|
`KEYCLOAK_SMS_SERVICE_TOKEN`) должны ссылаться на один `remote`.
|
||||||
|
|
||||||
|
`literal: ""` разрешён только для заведомо пустого optional-параметра, например
|
||||||
|
OTLP auth при self-hosted SigNoz или выключенного CAPTCHA server key. Секреты
|
||||||
|
не записывайте в JSON. Не заводите planned/unused Message Safety DB, quarantine
|
||||||
|
S3 и Bitrix sync credentials до появления потребляющего кода.
|
||||||
|
|
||||||
|
Загружайте значения в Selectel через скрытый prompt/stdin. Не передавайте value
|
||||||
|
позиционным аргументом CLI и не включайте `set -x`/`curl -v`.
|
||||||
|
|
||||||
|
## 3. Установка на ВМ
|
||||||
|
|
||||||
|
Повторный запуск `deployment/scripts/setup-vm.sh` после копирования проекта
|
||||||
|
устанавливает loader, launcher, systemd template и этот runbook. Вручную:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo install -d -m 0700 /etc/han/secrets /etc/han/credentials
|
||||||
|
sudo install -d -m 0755 /usr/local/lib/han-secrets
|
||||||
|
sudo install -m 0750 secrets_loader.py han-secrets /usr/local/lib/han-secrets/
|
||||||
|
sudo install -m 0750 han-compose /usr/local/bin/han-compose
|
||||||
|
sudo install -m 0644 han-secrets@.service /etc/systemd/system/
|
||||||
|
sudo install -m 0600 config.example.json \
|
||||||
|
/etc/han/secrets/production-like.selectel.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Обычный `/opt/han-chat/backend/.env` содержит только несекретные параметры.
|
||||||
|
Штатный режим:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
SECRETS_SOURCE=selectel
|
||||||
|
APP_ENV=production-like
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Bootstrap credential
|
||||||
|
|
||||||
|
Не храните пароль service user в `.env` или JSON. На целевой ВМ:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo systemd-creds encrypt --name=selectel-service-user-password - \
|
||||||
|
/etc/han/credentials/production.selectel-password.cred
|
||||||
|
sudo chmod 0600 /etc/han/credentials/production.selectel-password.cred
|
||||||
|
```
|
||||||
|
|
||||||
|
Введите пароль через интерактивный stdin. Unit передаёт расшифрованный файл
|
||||||
|
через приватный `$CREDENTIALS_DIRECTORY`. Root всё равно может его извлечь;
|
||||||
|
после инцидента credential и все доступные ему секреты необходимо ротировать.
|
||||||
|
|
||||||
|
## 5. Проверка и запуск
|
||||||
|
|
||||||
|
Все команды, которым нужны Compose secrets, запускайте от root через wrapper:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd /opt/han-chat/backend
|
||||||
|
sudo ./scripts/validate-env .env
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable han-secrets@production.service
|
||||||
|
sudo systemctl restart han-secrets@production.service
|
||||||
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
|
sudo deployment/secrets/han-compose up -d --wait
|
||||||
|
```
|
||||||
|
|
||||||
|
Selectel sync запускается именно unit-файлом: только он предоставляет
|
||||||
|
расшифрованный bootstrap credential через `$CREDENTIALS_DIRECTORY`.
|
||||||
|
`han-compose` и ops-скрипты используют уже синхронизированный manifest и
|
||||||
|
отказываются работать, если `SECRETS_SOURCE`/loader config не совпадают с
|
||||||
|
runtime state. После смены source, provider version или JSON-карты сначала
|
||||||
|
выполняйте `systemctl restart han-secrets@production.service`.
|
||||||
|
|
||||||
|
Не используйте `docker compose config` без `--quiet`, `docker inspect` для
|
||||||
|
поиска конфигурации, `env`, `strace`, core dump или debug HTTP proxy. Проверка
|
||||||
|
приёмки должна подтвердить отсутствие canary value в `docker inspect`, stdout,
|
||||||
|
json logs, traces и shell history.
|
||||||
|
|
||||||
|
На выделенной только под HAN Chat ВМ после canary и проверки file fallback
|
||||||
|
установите fail-closed ordering:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo install -d -m 0755 /etc/systemd/system/docker.service.d
|
||||||
|
sudo install -m 0644 \
|
||||||
|
deployment/secrets/docker-han-secrets.conf.example \
|
||||||
|
/etc/systemd/system/docker.service.d/han-secrets.conf
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl restart docker
|
||||||
|
```
|
||||||
|
|
||||||
|
После этого проведите reboot rehearsal: materializer должен завершиться до
|
||||||
|
autorestart контейнеров. Ошибка Selectel намеренно блокирует Docker. На ВМ с
|
||||||
|
другими workloads такой глобальный `Requires=` запрещён: нужен отдельный Docker
|
||||||
|
daemon/VM, иначе fail-closed HAN остановит несвязанные системы.
|
||||||
|
|
||||||
|
## 6. Явный file fallback
|
||||||
|
|
||||||
|
Подготовьте отдельную карту
|
||||||
|
`/etc/han/secrets/production-like.file.json`: скопируйте Selectel-карту,
|
||||||
|
установите `"mode": "file"`, удалите `selectel` и `http`, добавьте:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"file": {
|
||||||
|
"path": "/etc/han/break-glass/secrets.env",
|
||||||
|
"max_bytes": 1048576
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
`secrets` map остаётся тем же. Для записей с `literal: ""` строка в fallback
|
||||||
|
не нужна; остальные канонические ключи обязательны. Fallback parser не исполняет
|
||||||
|
shell: запрещены `export`, substitutions, multiline, неизвестные и дублирующиеся
|
||||||
|
ключи. Файл — `root:root 0600`.
|
||||||
|
|
||||||
|
При инциденте доставьте recovery-файл из защищённой офлайн-копии и только затем
|
||||||
|
явно измените `.env`:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
SECRETS_SOURCE=file
|
||||||
|
```
|
||||||
|
|
||||||
|
Перезапустите `han-secrets@production.service`, затем выполните
|
||||||
|
validate/recreate через wrappers. После восстановления Selectel верните
|
||||||
|
`SECRETS_SOURCE=selectel`, снова перезапустите unit, повторите проверки и
|
||||||
|
удалите recovery-файл.
|
||||||
|
Не храните его постоянно на ВМ: это вернуло бы исходный риск монолитного `.env`.
|
||||||
|
|
||||||
|
## 7. Ротация и rollback
|
||||||
|
|
||||||
|
1. Добавьте новую версию секрета, не меняя имя.
|
||||||
|
2. Для canary при необходимости временно pin числовой `version` в JSON-карте.
|
||||||
|
3. Выполните sync, validation и smoke без вывода конфигурации.
|
||||||
|
4. Сделайте версию current, удалите pin, снова sync и пересоздайте только
|
||||||
|
потребителей.
|
||||||
|
5. Для rollback активируйте предыдущую provider version; не храните snapshot
|
||||||
|
старого `.env`.
|
||||||
|
|
||||||
|
Selectel не позволяет удалить отдельную версию — только секрет целиком. Старые
|
||||||
|
значения должны быть отозваны в PostgreSQL/S3/Bitrix/i-Digital после окна
|
||||||
|
rollback. Все значения из прежнего `.env` считайте раскрытыми и ротируйте после
|
||||||
|
перехода.
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"mode": "selectel",
|
||||||
|
"runtime_dir": "/run/han-chat/secrets",
|
||||||
|
"http": {
|
||||||
|
"timeout_seconds": 10,
|
||||||
|
"retries": 3,
|
||||||
|
"max_response_bytes": 1048576
|
||||||
|
},
|
||||||
|
"selectel": {
|
||||||
|
"account_id": "123456",
|
||||||
|
"username": "han-secrets-reader",
|
||||||
|
"project_name": "han-production",
|
||||||
|
"region": "ru-9",
|
||||||
|
"interface": "public",
|
||||||
|
"password_file": "selectel-service-user-password"
|
||||||
|
},
|
||||||
|
"secrets": {
|
||||||
|
"DATABASE_URL": {
|
||||||
|
"remote": "DATABASE_URL",
|
||||||
|
"consumers": ["api-backend", "api-migrate"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"BITRIX_DATABASE_URL": {
|
||||||
|
"remote": "BITRIX_DATABASE_URL",
|
||||||
|
"consumers": ["bitrix-local-app", "bitrix-local-migrate"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"BITRIX_SYNC_DATABASE_URL": {
|
||||||
|
"remote": "BITRIX_SYNC_DATABASE_URL",
|
||||||
|
"consumers": ["bitrix-sync", "bitrix-sync-migrate"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"SMS_DATABASE_URL": {
|
||||||
|
"remote": "SMS_DATABASE_URL",
|
||||||
|
"consumers": ["sms-service", "sms-worker", "sms-migrate"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"KEYCLOAK_DB_PASSWORD": {
|
||||||
|
"remote": "KEYCLOAK_DB_PASSWORD",
|
||||||
|
"consumers": ["keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_ADMIN_PASSWORD": {
|
||||||
|
"remote": "KEYCLOAK_ADMIN_PASSWORD",
|
||||||
|
"consumers": ["keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"CURSOR_HMAC_SECRET": {
|
||||||
|
"remote": "CURSOR_HMAC_SECRET",
|
||||||
|
"consumers": ["api-backend"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_OTP_HMAC_KEY": {
|
||||||
|
"remote": "KEYCLOAK_OTP_HMAC_KEY",
|
||||||
|
"consumers": ["keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_OTP_MOCK_CODE": {
|
||||||
|
"remote": "KEYCLOAK_OTP_MOCK_CODE",
|
||||||
|
"consumers": ["keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY": {
|
||||||
|
"remote": "KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
||||||
|
"consumers": ["keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_TOKEN_ENCRYPTION_KEY": {
|
||||||
|
"remote": "BITRIX_TOKEN_ENCRYPTION_KEY",
|
||||||
|
"consumers": ["api-backend", "bitrix-local-app", "bitrix-sync"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"REDIS_API_PASSWORD": {
|
||||||
|
"remote": "REDIS_API_PASSWORD",
|
||||||
|
"consumers": ["redis"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"REDIS_URL": {
|
||||||
|
"remote": "REDIS_URL",
|
||||||
|
"consumers": ["api-backend", "delivery-worker", "cleanup-worker"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"REDIS_REALTIME_URL": {
|
||||||
|
"remote": "REDIS_REALTIME_URL",
|
||||||
|
"consumers": ["api-backend"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"REDIS_SAFETY_PASSWORD": {
|
||||||
|
"remote": "REDIS_SAFETY_PASSWORD",
|
||||||
|
"consumers": ["redis"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"MESSAGE_SAFETY_REDIS_URL": {
|
||||||
|
"remote": "MESSAGE_SAFETY_REDIS_URL",
|
||||||
|
"consumers": ["message-safety", "safety-recovery-worker"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
},
|
||||||
|
"REDIS_HEALTH_PASSWORD": {
|
||||||
|
"remote": "REDIS_HEALTH_PASSWORD",
|
||||||
|
"consumers": ["redis", "redis-exporter"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"MESSAGE_SAFETY_SERVICE_TOKEN": {
|
||||||
|
"remote": "MESSAGE_SAFETY_SERVICE_TOKEN",
|
||||||
|
"consumers": ["api-backend", "message-safety"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_LOCAL_APP_INTERNAL_TOKEN": {
|
||||||
|
"remote": "BITRIX_LOCAL_APP_INTERNAL_TOKEN",
|
||||||
|
"consumers": ["api-backend", "bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_INTERNAL_API_TOKEN": {
|
||||||
|
"remote": "BITRIX_LOCAL_APP_INTERNAL_TOKEN",
|
||||||
|
"consumers": ["api-backend", "bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_API_FORWARD_TOKEN": {
|
||||||
|
"remote": "BITRIX_API_FORWARD_TOKEN",
|
||||||
|
"consumers": ["api-backend", "bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_API_INBOX_TOKEN": {
|
||||||
|
"remote": "BITRIX_API_FORWARD_TOKEN",
|
||||||
|
"consumers": ["api-backend", "bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_SYNC_SERVICE_TOKEN": {
|
||||||
|
"remote": "BITRIX_SYNC_SERVICE_TOKEN",
|
||||||
|
"consumers": ["api-backend", "bitrix-sync"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_SETTINGS_BRIDGE_TOKEN": {
|
||||||
|
"remote": "KEYCLOAK_SETTINGS_BRIDGE_TOKEN",
|
||||||
|
"consumers": ["api-backend", "keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"SMS_SERVICE_TOKEN": {
|
||||||
|
"remote": "SMS_SERVICE_TOKEN",
|
||||||
|
"consumers": ["sms-service", "sms-worker", "keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"KEYCLOAK_SMS_SERVICE_TOKEN": {
|
||||||
|
"remote": "SMS_SERVICE_TOKEN",
|
||||||
|
"consumers": ["sms-service", "sms-worker", "keycloak"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"IDGTL_SMS_API_KEY": {
|
||||||
|
"remote": "IDGTL_SMS_API_KEY",
|
||||||
|
"consumers": ["sms-service", "sms-worker"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"IDGTL_SMS_CALLBACK_USERNAME": {
|
||||||
|
"remote": "IDGTL_SMS_CALLBACK_USERNAME",
|
||||||
|
"consumers": ["sms-service", "sms-worker"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"IDGTL_SMS_CALLBACK_PASSWORD": {
|
||||||
|
"remote": "IDGTL_SMS_CALLBACK_PASSWORD",
|
||||||
|
"consumers": ["sms-service", "sms-worker"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_CLIENT_SECRET": {
|
||||||
|
"remote": "BITRIX_CLIENT_SECRET",
|
||||||
|
"consumers": ["bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"BITRIX_APPLICATION_TOKEN": {
|
||||||
|
"remote": "BITRIX_APPLICATION_TOKEN",
|
||||||
|
"consumers": ["bitrix-local-app"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"SELECTEL_S3_SECRET_KEY": {
|
||||||
|
"remote": "SELECTEL_S3_SECRET_KEY",
|
||||||
|
"consumers": ["api-backend"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"SELECTEL_S3_ACCESS_KEY": {
|
||||||
|
"remote": "SELECTEL_S3_ACCESS_KEY",
|
||||||
|
"consumers": ["api-backend"],
|
||||||
|
"max_bytes": 1024
|
||||||
|
},
|
||||||
|
"OTEL_REMOTE_AUTH_HEADER": {
|
||||||
|
"literal": "",
|
||||||
|
"consumers": ["otel-collector"],
|
||||||
|
"max_bytes": 4096
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
[Unit]
|
||||||
|
# Enable only on a dedicated HAN Chat Docker host after the Selectel canary and
|
||||||
|
# file fallback have both passed. A failed secret sync intentionally blocks
|
||||||
|
# Docker startup so containers cannot race an empty /run directory.
|
||||||
|
Requires=han-secrets@production.service
|
||||||
|
After=han-secrets@production.service
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
DEPLOY_DIR=${HAN_DEPLOY_DIR:-/opt/han-chat/backend}
|
||||||
|
SOURCE_DIR=$(CDPATH= cd -- "$(dirname "$0")" && pwd)
|
||||||
|
[ ! -f "$SOURCE_DIR/../../docker-compose.yml" ] || \
|
||||||
|
DEPLOY_DIR=$(CDPATH= cd -- "$SOURCE_DIR/../.." && pwd)
|
||||||
|
cd "$DEPLOY_DIR"
|
||||||
|
|
||||||
|
CONFIG_FILE=${CONFIG_FILE:-.env}
|
||||||
|
LAUNCHER=${HAN_SECRETS_LAUNCHER:-/usr/local/lib/han-secrets/han-secrets}
|
||||||
|
[ -x "$LAUNCHER" ] || LAUNCHER=deployment/secrets/han-secrets
|
||||||
|
|
||||||
|
exec "$LAUNCHER" run --config "$CONFIG_FILE" -- \
|
||||||
|
docker compose --env-file "$CONFIG_FILE" "$@"
|
||||||
@@ -0,0 +1,175 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Synchronize runtime secrets and execute a command without exporting their values."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from secrets_loader import LoaderError, load_json, run
|
||||||
|
|
||||||
|
|
||||||
|
def load_public_config(path: Path) -> dict[str, str]:
|
||||||
|
values: dict[str, str] = {}
|
||||||
|
try:
|
||||||
|
lines = path.read_text(encoding="utf-8").splitlines()
|
||||||
|
except OSError as exc:
|
||||||
|
raise LoaderError(
|
||||||
|
f"cannot read non-secret config: {exc.strerror or exc.__class__.__name__}"
|
||||||
|
) from None
|
||||||
|
for number, raw in enumerate(lines, 1):
|
||||||
|
line = raw.strip()
|
||||||
|
if not line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
if "=" not in line:
|
||||||
|
raise LoaderError(f"non-secret config has invalid syntax at line {number}")
|
||||||
|
key, value = line.split("=", 1)
|
||||||
|
key = key.strip()
|
||||||
|
if not key or key in values:
|
||||||
|
raise LoaderError(f"non-secret config has an invalid key at line {number}")
|
||||||
|
values[key] = value.strip()
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def loader_config_path(
|
||||||
|
public: dict[str, str],
|
||||||
|
explicit: Path | None,
|
||||||
|
environ: dict[str, str],
|
||||||
|
) -> tuple[str, Path]:
|
||||||
|
source = public.get("SECRETS_SOURCE")
|
||||||
|
if source not in {"selectel", "file"}:
|
||||||
|
raise LoaderError("SECRETS_SOURCE must explicitly be selectel or file")
|
||||||
|
if explicit is not None:
|
||||||
|
return source, explicit
|
||||||
|
override = environ.get(f"HAN_SECRETS_{source.upper()}_CONFIG")
|
||||||
|
if override:
|
||||||
|
return source, Path(override)
|
||||||
|
environment = public.get("APP_ENV", "production")
|
||||||
|
return source, Path(f"/etc/han/secrets/{environment}.{source}.json")
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_environment(
|
||||||
|
config_path: Path,
|
||||||
|
source: str,
|
||||||
|
environ: dict[str, str],
|
||||||
|
*,
|
||||||
|
synchronize: bool,
|
||||||
|
) -> dict[str, str]:
|
||||||
|
document = load_json(config_path)
|
||||||
|
if document.get("mode") != source:
|
||||||
|
raise LoaderError("selected loader configuration mode does not match SECRETS_SOURCE")
|
||||||
|
runtime = document.get("runtime_dir")
|
||||||
|
if not isinstance(runtime, str) or not Path(runtime).is_absolute():
|
||||||
|
raise LoaderError("loader configuration has an invalid runtime_dir")
|
||||||
|
runtime_dir = Path(runtime)
|
||||||
|
manifest = runtime_dir / "manifest"
|
||||||
|
state_path = runtime_dir / "state.json"
|
||||||
|
consumers: list[str] = []
|
||||||
|
if synchronize or (source == "file" and not state_path.is_file()):
|
||||||
|
consumers = run(config_path, environ=environ)
|
||||||
|
state = {
|
||||||
|
"version": 1,
|
||||||
|
"source": source,
|
||||||
|
"loader_config": str(config_path.resolve()),
|
||||||
|
}
|
||||||
|
descriptor, temporary = tempfile.mkstemp(
|
||||||
|
prefix=".state.", suffix=".tmp", dir=runtime_dir
|
||||||
|
)
|
||||||
|
temporary_path = Path(temporary)
|
||||||
|
try:
|
||||||
|
os.chmod(temporary_path, 0o600)
|
||||||
|
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
|
||||||
|
descriptor = -1
|
||||||
|
json.dump(state, stream, separators=(",", ":"))
|
||||||
|
stream.write("\n")
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
os.replace(temporary_path, state_path)
|
||||||
|
except BaseException:
|
||||||
|
if descriptor >= 0:
|
||||||
|
os.close(descriptor)
|
||||||
|
temporary_path.unlink(missing_ok=True)
|
||||||
|
raise
|
||||||
|
elif not state_path.is_file():
|
||||||
|
raise LoaderError(
|
||||||
|
"runtime secrets are not synchronized; restart han-secrets systemd unit"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
state = load_json(state_path)
|
||||||
|
if (
|
||||||
|
state.get("version") != 1
|
||||||
|
or state.get("source") != source
|
||||||
|
or state.get("loader_config") != str(config_path.resolve())
|
||||||
|
):
|
||||||
|
raise LoaderError(
|
||||||
|
"runtime secret state does not match selected source/config; synchronize first"
|
||||||
|
)
|
||||||
|
if not manifest.is_file():
|
||||||
|
raise LoaderError("runtime secret manifest was not materialized")
|
||||||
|
manifest_entries: dict[str, str] = {}
|
||||||
|
for line in manifest.read_text(encoding="utf-8").splitlines():
|
||||||
|
key, value_path = line.split("=", 1)
|
||||||
|
manifest_entries[key] = value_path
|
||||||
|
specs = document.get("secrets")
|
||||||
|
if not isinstance(specs, dict) or set(manifest_entries) != set(specs):
|
||||||
|
raise LoaderError("runtime secret manifest does not match loader configuration")
|
||||||
|
child = dict(environ)
|
||||||
|
child["HAN_SECRETS_ACTIVE"] = "1"
|
||||||
|
child["HAN_RUNTIME_SECRET_DIR"] = str(runtime_dir)
|
||||||
|
child["HAN_RUNTIME_SECRET_MANIFEST"] = str(manifest)
|
||||||
|
for key, value_path in manifest_entries.items():
|
||||||
|
if not Path(value_path).is_file():
|
||||||
|
raise LoaderError("runtime secret manifest references a missing file")
|
||||||
|
child[f"{key}_FILE"] = value_path
|
||||||
|
if synchronize or consumers:
|
||||||
|
print(
|
||||||
|
f"han-secrets: synchronized {len(consumers)} service scope(s) from {source}",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return child
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description=__doc__)
|
||||||
|
parser.add_argument("action", choices=("sync", "run"))
|
||||||
|
parser.add_argument("--config", type=Path, default=Path(".env"))
|
||||||
|
parser.add_argument("--loader-config", type=Path)
|
||||||
|
arguments, command = parser.parse_known_args(argv)
|
||||||
|
if command and command[0] == "--":
|
||||||
|
command.pop(0)
|
||||||
|
if arguments.action == "run" and not command:
|
||||||
|
parser.error("run requires a command after --")
|
||||||
|
if arguments.action == "sync" and command:
|
||||||
|
parser.error("sync does not accept a command")
|
||||||
|
|
||||||
|
environment = dict(os.environ)
|
||||||
|
try:
|
||||||
|
public = load_public_config(arguments.config)
|
||||||
|
source, loader_config = loader_config_path(
|
||||||
|
public, arguments.loader_config, environment
|
||||||
|
)
|
||||||
|
child = prepare_environment(
|
||||||
|
loader_config,
|
||||||
|
source,
|
||||||
|
environment,
|
||||||
|
synchronize=arguments.action == "sync",
|
||||||
|
)
|
||||||
|
except (LoaderError, OSError, ValueError, json.JSONDecodeError) as exc:
|
||||||
|
message = str(exc) if isinstance(exc, LoaderError) else exc.__class__.__name__
|
||||||
|
print(f"han-secrets: {message}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
if arguments.action == "sync":
|
||||||
|
return 0
|
||||||
|
if os.name == "nt":
|
||||||
|
return subprocess.call(command, env=child)
|
||||||
|
os.execvpe(command[0], command, child)
|
||||||
|
return 127
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Materialize HAN service secrets (%i)
|
||||||
|
Documentation=file:/usr/local/share/doc/han-secrets/SELECTEL_RUNBOOK.ru.md
|
||||||
|
Wants=network-online.target
|
||||||
|
After=network-online.target
|
||||||
|
Before=han-stack@%i.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
User=root
|
||||||
|
Group=root
|
||||||
|
UMask=0077
|
||||||
|
RuntimeDirectory=han-chat/secrets
|
||||||
|
RuntimeDirectoryMode=0700
|
||||||
|
ExecStart=/usr/bin/python3 /usr/local/lib/han-secrets/han-secrets sync --config /opt/han-chat/backend/.env
|
||||||
|
LoadCredentialEncrypted=selectel-service-user-password:/etc/han/credentials/%i.selectel-password.cred
|
||||||
|
RemainAfterExit=yes
|
||||||
|
StandardOutput=null
|
||||||
|
StandardError=journal
|
||||||
|
SyslogIdentifier=han-secrets-%i
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
PrivateDevices=yes
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
ProtectKernelTunables=yes
|
||||||
|
ProtectKernelModules=yes
|
||||||
|
ProtectKernelLogs=yes
|
||||||
|
ProtectControlGroups=yes
|
||||||
|
ProtectClock=yes
|
||||||
|
RestrictRealtime=yes
|
||||||
|
RestrictSUIDSGID=yes
|
||||||
|
LockPersonality=yes
|
||||||
|
MemoryDenyWriteExecute=yes
|
||||||
|
LimitCORE=0
|
||||||
|
SystemCallArchitectures=native
|
||||||
|
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||||
|
CapabilityBoundingSet=
|
||||||
|
AmbientCapabilities=
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
@@ -0,0 +1,682 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Materialize narrowly scoped service dotenv files from Selectel Secrets Manager."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import base64
|
||||||
|
import binascii
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import random
|
||||||
|
import re
|
||||||
|
import ssl
|
||||||
|
import stat
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Callable, Mapping, NoReturn
|
||||||
|
|
||||||
|
DEFAULT_IDENTITY_URL = "https://cloud.api.selcloud.ru/identity/v3/auth/tokens"
|
||||||
|
MAX_CONFIG_BYTES = 1_048_576
|
||||||
|
MAX_HTTP_BYTES = 1_048_576
|
||||||
|
MAX_SECRET_BYTES = 65_536
|
||||||
|
RETRYABLE_STATUS = frozenset({408, 425, 429, 500, 502, 503, 504})
|
||||||
|
ENV_NAME_RE = re.compile(r"^[A-Z][A-Z0-9_]*$")
|
||||||
|
SERVICE_NAME_RE = re.compile(r"^[a-zA-Z0-9][a-zA-Z0-9_.-]*$")
|
||||||
|
DOTENV_LINE_RE = re.compile(r"^([A-Z][A-Z0-9_]*)=(.*)$")
|
||||||
|
|
||||||
|
|
||||||
|
class LoaderError(Exception):
|
||||||
|
"""An expected, already-redacted loader failure."""
|
||||||
|
|
||||||
|
|
||||||
|
def fail(message: str) -> NoReturn:
|
||||||
|
raise LoaderError(message)
|
||||||
|
|
||||||
|
|
||||||
|
def _object(value: Any, label: str) -> dict[str, Any]:
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
fail(f"{label} must be an object")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _only_keys(value: Mapping[str, Any], allowed: set[str], label: str) -> None:
|
||||||
|
unknown = sorted(set(value) - allowed)
|
||||||
|
if unknown:
|
||||||
|
fail(f"{label} contains unsupported fields: {', '.join(unknown)}")
|
||||||
|
|
||||||
|
|
||||||
|
def _required_string(value: Mapping[str, Any], key: str, label: str) -> str:
|
||||||
|
item = value.get(key)
|
||||||
|
if not isinstance(item, str) or not item:
|
||||||
|
fail(f"{label}.{key} must be a non-empty string")
|
||||||
|
return item
|
||||||
|
|
||||||
|
|
||||||
|
def _bounded_int(value: Any, label: str, minimum: int, maximum: int) -> int:
|
||||||
|
if isinstance(value, bool) or not isinstance(value, int) or not minimum <= value <= maximum:
|
||||||
|
fail(f"{label} must be an integer from {minimum} through {maximum}")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def read_limited(path: Path, limit: int, label: str) -> bytes:
|
||||||
|
try:
|
||||||
|
with path.open("rb") as stream:
|
||||||
|
data = stream.read(limit + 1)
|
||||||
|
except OSError as exc:
|
||||||
|
fail(f"cannot read {label}: {exc.strerror or exc.__class__.__name__}")
|
||||||
|
if len(data) > limit:
|
||||||
|
fail(f"{label} exceeds {limit} bytes")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def require_private_regular_file(path: Path, label: str) -> None:
|
||||||
|
try:
|
||||||
|
metadata = path.lstat()
|
||||||
|
except OSError as exc:
|
||||||
|
fail(f"cannot inspect {label}: {exc.strerror or exc.__class__.__name__}")
|
||||||
|
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
|
||||||
|
fail(f"{label} must be a regular file and not a symlink")
|
||||||
|
if os.name != "nt" and stat.S_IMODE(metadata.st_mode) & 0o077:
|
||||||
|
fail(f"{label} must not be accessible by group or other users")
|
||||||
|
|
||||||
|
|
||||||
|
def load_json(path: Path) -> dict[str, Any]:
|
||||||
|
raw = read_limited(path, MAX_CONFIG_BYTES, "configuration")
|
||||||
|
try:
|
||||||
|
document = json.loads(raw.decode("utf-8"))
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
fail("configuration is not valid UTF-8 JSON")
|
||||||
|
return _object(document, "configuration")
|
||||||
|
|
||||||
|
|
||||||
|
def credential_value(selectel: Mapping[str, Any], environ: Mapping[str, str]) -> str:
|
||||||
|
methods = sum(key in selectel for key in ("password_file", "password_env"))
|
||||||
|
if methods != 1:
|
||||||
|
fail("selectel must set exactly one of password_file or password_env")
|
||||||
|
if "password_env" in selectel:
|
||||||
|
variable = _required_string(selectel, "password_env", "selectel")
|
||||||
|
if not ENV_NAME_RE.fullmatch(variable):
|
||||||
|
fail("selectel.password_env is not a valid environment variable name")
|
||||||
|
value = environ.get(variable)
|
||||||
|
if value is None or not value:
|
||||||
|
fail(f"credential environment variable {variable} is not set")
|
||||||
|
return value
|
||||||
|
|
||||||
|
configured = Path(_required_string(selectel, "password_file", "selectel"))
|
||||||
|
if configured.is_absolute():
|
||||||
|
path = configured
|
||||||
|
else:
|
||||||
|
directory = environ.get("CREDENTIALS_DIRECTORY")
|
||||||
|
if not directory:
|
||||||
|
fail("relative password_file requires CREDENTIALS_DIRECTORY")
|
||||||
|
path = Path(directory) / configured
|
||||||
|
require_private_regular_file(path, "credential")
|
||||||
|
raw = read_limited(path, 16_384, "credential")
|
||||||
|
try:
|
||||||
|
value = raw.decode("utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
fail("credential is not valid UTF-8")
|
||||||
|
value = value.removesuffix("\n").removesuffix("\r")
|
||||||
|
if not value or "\n" in value or "\r" in value or "\x00" in value:
|
||||||
|
fail("credential must contain exactly one non-empty text line")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||||
|
def redirect_request(self, req: Any, fp: Any, code: int, msg: str, headers: Any, newurl: str) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class HTTPResult:
|
||||||
|
status: int
|
||||||
|
headers: Mapping[str, str]
|
||||||
|
body: bytes
|
||||||
|
|
||||||
|
|
||||||
|
class HTTPClient:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
timeout: float,
|
||||||
|
retries: int,
|
||||||
|
max_response_bytes: int,
|
||||||
|
cafile: str | None = None,
|
||||||
|
opener: Any | None = None,
|
||||||
|
sleeper: Callable[[float], None] = time.sleep,
|
||||||
|
jitter: Callable[[], float] = random.random,
|
||||||
|
) -> None:
|
||||||
|
self.timeout = timeout
|
||||||
|
self.retries = retries
|
||||||
|
self.max_response_bytes = max_response_bytes
|
||||||
|
self.sleeper = sleeper
|
||||||
|
self.jitter = jitter
|
||||||
|
if opener is None:
|
||||||
|
try:
|
||||||
|
context = ssl.create_default_context(cafile=cafile)
|
||||||
|
except (OSError, ssl.SSLError) as exc:
|
||||||
|
fail(f"cannot initialize TLS trust store: {exc.__class__.__name__}")
|
||||||
|
self.opener = urllib.request.build_opener(
|
||||||
|
urllib.request.HTTPSHandler(context=context), NoRedirect()
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
self.opener = opener
|
||||||
|
|
||||||
|
def request(
|
||||||
|
self,
|
||||||
|
method: str,
|
||||||
|
url: str,
|
||||||
|
*,
|
||||||
|
headers: Mapping[str, str] | None = None,
|
||||||
|
body: bytes | None = None,
|
||||||
|
expected: frozenset[int],
|
||||||
|
) -> HTTPResult:
|
||||||
|
parsed = urllib.parse.urlsplit(url)
|
||||||
|
if parsed.scheme != "https" or not parsed.netloc or parsed.username or parsed.password:
|
||||||
|
fail("provider endpoint must be an HTTPS URL without embedded credentials")
|
||||||
|
request = urllib.request.Request(
|
||||||
|
url, data=body, headers=dict(headers or {}), method=method
|
||||||
|
)
|
||||||
|
for attempt in range(self.retries + 1):
|
||||||
|
try:
|
||||||
|
with self.opener.open(request, timeout=self.timeout) as response:
|
||||||
|
status = int(response.status)
|
||||||
|
content_length = response.headers.get("Content-Length")
|
||||||
|
if content_length:
|
||||||
|
try:
|
||||||
|
if int(content_length) > self.max_response_bytes:
|
||||||
|
fail("provider response exceeds configured limit")
|
||||||
|
except ValueError:
|
||||||
|
fail("provider returned an invalid Content-Length")
|
||||||
|
response_body = response.read(self.max_response_bytes + 1)
|
||||||
|
if len(response_body) > self.max_response_bytes:
|
||||||
|
fail("provider response exceeds configured limit")
|
||||||
|
if status not in expected:
|
||||||
|
fail(f"provider request failed with HTTP {status}")
|
||||||
|
return HTTPResult(status, response.headers, response_body)
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
status = int(exc.code)
|
||||||
|
if status not in RETRYABLE_STATUS or attempt >= self.retries:
|
||||||
|
fail(f"provider request failed with HTTP {status}")
|
||||||
|
except (urllib.error.URLError, TimeoutError, OSError):
|
||||||
|
if attempt >= self.retries:
|
||||||
|
fail("provider request failed after retries")
|
||||||
|
delay = min(8.0, 0.25 * (2**attempt)) * (0.5 + self.jitter())
|
||||||
|
self.sleeper(delay)
|
||||||
|
fail("provider request failed")
|
||||||
|
|
||||||
|
|
||||||
|
def parse_json_response(result: HTTPResult, label: str) -> dict[str, Any]:
|
||||||
|
try:
|
||||||
|
return _object(json.loads(result.body.decode("utf-8")), label)
|
||||||
|
except (UnicodeDecodeError, json.JSONDecodeError):
|
||||||
|
fail(f"{label} is not valid JSON")
|
||||||
|
|
||||||
|
|
||||||
|
def project_token_and_catalog(
|
||||||
|
client: HTTPClient, selectel: Mapping[str, Any], password: str
|
||||||
|
) -> tuple[str, list[Any]]:
|
||||||
|
identity_url = selectel.get("identity_url", DEFAULT_IDENTITY_URL)
|
||||||
|
if not isinstance(identity_url, str):
|
||||||
|
fail("selectel.identity_url must be a string")
|
||||||
|
account_id = _required_string(selectel, "account_id", "selectel")
|
||||||
|
username = _required_string(selectel, "username", "selectel")
|
||||||
|
project_name = _required_string(selectel, "project_name", "selectel")
|
||||||
|
payload = {
|
||||||
|
"auth": {
|
||||||
|
"identity": {
|
||||||
|
"methods": ["password"],
|
||||||
|
"password": {
|
||||||
|
"user": {
|
||||||
|
"name": username,
|
||||||
|
"domain": {"name": account_id},
|
||||||
|
"password": password,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"scope": {
|
||||||
|
"project": {
|
||||||
|
"name": project_name,
|
||||||
|
"domain": {"name": account_id},
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result = client.request(
|
||||||
|
"POST",
|
||||||
|
identity_url,
|
||||||
|
headers={"Content-Type": "application/json", "Accept": "application/json"},
|
||||||
|
body=json.dumps(payload, separators=(",", ":")).encode("utf-8"),
|
||||||
|
expected=frozenset({201}),
|
||||||
|
)
|
||||||
|
token = result.headers.get("X-Subject-Token")
|
||||||
|
if not isinstance(token, str) or not token:
|
||||||
|
fail("identity response omitted X-Subject-Token")
|
||||||
|
document = parse_json_response(result, "identity response")
|
||||||
|
token_data = document.get("token")
|
||||||
|
if not isinstance(token_data, dict):
|
||||||
|
fail("identity response omitted token metadata")
|
||||||
|
project = token_data.get("project")
|
||||||
|
if not isinstance(project, dict) or not project.get("id"):
|
||||||
|
fail("identity token is not project-scoped")
|
||||||
|
catalog = token_data.get("catalog")
|
||||||
|
if not isinstance(catalog, list):
|
||||||
|
fail("identity response omitted service catalog")
|
||||||
|
return token, catalog
|
||||||
|
|
||||||
|
|
||||||
|
def secrets_endpoint(catalog: list[Any], region: str, interface: str) -> str:
|
||||||
|
matches: list[str] = []
|
||||||
|
for service in catalog:
|
||||||
|
if not isinstance(service, dict) or service.get("type") != "secrets-manager":
|
||||||
|
continue
|
||||||
|
endpoints = service.get("endpoints")
|
||||||
|
if not isinstance(endpoints, list):
|
||||||
|
continue
|
||||||
|
for endpoint in endpoints:
|
||||||
|
if (
|
||||||
|
isinstance(endpoint, dict)
|
||||||
|
and endpoint.get("region") == region
|
||||||
|
and endpoint.get("interface") == interface
|
||||||
|
and isinstance(endpoint.get("url"), str)
|
||||||
|
):
|
||||||
|
matches.append(endpoint["url"].rstrip("/"))
|
||||||
|
if len(matches) != 1:
|
||||||
|
fail("service catalog did not contain exactly one matching Secrets Manager endpoint")
|
||||||
|
return matches[0]
|
||||||
|
|
||||||
|
|
||||||
|
def decode_secret(document: Mapping[str, Any], name: str, limit: int) -> bytes:
|
||||||
|
# GET /v1/{name} returns the current value inside ``version`` while
|
||||||
|
# GET /v1/{name}/versions/{id} returns a version object directly.
|
||||||
|
payload: Mapping[str, Any] = document
|
||||||
|
version = document.get("version")
|
||||||
|
if isinstance(version, dict):
|
||||||
|
payload = version
|
||||||
|
encoded = payload.get("value")
|
||||||
|
if not isinstance(encoded, str):
|
||||||
|
fail(f"secret {name} response omitted base64 value")
|
||||||
|
try:
|
||||||
|
value = base64.b64decode(encoded, validate=True)
|
||||||
|
except (binascii.Error, ValueError):
|
||||||
|
fail(f"secret {name} has invalid base64 encoding")
|
||||||
|
if not value:
|
||||||
|
fail(f"secret {name} is empty")
|
||||||
|
if len(value) > limit:
|
||||||
|
fail(f"secret {name} exceeds its configured limit")
|
||||||
|
if b"\x00" in value or b"\n" in value or b"\r" in value:
|
||||||
|
fail(f"secret {name} cannot be represented as a dotenv value")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_selectel(
|
||||||
|
config: Mapping[str, Any],
|
||||||
|
specs: Mapping[str, Mapping[str, Any]],
|
||||||
|
environ: Mapping[str, str],
|
||||||
|
client_factory: Callable[..., HTTPClient] = HTTPClient,
|
||||||
|
) -> dict[str, bytes]:
|
||||||
|
selectel = _object(config.get("selectel"), "selectel")
|
||||||
|
_only_keys(
|
||||||
|
selectel,
|
||||||
|
{
|
||||||
|
"account_id",
|
||||||
|
"username",
|
||||||
|
"project_name",
|
||||||
|
"region",
|
||||||
|
"interface",
|
||||||
|
"password_file",
|
||||||
|
"password_env",
|
||||||
|
"identity_url",
|
||||||
|
"secrets_url",
|
||||||
|
"ca_file",
|
||||||
|
},
|
||||||
|
"selectel",
|
||||||
|
)
|
||||||
|
http = _object(config.get("http", {}), "http")
|
||||||
|
_only_keys(http, {"timeout_seconds", "retries", "max_response_bytes"}, "http")
|
||||||
|
timeout = http.get("timeout_seconds", 10)
|
||||||
|
if isinstance(timeout, bool) or not isinstance(timeout, (int, float)) or not 0.1 <= timeout <= 60:
|
||||||
|
fail("http.timeout_seconds must be from 0.1 through 60")
|
||||||
|
retries = _bounded_int(http.get("retries", 3), "http.retries", 0, 8)
|
||||||
|
response_limit = _bounded_int(
|
||||||
|
http.get("max_response_bytes", MAX_HTTP_BYTES),
|
||||||
|
"http.max_response_bytes",
|
||||||
|
1024,
|
||||||
|
4 * MAX_HTTP_BYTES,
|
||||||
|
)
|
||||||
|
cafile = selectel.get("ca_file")
|
||||||
|
if cafile is not None and (not isinstance(cafile, str) or not cafile):
|
||||||
|
fail("selectel.ca_file must be a non-empty string")
|
||||||
|
client = client_factory(
|
||||||
|
timeout=float(timeout),
|
||||||
|
retries=retries,
|
||||||
|
max_response_bytes=response_limit,
|
||||||
|
cafile=cafile,
|
||||||
|
)
|
||||||
|
password = credential_value(selectel, environ)
|
||||||
|
token, catalog = project_token_and_catalog(client, selectel, password)
|
||||||
|
region = _required_string(selectel, "region", "selectel")
|
||||||
|
interface = selectel.get("interface", "public")
|
||||||
|
if interface not in {"public", "internal"}:
|
||||||
|
fail("selectel.interface must be public or internal")
|
||||||
|
override = selectel.get("secrets_url")
|
||||||
|
if override is not None and (not isinstance(override, str) or not override):
|
||||||
|
fail("selectel.secrets_url must be a non-empty string")
|
||||||
|
base_url = override.rstrip("/") if override else secrets_endpoint(catalog, region, interface)
|
||||||
|
|
||||||
|
values: dict[str, bytes] = {}
|
||||||
|
fetched: dict[tuple[str, int | None], bytes] = {}
|
||||||
|
for canonical, spec in specs.items():
|
||||||
|
if "literal" in spec:
|
||||||
|
values[canonical] = b""
|
||||||
|
continue
|
||||||
|
remote = _required_string(spec, "remote", f"secrets.{canonical}")
|
||||||
|
version = spec.get("version")
|
||||||
|
version_id: int | None = None
|
||||||
|
if version is not None:
|
||||||
|
version_id = _bounded_int(
|
||||||
|
version, f"secrets.{canonical}.version", 1, 2_147_483_647
|
||||||
|
)
|
||||||
|
cache_key = (remote, version_id)
|
||||||
|
if cache_key not in fetched:
|
||||||
|
path = f"/v1/{urllib.parse.quote(remote, safe='')}"
|
||||||
|
if version_id is not None:
|
||||||
|
path += f"/versions/{version_id}"
|
||||||
|
try:
|
||||||
|
result = client.request(
|
||||||
|
"GET",
|
||||||
|
base_url + path,
|
||||||
|
headers={"X-Auth-Token": token, "Accept": "application/json"},
|
||||||
|
expected=frozenset({200}),
|
||||||
|
)
|
||||||
|
document = parse_json_response(result, f"secret {canonical} response")
|
||||||
|
fetched[cache_key] = decode_secret(
|
||||||
|
document, canonical, MAX_SECRET_BYTES
|
||||||
|
)
|
||||||
|
except LoaderError as exc:
|
||||||
|
fail(f"cannot load {canonical}: {exc}")
|
||||||
|
limit = _bounded_int(
|
||||||
|
spec.get("max_bytes", MAX_SECRET_BYTES),
|
||||||
|
f"secrets.{canonical}.max_bytes",
|
||||||
|
1,
|
||||||
|
MAX_SECRET_BYTES,
|
||||||
|
)
|
||||||
|
value = fetched[cache_key]
|
||||||
|
if len(value) > limit:
|
||||||
|
fail(f"secret {canonical} exceeds its configured limit")
|
||||||
|
values[canonical] = value
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def parse_dotenv(path: Path, expected: set[str], max_bytes: int) -> dict[str, bytes]:
|
||||||
|
require_private_regular_file(path, "fallback dotenv")
|
||||||
|
raw = read_limited(path, max_bytes, "fallback dotenv")
|
||||||
|
try:
|
||||||
|
text = raw.decode("utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
fail("fallback dotenv is not valid UTF-8")
|
||||||
|
values: dict[str, bytes] = {}
|
||||||
|
for number, line in enumerate(text.splitlines(), 1):
|
||||||
|
if not line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
match = DOTENV_LINE_RE.fullmatch(line)
|
||||||
|
if not match:
|
||||||
|
fail(f"fallback dotenv has invalid syntax at line {number}")
|
||||||
|
name, encoded_value = match.groups()
|
||||||
|
if name not in expected:
|
||||||
|
fail(f"fallback dotenv contains undeclared key {name}")
|
||||||
|
if name in values:
|
||||||
|
fail(f"fallback dotenv contains duplicate key {name}")
|
||||||
|
if encoded_value.startswith('"'):
|
||||||
|
try:
|
||||||
|
decoded = json.loads(encoded_value)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
fail(f"fallback dotenv has invalid quoted value at line {number}")
|
||||||
|
if not isinstance(decoded, str):
|
||||||
|
fail(f"fallback dotenv has invalid quoted value at line {number}")
|
||||||
|
value = decoded.encode("utf-8")
|
||||||
|
elif encoded_value.startswith("'"):
|
||||||
|
if len(encoded_value) < 2 or not encoded_value.endswith("'"):
|
||||||
|
fail(f"fallback dotenv has invalid quoted value at line {number}")
|
||||||
|
value = encoded_value[1:-1].encode("utf-8")
|
||||||
|
else:
|
||||||
|
if any(character.isspace() for character in encoded_value) or any(
|
||||||
|
character in encoded_value for character in ("'", '"', "`", "$", "\\")
|
||||||
|
):
|
||||||
|
fail(f"fallback dotenv requires quoting at line {number}")
|
||||||
|
value = encoded_value.encode("utf-8")
|
||||||
|
if not value or b"\x00" in value or b"\n" in value or b"\r" in value:
|
||||||
|
fail(f"fallback dotenv has an empty or unsafe value for {name}")
|
||||||
|
values[name] = value
|
||||||
|
missing = sorted(expected - set(values))
|
||||||
|
if missing:
|
||||||
|
fail(f"fallback dotenv is missing declared keys: {', '.join(missing)}")
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
def validate_specs(config: Mapping[str, Any]) -> dict[str, dict[str, Any]]:
|
||||||
|
raw_specs = _object(config.get("secrets"), "secrets")
|
||||||
|
if not raw_specs:
|
||||||
|
fail("secrets must not be empty")
|
||||||
|
specs: dict[str, dict[str, Any]] = {}
|
||||||
|
for canonical, raw_spec in raw_specs.items():
|
||||||
|
if not isinstance(canonical, str) or not ENV_NAME_RE.fullmatch(canonical):
|
||||||
|
fail("every canonical secret name must be an uppercase environment name")
|
||||||
|
spec = _object(raw_spec, f"secrets.{canonical}")
|
||||||
|
_only_keys(
|
||||||
|
spec,
|
||||||
|
{"remote", "consumers", "max_bytes", "version", "literal"},
|
||||||
|
f"secrets.{canonical}",
|
||||||
|
)
|
||||||
|
has_remote = "remote" in spec
|
||||||
|
has_literal = "literal" in spec
|
||||||
|
if has_remote == has_literal:
|
||||||
|
fail(f"secrets.{canonical} must set exactly one of remote or literal")
|
||||||
|
if has_literal and spec["literal"] != "":
|
||||||
|
fail(f"secrets.{canonical}.literal may only be an empty string")
|
||||||
|
consumers = spec.get("consumers")
|
||||||
|
if not isinstance(consumers, list) or not consumers:
|
||||||
|
fail(f"secrets.{canonical}.consumers must be a non-empty array")
|
||||||
|
if len(consumers) != len(set(item for item in consumers if isinstance(item, str))):
|
||||||
|
fail(f"secrets.{canonical}.consumers contains duplicates or invalid values")
|
||||||
|
for consumer in consumers:
|
||||||
|
if not isinstance(consumer, str) or not SERVICE_NAME_RE.fullmatch(consumer):
|
||||||
|
fail(f"secrets.{canonical}.consumers contains an invalid service name")
|
||||||
|
specs[canonical] = spec
|
||||||
|
return specs
|
||||||
|
|
||||||
|
|
||||||
|
def dotenv_quote(value: bytes, name: str) -> str:
|
||||||
|
try:
|
||||||
|
text = value.decode("utf-8")
|
||||||
|
except UnicodeDecodeError:
|
||||||
|
fail(f"secret {name} is not valid UTF-8")
|
||||||
|
return json.dumps(text, ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
def materialize(runtime_dir: Path, specs: Mapping[str, Mapping[str, Any]], values: Mapping[str, bytes]) -> None:
|
||||||
|
try:
|
||||||
|
runtime_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||||
|
if runtime_dir.is_symlink():
|
||||||
|
fail("runtime directory must not be a symlink")
|
||||||
|
os.chmod(runtime_dir, 0o700)
|
||||||
|
except OSError as exc:
|
||||||
|
fail(f"cannot prepare runtime directory: {exc.strerror or exc.__class__.__name__}")
|
||||||
|
consumers = sorted(
|
||||||
|
{consumer for spec in specs.values() for consumer in spec["consumers"]}
|
||||||
|
)
|
||||||
|
staged: list[tuple[Path, Path]] = []
|
||||||
|
try:
|
||||||
|
for consumer in consumers:
|
||||||
|
lines = [
|
||||||
|
f"{name}={dotenv_quote(values[name], name)}\n"
|
||||||
|
for name, spec in sorted(specs.items())
|
||||||
|
if consumer in spec["consumers"]
|
||||||
|
]
|
||||||
|
descriptor, temporary = tempfile.mkstemp(
|
||||||
|
prefix=f".{consumer}.", suffix=".tmp", dir=runtime_dir
|
||||||
|
)
|
||||||
|
temporary_path = Path(temporary)
|
||||||
|
try:
|
||||||
|
os.chmod(temporary_path, 0o600)
|
||||||
|
stream = os.fdopen(descriptor, "w", encoding="utf-8", newline="\n")
|
||||||
|
descriptor = -1
|
||||||
|
with stream:
|
||||||
|
stream.writelines(lines)
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
except BaseException:
|
||||||
|
if descriptor >= 0:
|
||||||
|
os.close(descriptor)
|
||||||
|
raise
|
||||||
|
staged.append((temporary_path, runtime_dir / f"{consumer}.env"))
|
||||||
|
for temporary_path, destination in staged:
|
||||||
|
os.replace(temporary_path, destination)
|
||||||
|
value_paths: dict[str, Path] = {}
|
||||||
|
for name, value in sorted(values.items()):
|
||||||
|
descriptor, temporary = tempfile.mkstemp(
|
||||||
|
prefix=f".{name}.", suffix=".tmp", dir=runtime_dir
|
||||||
|
)
|
||||||
|
temporary_path = Path(temporary)
|
||||||
|
try:
|
||||||
|
# Compose implements local secrets as bind mounts. The protected
|
||||||
|
# 0700 parent prevents host users from traversing to this 0444
|
||||||
|
# file while allowing a non-root container UID to read its mount.
|
||||||
|
os.chmod(temporary_path, 0o444)
|
||||||
|
with os.fdopen(descriptor, "wb") as stream:
|
||||||
|
descriptor = -1
|
||||||
|
stream.write(value)
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
except BaseException:
|
||||||
|
if descriptor >= 0:
|
||||||
|
os.close(descriptor)
|
||||||
|
temporary_path.unlink(missing_ok=True)
|
||||||
|
raise
|
||||||
|
destination = runtime_dir / name
|
||||||
|
os.replace(temporary_path, destination)
|
||||||
|
value_paths[name] = destination
|
||||||
|
|
||||||
|
manifest_lines = [
|
||||||
|
f"{name}={path.resolve()}\n" for name, path in sorted(value_paths.items())
|
||||||
|
]
|
||||||
|
descriptor, temporary = tempfile.mkstemp(
|
||||||
|
prefix=".manifest.", suffix=".tmp", dir=runtime_dir
|
||||||
|
)
|
||||||
|
manifest_path = Path(temporary)
|
||||||
|
try:
|
||||||
|
os.chmod(manifest_path, 0o600)
|
||||||
|
with os.fdopen(descriptor, "w", encoding="utf-8", newline="\n") as stream:
|
||||||
|
descriptor = -1
|
||||||
|
stream.writelines(manifest_lines)
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
except BaseException:
|
||||||
|
if descriptor >= 0:
|
||||||
|
os.close(descriptor)
|
||||||
|
manifest_path.unlink(missing_ok=True)
|
||||||
|
raise
|
||||||
|
os.replace(manifest_path, runtime_dir / "manifest")
|
||||||
|
if hasattr(os, "O_DIRECTORY"):
|
||||||
|
directory_fd = os.open(runtime_dir, os.O_RDONLY | os.O_DIRECTORY)
|
||||||
|
try:
|
||||||
|
os.fsync(directory_fd)
|
||||||
|
finally:
|
||||||
|
os.close(directory_fd)
|
||||||
|
except OSError as exc:
|
||||||
|
fail(f"cannot atomically materialize service files: {exc.strerror or exc.__class__.__name__}")
|
||||||
|
finally:
|
||||||
|
for temporary_path, _ in staged:
|
||||||
|
try:
|
||||||
|
temporary_path.unlink(missing_ok=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def run(
|
||||||
|
config_path: Path,
|
||||||
|
*,
|
||||||
|
runtime_override: Path | None = None,
|
||||||
|
environ: Mapping[str, str] | None = None,
|
||||||
|
client_factory: Callable[..., HTTPClient] = HTTPClient,
|
||||||
|
) -> list[str]:
|
||||||
|
os.umask(0o077)
|
||||||
|
environment = os.environ if environ is None else environ
|
||||||
|
config = load_json(config_path)
|
||||||
|
_only_keys(config, {"version", "mode", "runtime_dir", "http", "selectel", "file", "secrets"}, "configuration")
|
||||||
|
if config.get("version") != 1:
|
||||||
|
fail("configuration.version must be 1")
|
||||||
|
mode = config.get("mode")
|
||||||
|
if mode not in {"selectel", "file"}:
|
||||||
|
fail("configuration.mode must explicitly be selectel or file")
|
||||||
|
specs = validate_specs(config)
|
||||||
|
if runtime_override is None:
|
||||||
|
configured_runtime = config.get("runtime_dir")
|
||||||
|
if not isinstance(configured_runtime, str) or not configured_runtime:
|
||||||
|
fail("configuration.runtime_dir must be a non-empty string")
|
||||||
|
runtime_dir = Path(configured_runtime)
|
||||||
|
else:
|
||||||
|
runtime_dir = runtime_override
|
||||||
|
if not runtime_dir.is_absolute():
|
||||||
|
fail("runtime directory must be an absolute path")
|
||||||
|
|
||||||
|
if mode == "selectel":
|
||||||
|
if "file" in config:
|
||||||
|
fail("file settings are forbidden in selectel mode")
|
||||||
|
values = fetch_selectel(config, specs, environment, client_factory)
|
||||||
|
else:
|
||||||
|
if "selectel" in config or "http" in config:
|
||||||
|
fail("selectel and http settings are forbidden in file mode")
|
||||||
|
file_config = _object(config.get("file"), "file")
|
||||||
|
_only_keys(file_config, {"path", "max_bytes"}, "file")
|
||||||
|
source = Path(_required_string(file_config, "path", "file"))
|
||||||
|
if not source.is_absolute():
|
||||||
|
fail("file.path must be absolute")
|
||||||
|
max_bytes = _bounded_int(
|
||||||
|
file_config.get("max_bytes", MAX_CONFIG_BYTES),
|
||||||
|
"file.max_bytes",
|
||||||
|
1,
|
||||||
|
4 * MAX_CONFIG_BYTES,
|
||||||
|
)
|
||||||
|
expected = {name for name, spec in specs.items() if "literal" not in spec}
|
||||||
|
values = parse_dotenv(source, expected, max_bytes)
|
||||||
|
values.update(
|
||||||
|
{name: b"" for name, spec in specs.items() if "literal" in spec}
|
||||||
|
)
|
||||||
|
for canonical, value in values.items():
|
||||||
|
limit = _bounded_int(
|
||||||
|
specs[canonical].get("max_bytes", MAX_SECRET_BYTES),
|
||||||
|
f"secrets.{canonical}.max_bytes",
|
||||||
|
1,
|
||||||
|
MAX_SECRET_BYTES,
|
||||||
|
)
|
||||||
|
if len(value) > limit:
|
||||||
|
fail(f"secret {canonical} exceeds its configured limit")
|
||||||
|
|
||||||
|
materialize(runtime_dir, specs, values)
|
||||||
|
return sorted({consumer for spec in specs.values() for consumer in spec["consumers"]})
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Materialize per-service secret dotenv files")
|
||||||
|
parser.add_argument("--config", required=True, type=Path)
|
||||||
|
parser.add_argument("--runtime-dir", type=Path)
|
||||||
|
arguments = parser.parse_args(argv)
|
||||||
|
try:
|
||||||
|
consumers = run(arguments.config, runtime_override=arguments.runtime_dir)
|
||||||
|
except LoaderError as exc:
|
||||||
|
print(f"secrets-loader: {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
print(f"secrets-loader: materialized {len(consumers)} service file(s)", file=sys.stderr)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -1,28 +1,59 @@
|
|||||||
x-no-sms-secrets: &no-sms-secrets
|
x-api-secret-environment: &api-secret-environment
|
||||||
SMS_DATABASE_URL: ""
|
HAN_SECRET_VARS: >-
|
||||||
SMS_SERVICE_TOKEN: ""
|
DATABASE_URL REDIS_URL REDIS_REALTIME_URL MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
KEYCLOAK_SMS_SERVICE_TOKEN: ""
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN BITRIX_API_INBOX_TOKEN
|
||||||
IDGTL_SMS_API_KEY: ""
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN SELECTEL_S3_ACCESS_KEY SELECTEL_S3_SECRET_KEY
|
||||||
IDGTL_SMS_CALLBACK_USERNAME: ""
|
CURSOR_HMAC_SECRET
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD: ""
|
DATABASE_URL_FILE: /run/secrets/api_database_url
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ""
|
REDIS_URL_FILE: /run/secrets/api_redis_url
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ""
|
REDIS_REALTIME_URL_FILE: /run/secrets/api_redis_realtime_url
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ""
|
MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token
|
||||||
|
BITRIX_LOCAL_APP_INTERNAL_TOKEN_FILE: /run/secrets/bitrix_local_app_internal_token
|
||||||
|
BITRIX_API_INBOX_TOKEN_FILE: /run/secrets/bitrix_api_inbox_token
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token
|
||||||
|
SELECTEL_S3_ACCESS_KEY_FILE: /run/secrets/selectel_s3_access_key
|
||||||
|
SELECTEL_S3_SECRET_KEY_FILE: /run/secrets/selectel_s3_secret_key
|
||||||
|
CURSOR_HMAC_SECRET_FILE: /run/secrets/cursor_hmac_secret
|
||||||
|
|
||||||
|
x-api-secrets: &api-secrets
|
||||||
|
- api_database_url
|
||||||
|
- api_redis_url
|
||||||
|
- api_redis_realtime_url
|
||||||
|
- message_safety_service_token
|
||||||
|
- bitrix_local_app_internal_token
|
||||||
|
- bitrix_api_inbox_token
|
||||||
|
- keycloak_settings_bridge_token
|
||||||
|
- selectel_s3_access_key
|
||||||
|
- selectel_s3_secret_key
|
||||||
|
- cursor_hmac_secret
|
||||||
|
|
||||||
x-api-runtime: &api-runtime
|
x-api-runtime: &api-runtime
|
||||||
build:
|
build:
|
||||||
context: ../../api-backend
|
context: ../../api-backend
|
||||||
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
image: ${API_BACKEND_IMAGE:-han-chat-api-backend:local}
|
||||||
env_file:
|
|
||||||
- path: ../../.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
<<: *no-sms-secrets
|
<<: *api-secret-environment
|
||||||
NOTIFICATIONS_TOKEN_PRODUCER_TEST: ${NOTIFICATIONS_TOKEN_PRODUCER_TEST:?NOTIFICATIONS_TOKEN_PRODUCER_TEST is required}
|
|
||||||
APP_ENV: ${APP_ENV:-production-like}
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
||||||
|
KEYCLOAK_PUBLIC_URL: ${KEYCLOAK_PUBLIC_URL}
|
||||||
|
KEYCLOAK_INTERNAL_URL: ${KEYCLOAK_INTERNAL_URL:-http://keycloak:8080/auth}
|
||||||
|
KEYCLOAK_REALM: ${KEYCLOAK_REALM:-han-chat}
|
||||||
|
KEYCLOAK_AUDIENCE: ${KEYCLOAK_AUDIENCE:-han-chat-api}
|
||||||
|
MESSAGE_SAFETY_URL: ${MESSAGE_SAFETY_URL:-http://message-safety:8080}
|
||||||
|
MESSAGE_SAFETY_POST_TIMEOUT_SEC: ${MESSAGE_SAFETY_POST_TIMEOUT_SEC:-5}
|
||||||
|
MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC: ${MESSAGE_SAFETY_TASK_POLL_INTERVAL_SEC:-2}
|
||||||
|
MESSAGE_SAFETY_TASK_POLL_MAX_SEC: ${MESSAGE_SAFETY_TASK_POLL_MAX_SEC:-300}
|
||||||
|
BITRIX_LOCAL_APP_BASE_URL: ${BITRIX_LOCAL_APP_BASE_URL:-http://bitrix-local-app:8080}
|
||||||
|
BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC: ${BITRIX_LOCAL_APP_HTTP_TIMEOUT_SEC:-20}
|
||||||
|
SELECTEL_S3_ENDPOINT_URL: ${SELECTEL_S3_ENDPOINT_URL}
|
||||||
|
SELECTEL_S3_BUCKET_DOCUMENTS: ${SELECTEL_S3_BUCKET_DOCUMENTS}
|
||||||
|
SELECTEL_S3_BUCKET_ATTACHMENTS: ${SELECTEL_S3_BUCKET_ATTACHMENTS}
|
||||||
|
SELECTEL_S3_BUCKET_QUARANTINE: ${SELECTEL_S3_BUCKET_QUARANTINE}
|
||||||
|
TRUSTED_PROXY_CIDRS: ${TRUSTED_PROXY_CIDRS:-127.0.0.1/32}
|
||||||
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
||||||
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_API:-api-backend}
|
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_API:-api-backend}
|
||||||
|
secrets: *api-secrets
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
source: ${PG_CA_HOST_PATH}
|
source: ${PG_CA_HOST_PATH}
|
||||||
@@ -30,32 +61,48 @@ x-api-runtime: &api-runtime
|
|||||||
read_only: true
|
read_only: true
|
||||||
networks: [backend, observability, egress]
|
networks: [backend, observability, egress]
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
|
|
||||||
x-sms-runtime: &sms-runtime
|
x-sms-api-environment: &sms-api-environment
|
||||||
build:
|
HAN_SECRET_VARS: >-
|
||||||
context: ../../sms-service
|
SMS_DATABASE_URL SMS_SERVICE_TOKEN IDGTL_SMS_CALLBACK_USERNAME
|
||||||
image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local}
|
IDGTL_SMS_CALLBACK_PASSWORD
|
||||||
environment:
|
SMS_DATABASE_URL_FILE: /run/secrets/sms_database_url
|
||||||
SMS_DATABASE_URL: ${SMS_DATABASE_URL}
|
SMS_SERVICE_TOKEN_FILE: /run/secrets/sms_service_token
|
||||||
SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN}
|
IDGTL_SMS_CALLBACK_USERNAME_FILE: /run/secrets/idgtl_sms_callback_username
|
||||||
|
IDGTL_SMS_CALLBACK_PASSWORD_FILE: /run/secrets/idgtl_sms_callback_password
|
||||||
IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru}
|
IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru}
|
||||||
IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL}
|
IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL}
|
||||||
IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME}
|
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD}
|
|
||||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||||
APP_ENV: ${APP_ENV:-production-like}
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
||||||
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
||||||
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_API:-sms-service}
|
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_API:-sms-service}
|
||||||
|
|
||||||
|
x-sms-api-secrets: &sms-api-secrets
|
||||||
|
- sms_database_url
|
||||||
|
- sms_service_token
|
||||||
|
- idgtl_sms_callback_username
|
||||||
|
- idgtl_sms_callback_password
|
||||||
|
|
||||||
|
x-sms-runtime: &sms-runtime
|
||||||
|
build:
|
||||||
|
context: ../../sms-service
|
||||||
|
image: ${SMS_SERVICE_IMAGE:-han-chat-sms-service:local}
|
||||||
|
environment: *sms-api-environment
|
||||||
|
secrets: *sms-api-secrets
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
source: ${PG_CA_HOST_PATH}
|
source: ${PG_CA_HOST_PATH}
|
||||||
target: /run/secrets/pg-ca.pem
|
target: /run/secrets/pg-ca.pem
|
||||||
read_only: true
|
read_only: true
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
@@ -80,21 +127,31 @@ services:
|
|||||||
- /tmp:size=8m,mode=1777
|
- /tmp:size=8m,mode=1777
|
||||||
cap_drop: ["ALL"]
|
cap_drop: ["ALL"]
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
|
|
||||||
keycloak:
|
keycloak:
|
||||||
build:
|
build:
|
||||||
context: ../../keycloak
|
context: ../../keycloak
|
||||||
image: ${KEYCLOAK_IMAGE:-han-chat-keycloak:local}
|
image: ${KEYCLOAK_IMAGE:-han-chat-keycloak:local}
|
||||||
env_file:
|
|
||||||
- path: ../../.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
<<: *no-sms-secrets
|
HAN_SECRET_VARS: >-
|
||||||
|
KC_DB_PASSWORD KC_BOOTSTRAP_ADMIN_PASSWORD
|
||||||
|
KEYCLOAK_OTP_MOCK_CODE
|
||||||
|
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY
|
||||||
|
KEYCLOAK_OTP_HMAC_KEY KEYCLOAK_SETTINGS_BRIDGE_TOKEN
|
||||||
|
KEYCLOAK_SMS_SERVICE_TOKEN
|
||||||
|
KC_DB_PASSWORD_FILE: /run/secrets/keycloak_db_password
|
||||||
|
KC_BOOTSTRAP_ADMIN_PASSWORD_FILE: /run/secrets/keycloak_admin_password
|
||||||
|
KEYCLOAK_OTP_MOCK_CODE_FILE: /run/secrets/keycloak_otp_mock_code
|
||||||
|
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: /run/secrets/keycloak_yandex_captcha_server_key
|
||||||
|
KEYCLOAK_OTP_HMAC_KEY_FILE: /run/secrets/keycloak_otp_hmac_key
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token
|
||||||
|
KEYCLOAK_SMS_SERVICE_TOKEN_FILE: /run/secrets/keycloak_sms_service_token
|
||||||
KC_DB: postgres
|
KC_DB: postgres
|
||||||
KC_DB_URL: ${KEYCLOAK_DB_URL}
|
KC_DB_URL: ${KEYCLOAK_DB_URL}
|
||||||
KC_DB_SCHEMA: ${KEYCLOAK_DB_SCHEMA:-keycloak}
|
|
||||||
KC_DB_USERNAME: ${KEYCLOAK_DB_USERNAME}
|
KC_DB_USERNAME: ${KEYCLOAK_DB_USERNAME}
|
||||||
KC_DB_PASSWORD: ${KEYCLOAK_DB_PASSWORD}
|
KC_DB_SCHEMA: ${KEYCLOAK_DB_SCHEMA:-keycloak}
|
||||||
KC_PROXY_HEADERS: xforwarded
|
KC_PROXY_HEADERS: xforwarded
|
||||||
KC_HTTP_ENABLED: "true"
|
KC_HTTP_ENABLED: "true"
|
||||||
KC_HTTP_RELATIVE_PATH: /auth
|
KC_HTTP_RELATIVE_PATH: /auth
|
||||||
@@ -102,18 +159,20 @@ services:
|
|||||||
KC_METRICS_ENABLED: "true"
|
KC_METRICS_ENABLED: "true"
|
||||||
KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL}
|
KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL}
|
||||||
KC_BOOTSTRAP_ADMIN_USERNAME: ${KEYCLOAK_ADMIN}
|
KC_BOOTSTRAP_ADMIN_USERNAME: ${KEYCLOAK_ADMIN}
|
||||||
KC_BOOTSTRAP_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
|
|
||||||
KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-false}
|
KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-false}
|
||||||
KEYCLOAK_OTP_MOCK_CODE: ${KEYCLOAK_OTP_MOCK_CODE:-}
|
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false}
|
KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false}
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}
|
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY:-}
|
|
||||||
KEYCLOAK_OTP_HMAC_KEY: ${KEYCLOAK_OTP_HMAC_KEY:?KEYCLOAK_OTP_HMAC_KEY is required}
|
|
||||||
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300}
|
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300}
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp}
|
KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp}
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN:?KEYCLOAK_SETTINGS_BRIDGE_TOKEN is required}
|
|
||||||
KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080}
|
KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080}
|
||||||
KEYCLOAK_SMS_SERVICE_TOKEN: ${KEYCLOAK_SMS_SERVICE_TOKEN:?KEYCLOAK_SMS_SERVICE_TOKEN is required}
|
secrets:
|
||||||
|
- keycloak_db_password
|
||||||
|
- keycloak_admin_password
|
||||||
|
- keycloak_otp_mock_code
|
||||||
|
- keycloak_yandex_captcha_server_key
|
||||||
|
- keycloak_otp_hmac_key
|
||||||
|
- keycloak_settings_bridge_token
|
||||||
|
- keycloak_sms_service_token
|
||||||
command: ["start", "--optimized", "--import-realm"]
|
command: ["start", "--optimized", "--import-realm"]
|
||||||
expose: ["8080", "9000"]
|
expose: ["8080", "9000"]
|
||||||
volumes:
|
volumes:
|
||||||
@@ -130,6 +189,8 @@ services:
|
|||||||
start_period: 60s
|
start_period: 60s
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
@@ -148,22 +209,21 @@ services:
|
|||||||
|
|
||||||
sms-worker:
|
sms-worker:
|
||||||
<<: *sms-runtime
|
<<: *sms-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-sms-worker"]
|
command: ["han-sms-worker"]
|
||||||
environment:
|
environment:
|
||||||
SMS_DATABASE_URL: ${SMS_DATABASE_URL}
|
<<: *sms-api-environment
|
||||||
SMS_SERVICE_TOKEN: ${SMS_SERVICE_TOKEN}
|
HAN_SECRET_VARS: >-
|
||||||
IDGTL_SMS_BASE_URL: ${IDGTL_SMS_BASE_URL:-https://direct.i-dgtl.ru}
|
SMS_DATABASE_URL SMS_SERVICE_TOKEN IDGTL_SMS_API_KEY
|
||||||
IDGTL_SMS_API_KEY: ${IDGTL_SMS_API_KEY:?IDGTL_SMS_API_KEY is required}
|
IDGTL_SMS_CALLBACK_USERNAME IDGTL_SMS_CALLBACK_PASSWORD
|
||||||
IDGTL_SMS_CALLBACK_PUBLIC_URL: ${IDGTL_SMS_CALLBACK_PUBLIC_URL}
|
IDGTL_SMS_API_KEY_FILE: /run/secrets/idgtl_sms_api_key
|
||||||
IDGTL_SMS_CALLBACK_USERNAME: ${IDGTL_SMS_CALLBACK_USERNAME}
|
|
||||||
IDGTL_SMS_CALLBACK_PASSWORD: ${IDGTL_SMS_CALLBACK_PASSWORD}
|
|
||||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
|
||||||
APP_ENV: ${APP_ENV:-production-like}
|
|
||||||
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
|
||||||
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-http://otel-collector:4317}
|
|
||||||
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_WORKER:-sms-worker}
|
OTEL_SERVICE_NAME: ${OTEL_SERVICE_NAME_SMS_WORKER:-sms-worker}
|
||||||
SMS_METRICS_PORT: ${SMS_METRICS_PORT:-9464}
|
SMS_METRICS_PORT: ${SMS_METRICS_PORT:-9464}
|
||||||
|
secrets:
|
||||||
|
- sms_database_url
|
||||||
|
- sms_service_token
|
||||||
|
- idgtl_sms_api_key
|
||||||
|
- idgtl_sms_callback_username
|
||||||
|
- idgtl_sms_callback_password
|
||||||
expose: ["9464"]
|
expose: ["9464"]
|
||||||
networks: [backend, observability, egress]
|
networks: [backend, observability, egress]
|
||||||
depends_on:
|
depends_on:
|
||||||
@@ -180,17 +240,17 @@ services:
|
|||||||
build:
|
build:
|
||||||
context: ../../message-safety
|
context: ../../message-safety
|
||||||
image: ${MESSAGE_SAFETY_IMAGE:-han-chat-message-safety:local}
|
image: ${MESSAGE_SAFETY_IMAGE:-han-chat-message-safety:local}
|
||||||
env_file:
|
|
||||||
- path: ../../.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
<<: *no-sms-secrets
|
HAN_SECRET_VARS: MESSAGE_SAFETY_REDIS_URL MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
|
MESSAGE_SAFETY_REDIS_URL_FILE: /run/secrets/message_safety_redis_url
|
||||||
|
MESSAGE_SAFETY_SERVICE_TOKEN_FILE: /run/secrets/message_safety_service_token
|
||||||
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
|
MESSAGE_SAFETY_RULES_VERSION: ${MESSAGE_SAFETY_RULES_VERSION:-2026-01-01}
|
||||||
|
MESSAGE_SAFETY_TASK_TTL_SEC: ${MESSAGE_SAFETY_TASK_TTL_SEC:-900}
|
||||||
|
secrets:
|
||||||
|
- message_safety_redis_url
|
||||||
|
- message_safety_service_token
|
||||||
expose: ["8080"]
|
expose: ["8080"]
|
||||||
volumes:
|
|
||||||
- type: bind
|
|
||||||
source: ${PG_CA_HOST_PATH}
|
|
||||||
target: /run/secrets/pg-ca.pem
|
|
||||||
read_only: true
|
|
||||||
networks: [backend, observability, egress]
|
networks: [backend, observability, egress]
|
||||||
depends_on:
|
depends_on:
|
||||||
redis: {condition: service_healthy}
|
redis: {condition: service_healthy}
|
||||||
@@ -202,6 +262,8 @@ services:
|
|||||||
start_period: 30s
|
start_period: 30s
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
@@ -223,7 +285,6 @@ services:
|
|||||||
|
|
||||||
delivery-worker:
|
delivery-worker:
|
||||||
<<: *api-runtime
|
<<: *api-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-delivery-worker"]
|
command: ["han-delivery-worker"]
|
||||||
depends_on:
|
depends_on:
|
||||||
api-backend: {condition: service_healthy}
|
api-backend: {condition: service_healthy}
|
||||||
@@ -238,7 +299,6 @@ services:
|
|||||||
|
|
||||||
safety-recovery-worker:
|
safety-recovery-worker:
|
||||||
<<: *api-runtime
|
<<: *api-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-safety-worker"]
|
command: ["han-safety-worker"]
|
||||||
depends_on:
|
depends_on:
|
||||||
api-backend: {condition: service_healthy}
|
api-backend: {condition: service_healthy}
|
||||||
@@ -253,7 +313,6 @@ services:
|
|||||||
|
|
||||||
cleanup-worker:
|
cleanup-worker:
|
||||||
<<: *api-runtime
|
<<: *api-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-cleanup-worker"]
|
command: ["han-cleanup-worker"]
|
||||||
depends_on:
|
depends_on:
|
||||||
api-backend: {condition: service_healthy}
|
api-backend: {condition: service_healthy}
|
||||||
@@ -267,7 +326,6 @@ services:
|
|||||||
|
|
||||||
notification-expire-worker:
|
notification-expire-worker:
|
||||||
<<: *api-runtime
|
<<: *api-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-notification-expire-worker"]
|
command: ["han-notification-expire-worker"]
|
||||||
depends_on:
|
depends_on:
|
||||||
api-backend: {condition: service_healthy}
|
api-backend: {condition: service_healthy}
|
||||||
@@ -281,7 +339,6 @@ services:
|
|||||||
|
|
||||||
notification-draft-cleanup-worker:
|
notification-draft-cleanup-worker:
|
||||||
<<: *api-runtime
|
<<: *api-runtime
|
||||||
entrypoint: []
|
|
||||||
command: ["han-notification-draft-cleanup-worker"]
|
command: ["han-notification-draft-cleanup-worker"]
|
||||||
depends_on:
|
depends_on:
|
||||||
api-backend: {condition: service_healthy}
|
api-backend: {condition: service_healthy}
|
||||||
@@ -298,11 +355,33 @@ services:
|
|||||||
build:
|
build:
|
||||||
context: ../../bitrix-local-app
|
context: ../../bitrix-local-app
|
||||||
image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local}
|
image: ${BITRIX_LOCAL_APP_IMAGE:-han-chat-bitrix-local-app:local}
|
||||||
env_file:
|
|
||||||
- path: ../../.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
<<: *no-sms-secrets
|
HAN_SECRET_VARS: >-
|
||||||
|
BITRIX_DATABASE_URL BITRIX_CLIENT_SECRET
|
||||||
|
BITRIX_APPLICATION_TOKEN BITRIX_INTERNAL_API_TOKEN
|
||||||
|
BITRIX_API_FORWARD_TOKEN BITRIX_TOKEN_ENCRYPTION_KEY
|
||||||
|
BITRIX_DATABASE_URL_FILE: /run/secrets/bitrix_database_url
|
||||||
|
BITRIX_CLIENT_SECRET_FILE: /run/secrets/bitrix_client_secret
|
||||||
|
BITRIX_APPLICATION_TOKEN_FILE: /run/secrets/bitrix_application_token
|
||||||
|
BITRIX_INTERNAL_API_TOKEN_FILE: /run/secrets/bitrix_internal_api_token
|
||||||
|
BITRIX_API_FORWARD_TOKEN_FILE: /run/secrets/bitrix_api_forward_token
|
||||||
|
BITRIX_TOKEN_ENCRYPTION_KEY_FILE: /run/secrets/bitrix_token_encryption_key
|
||||||
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
|
BITRIX_CLIENT_ID: ${BITRIX_CLIENT_ID}
|
||||||
|
BITRIX_CONNECTOR_ID: ${BITRIX_CONNECTOR_ID:-han_mobile_app}
|
||||||
|
BITRIX_CONNECTOR_NAME: ${BITRIX_CONNECTOR_NAME:-HAN Mobile App}
|
||||||
|
BITRIX_OPEN_LINE_ID: ${BITRIX_OPEN_LINE_ID}
|
||||||
|
BITRIX_EXPECTED_DOMAIN: ${BITRIX_EXPECTED_DOMAIN}
|
||||||
|
BITRIX_PUBLIC_BASE_URL: ${BITRIX_PUBLIC_BASE_URL}
|
||||||
|
BITRIX_API_FORWARD_URL: ${BITRIX_API_FORWARD_URL:-http://api-backend:8000/internal/openlines/v1/inbox}
|
||||||
|
BITRIX_HTTP_TIMEOUT_SEC: ${BITRIX_HTTP_TIMEOUT_SEC:-10}
|
||||||
|
secrets:
|
||||||
|
- bitrix_database_url
|
||||||
|
- bitrix_client_secret
|
||||||
|
- bitrix_application_token
|
||||||
|
- bitrix_internal_api_token
|
||||||
|
- bitrix_api_forward_token
|
||||||
|
- bitrix_token_encryption_key
|
||||||
expose: ["8080"]
|
expose: ["8080"]
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
@@ -320,6 +399,8 @@ services:
|
|||||||
start_period: 30s
|
start_period: 30s
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
@@ -328,11 +409,15 @@ services:
|
|||||||
build:
|
build:
|
||||||
context: ../../bitrix-sync
|
context: ../../bitrix-sync
|
||||||
image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local}
|
image: ${BITRIX_SYNC_IMAGE:-han-chat-bitrix-sync:local}
|
||||||
env_file:
|
|
||||||
- path: ../../.env
|
|
||||||
required: false
|
|
||||||
environment:
|
environment:
|
||||||
<<: *no-sms-secrets
|
HAN_SECRET_VARS: BITRIX_SYNC_DATABASE_URL BITRIX_SYNC_SERVICE_TOKEN
|
||||||
|
BITRIX_SYNC_DATABASE_URL_FILE: /run/secrets/bitrix_sync_database_url
|
||||||
|
BITRIX_SYNC_SERVICE_TOKEN_FILE: /run/secrets/bitrix_sync_service_token
|
||||||
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
|
BITRIX_SYNC_ENABLED: ${BITRIX_SYNC_ENABLED:-false}
|
||||||
|
secrets:
|
||||||
|
- bitrix_sync_database_url
|
||||||
|
- bitrix_sync_service_token
|
||||||
expose: ["8080"]
|
expose: ["8080"]
|
||||||
volumes:
|
volumes:
|
||||||
- type: bind
|
- type: bind
|
||||||
@@ -353,3 +438,65 @@ services:
|
|||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
api_database_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL
|
||||||
|
api_redis_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_URL
|
||||||
|
api_redis_realtime_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_REALTIME_URL
|
||||||
|
message_safety_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_SERVICE_TOKEN
|
||||||
|
bitrix_local_app_internal_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_LOCAL_APP_INTERNAL_TOKEN
|
||||||
|
bitrix_api_inbox_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_INBOX_TOKEN
|
||||||
|
keycloak_settings_bridge_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN
|
||||||
|
selectel_s3_access_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_ACCESS_KEY
|
||||||
|
selectel_s3_secret_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SELECTEL_S3_SECRET_KEY
|
||||||
|
cursor_hmac_secret:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/CURSOR_HMAC_SECRET
|
||||||
|
sms_database_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SMS_DATABASE_URL
|
||||||
|
sms_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/SMS_SERVICE_TOKEN
|
||||||
|
idgtl_sms_api_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_API_KEY
|
||||||
|
idgtl_sms_callback_username:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_CALLBACK_USERNAME
|
||||||
|
idgtl_sms_callback_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/IDGTL_SMS_CALLBACK_PASSWORD
|
||||||
|
message_safety_redis_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/MESSAGE_SAFETY_REDIS_URL
|
||||||
|
bitrix_database_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_DATABASE_URL
|
||||||
|
bitrix_client_secret:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_CLIENT_SECRET
|
||||||
|
bitrix_application_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_APPLICATION_TOKEN
|
||||||
|
bitrix_internal_api_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_INTERNAL_API_TOKEN
|
||||||
|
bitrix_api_forward_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_API_FORWARD_TOKEN
|
||||||
|
bitrix_token_encryption_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_TOKEN_ENCRYPTION_KEY
|
||||||
|
bitrix_sync_database_url:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_SYNC_DATABASE_URL
|
||||||
|
bitrix_sync_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/BITRIX_SYNC_SERVICE_TOKEN
|
||||||
|
keycloak_db_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_DB_PASSWORD
|
||||||
|
keycloak_admin_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_ADMIN_PASSWORD
|
||||||
|
keycloak_otp_mock_code:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_MOCK_CODE
|
||||||
|
keycloak_yandex_captcha_server_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY
|
||||||
|
keycloak_otp_hmac_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_HMAC_KEY
|
||||||
|
keycloak_sms_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SMS_SERVICE_TOKEN
|
||||||
|
|||||||
@@ -21,7 +21,8 @@ RUN /opt/keycloak/bin/kc.sh build
|
|||||||
FROM quay.io/keycloak/keycloak:26.1.4
|
FROM quay.io/keycloak/keycloak:26.1.4
|
||||||
COPY --from=keycloak-build --chown=keycloak:keycloak /opt/keycloak/ /opt/keycloak/
|
COPY --from=keycloak-build --chown=keycloak:keycloak /opt/keycloak/ /opt/keycloak/
|
||||||
COPY --chown=keycloak:keycloak realm/han-chat-realm.json /opt/keycloak/data/import/han-chat-realm.json
|
COPY --chown=keycloak:keycloak realm/han-chat-realm.json /opt/keycloak/data/import/han-chat-realm.json
|
||||||
|
COPY --chown=keycloak:keycloak --chmod=0555 container-entrypoint.sh /opt/keycloak/bin/han-container-entrypoint
|
||||||
USER 1000
|
USER 1000
|
||||||
EXPOSE 8080 9000
|
EXPOSE 8080 9000
|
||||||
ENTRYPOINT ["/opt/keycloak/bin/kc.sh"]
|
ENTRYPOINT ["/opt/keycloak/bin/han-container-entrypoint"]
|
||||||
CMD ["start", "--optimized", "--import-realm"]
|
CMD ["start", "--optimized", "--import-realm"]
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec /opt/keycloak/bin/kc.sh "$@"
|
||||||
@@ -6,8 +6,20 @@ services:
|
|||||||
image: han-chat/keycloak:26.1.4-otp-1.0.0
|
image: han-chat/keycloak:26.1.4-otp-1.0.0
|
||||||
command: ["start", "--optimized", "--import-realm"]
|
command: ["start", "--optimized", "--import-realm"]
|
||||||
environment:
|
environment:
|
||||||
|
HAN_SECRET_VARS: >-
|
||||||
|
KC_DB_PASSWORD KC_BOOTSTRAP_ADMIN_PASSWORD KEYCLOAK_OTP_MOCK_CODE
|
||||||
|
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY KEYCLOAK_OTP_HMAC_KEY
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN KEYCLOAK_SMS_SERVICE_TOKEN
|
||||||
|
KC_DB_PASSWORD_FILE: /run/secrets/keycloak_db_password
|
||||||
|
KC_BOOTSTRAP_ADMIN_PASSWORD_FILE: /run/secrets/keycloak_admin_password
|
||||||
|
KEYCLOAK_OTP_MOCK_CODE_FILE: /run/secrets/keycloak_otp_mock_code
|
||||||
|
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: /run/secrets/keycloak_yandex_captcha_server_key
|
||||||
|
KEYCLOAK_OTP_HMAC_KEY_FILE: /run/secrets/keycloak_otp_hmac_key
|
||||||
|
KEYCLOAK_SETTINGS_BRIDGE_TOKEN_FILE: /run/secrets/keycloak_settings_bridge_token
|
||||||
|
KEYCLOAK_SMS_SERVICE_TOKEN_FILE: /run/secrets/keycloak_sms_service_token
|
||||||
KC_DB: postgres
|
KC_DB: postgres
|
||||||
KC_DB_URL: ${KEYCLOAK_DB_URL:?KEYCLOAK_DB_URL is required}
|
KC_DB_URL: ${KEYCLOAK_DB_URL:?KEYCLOAK_DB_URL is required}
|
||||||
|
KC_DB_USERNAME: ${KEYCLOAK_DB_USERNAME:?KEYCLOAK_DB_USERNAME is required}
|
||||||
KC_DB_URL_PROPERTIES: ${KC_DB_URL_PROPERTIES:-currentSchema=keycloak}
|
KC_DB_URL_PROPERTIES: ${KC_DB_URL_PROPERTIES:-currentSchema=keycloak}
|
||||||
KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL:-https://tohin.ru/auth}
|
KC_HOSTNAME: ${KEYCLOAK_PUBLIC_URL:-https://tohin.ru/auth}
|
||||||
KC_HOSTNAME_STRICT: "true"
|
KC_HOSTNAME_STRICT: "true"
|
||||||
@@ -19,21 +31,23 @@ services:
|
|||||||
KC_METRICS_ENABLED: "true"
|
KC_METRICS_ENABLED: "true"
|
||||||
KC_HTTP_MANAGEMENT_PORT: "9000"
|
KC_HTTP_MANAGEMENT_PORT: "9000"
|
||||||
KC_BOOTSTRAP_ADMIN_USERNAME: ${KC_BOOTSTRAP_ADMIN_USERNAME:?bootstrap admin username is required}
|
KC_BOOTSTRAP_ADMIN_USERNAME: ${KC_BOOTSTRAP_ADMIN_USERNAME:?bootstrap admin username is required}
|
||||||
KC_BOOTSTRAP_ADMIN_PASSWORD: ${KC_BOOTSTRAP_ADMIN_PASSWORD:?bootstrap admin password is required}
|
|
||||||
KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-true}
|
KEYCLOAK_OTP_MOCK_ENABLED: ${KEYCLOAK_OTP_MOCK_ENABLED:-true}
|
||||||
KEYCLOAK_OTP_MOCK_CODE: ${KEYCLOAK_OTP_MOCK_CODE:-}
|
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false}
|
KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ${KEYCLOAK_YANDEX_CAPTCHA_ENABLED:-false}
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}
|
KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}
|
||||||
KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ${KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY:-}
|
|
||||||
KEYCLOAK_OTP_HMAC_KEY: ${KEYCLOAK_OTP_HMAC_KEY:?OTP HMAC key is required}
|
|
||||||
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300}
|
KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC: ${KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC:-300}
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp}
|
KEYCLOAK_SETTINGS_BRIDGE_URL: ${KEYCLOAK_SETTINGS_BRIDGE_URL:-http://api-backend:8000/internal/settings/v1/otp}
|
||||||
KEYCLOAK_SETTINGS_BRIDGE_TOKEN: ${KEYCLOAK_SETTINGS_BRIDGE_TOKEN:?settings bridge token is required}
|
|
||||||
KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080}
|
KEYCLOAK_SMS_SERVICE_URL: ${KEYCLOAK_SMS_SERVICE_URL:-http://sms-service:8080}
|
||||||
KEYCLOAK_SMS_SERVICE_TOKEN: ${KEYCLOAK_SMS_SERVICE_TOKEN:-}
|
|
||||||
KC_LOG_CONSOLE_OUTPUT: json
|
KC_LOG_CONSOLE_OUTPUT: json
|
||||||
KC_LOG_LEVEL: ${KEYCLOAK_LOG_LEVEL:-INFO}
|
KC_LOG_LEVEL: ${KEYCLOAK_LOG_LEVEL:-INFO}
|
||||||
JAVA_OPTS_APPEND: ${KEYCLOAK_JAVA_OPTS:--XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=35}
|
JAVA_OPTS_APPEND: ${KEYCLOAK_JAVA_OPTS:--XX:MaxRAMPercentage=70 -XX:InitialRAMPercentage=35}
|
||||||
|
secrets:
|
||||||
|
- keycloak_db_password
|
||||||
|
- keycloak_admin_password
|
||||||
|
- keycloak_otp_mock_code
|
||||||
|
- keycloak_yandex_captcha_server_key
|
||||||
|
- keycloak_otp_hmac_key
|
||||||
|
- keycloak_settings_bridge_token
|
||||||
|
- keycloak_sms_service_token
|
||||||
expose:
|
expose:
|
||||||
- "8080"
|
- "8080"
|
||||||
- "9000"
|
- "9000"
|
||||||
@@ -56,6 +70,8 @@ services:
|
|||||||
- no-new-privileges:true
|
- no-new-privileges:true
|
||||||
cap_drop:
|
cap_drop:
|
||||||
- ALL
|
- ALL
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
stop_grace_period: 30s
|
stop_grace_period: 30s
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
|
||||||
@@ -64,3 +80,19 @@ networks:
|
|||||||
backend:
|
backend:
|
||||||
observability:
|
observability:
|
||||||
egress:
|
egress:
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
keycloak_db_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_DB_PASSWORD
|
||||||
|
keycloak_admin_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_ADMIN_PASSWORD
|
||||||
|
keycloak_otp_mock_code:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_MOCK_CODE
|
||||||
|
keycloak_yandex_captcha_server_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY
|
||||||
|
keycloak_otp_hmac_key:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_OTP_HMAC_KEY
|
||||||
|
keycloak_settings_bridge_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SETTINGS_BRIDGE_TOKEN
|
||||||
|
keycloak_sms_service_token:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/KEYCLOAK_SMS_SERVICE_TOKEN
|
||||||
|
|||||||
@@ -5,7 +5,9 @@ WORKDIR /service
|
|||||||
COPY app ./app
|
COPY app ./app
|
||||||
COPY pyproject.toml ./
|
COPY pyproject.toml ./
|
||||||
RUN pip install --no-cache-dir .
|
RUN pip install --no-cache-dir .
|
||||||
|
COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint
|
||||||
USER app
|
USER app
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health/live', timeout=2)"
|
||||||
|
ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"]
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080"]
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec "$@"
|
||||||
@@ -18,12 +18,13 @@ services:
|
|||||||
APP_ENV: ${APP_ENV:-production-like}
|
APP_ENV: ${APP_ENV:-production-like}
|
||||||
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
RELEASE_VERSION: ${RELEASE_VERSION:-unknown}
|
||||||
OTEL_REMOTE_ENDPOINT: ${OTEL_REMOTE_ENDPOINT}
|
OTEL_REMOTE_ENDPOINT: ${OTEL_REMOTE_ENDPOINT}
|
||||||
OTEL_REMOTE_AUTH_HEADER: ${OTEL_REMOTE_AUTH_HEADER:-}
|
|
||||||
OTEL_REMOTE_TLS_INSECURE: ${OTEL_REMOTE_TLS_INSECURE:-false}
|
OTEL_REMOTE_TLS_INSECURE: ${OTEL_REMOTE_TLS_INSECURE:-false}
|
||||||
expose: ["4317", "4318", "13133", "8888"]
|
expose: ["4317", "4318", "13133", "8888"]
|
||||||
volumes:
|
volumes:
|
||||||
- ./otel-collector.yaml:/etc/otelcol/config.yaml:ro
|
- ./otel-collector.yaml:/etc/otelcol/config.yaml:ro
|
||||||
- otel-queue:/var/lib/otelcol/queue
|
- otel-queue:/var/lib/otelcol/queue
|
||||||
|
secrets:
|
||||||
|
- otel_remote_auth_header
|
||||||
networks: [observability, egress]
|
networks: [observability, egress]
|
||||||
depends_on:
|
depends_on:
|
||||||
otel-queue-init: {condition: service_completed_successfully}
|
otel-queue-init: {condition: service_completed_successfully}
|
||||||
@@ -38,6 +39,8 @@ services:
|
|||||||
user: "10001:10001"
|
user: "10001:10001"
|
||||||
cap_drop: ["ALL"]
|
cap_drop: ["ALL"]
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
mem_limit: 512m
|
mem_limit: 512m
|
||||||
cpus: 0.5
|
cpus: 0.5
|
||||||
logging:
|
logging:
|
||||||
@@ -62,3 +65,7 @@ services:
|
|||||||
user: "10001:10001"
|
user: "10001:10001"
|
||||||
cap_drop: ["ALL"]
|
cap_drop: ["ALL"]
|
||||||
security_opt: ["no-new-privileges:true"]
|
security_opt: ["no-new-privileges:true"]
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
otel_remote_auth_header:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/OTEL_REMOTE_AUTH_HEADER
|
||||||
|
|||||||
@@ -100,7 +100,7 @@ exporters:
|
|||||||
otlp/remote:
|
otlp/remote:
|
||||||
endpoint: "${env:OTEL_REMOTE_ENDPOINT}"
|
endpoint: "${env:OTEL_REMOTE_ENDPOINT}"
|
||||||
headers:
|
headers:
|
||||||
authorization: "${env:OTEL_REMOTE_AUTH_HEADER}"
|
authorization: ${file:/run/secrets/otel_remote_auth_header}
|
||||||
tls:
|
tls:
|
||||||
insecure: "${env:OTEL_REMOTE_TLS_INSECURE}"
|
insecure: "${env:OTEL_REMOTE_TLS_INSECURE}"
|
||||||
sending_queue:
|
sending_queue:
|
||||||
|
|||||||
@@ -4,17 +4,18 @@ services:
|
|||||||
context: .
|
context: .
|
||||||
image: han-chat-redis:${RELEASE_VERSION:-local}
|
image: han-chat-redis:${RELEASE_VERSION:-local}
|
||||||
environment:
|
environment:
|
||||||
REDIS_API_PASSWORD: ${REDIS_API_PASSWORD}
|
|
||||||
REDIS_SAFETY_PASSWORD: ${REDIS_SAFETY_PASSWORD}
|
|
||||||
REDIS_HEALTH_PASSWORD: ${REDIS_HEALTH_PASSWORD}
|
|
||||||
REDIS_MAXMEMORY: ${REDIS_MAXMEMORY:-384mb}
|
REDIS_MAXMEMORY: ${REDIS_MAXMEMORY:-384mb}
|
||||||
REDIS_EVICTION_POLICY: ${REDIS_EVICTION_POLICY:-volatile-lru}
|
REDIS_EVICTION_POLICY: ${REDIS_EVICTION_POLICY:-volatile-lru}
|
||||||
|
secrets:
|
||||||
|
- redis_api_password
|
||||||
|
- redis_safety_password
|
||||||
|
- redis_health_password
|
||||||
expose: ["6379"]
|
expose: ["6379"]
|
||||||
volumes:
|
volumes:
|
||||||
- redis-data:/data
|
- redis-data:/data
|
||||||
networks: [backend]
|
networks: [backend]
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "redis-cli --no-auth-warning --user ops_health --pass \"$$REDIS_HEALTH_PASSWORD\" PING | grep -qx PONG"]
|
test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$(cat /run/secrets/redis_health_password)\" redis-cli --user ops_health PING | grep -qx PONG"]
|
||||||
interval: 10s
|
interval: 10s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 10
|
retries: 10
|
||||||
@@ -28,7 +29,16 @@ services:
|
|||||||
cap_drop: ["ALL"]
|
cap_drop: ["ALL"]
|
||||||
mem_limit: 512m
|
mem_limit: 512m
|
||||||
ulimits:
|
ulimits:
|
||||||
|
core: {soft: 0, hard: 0}
|
||||||
nofile: {soft: 65536, hard: 65536}
|
nofile: {soft: 65536, hard: 65536}
|
||||||
logging:
|
logging:
|
||||||
driver: json-file
|
driver: json-file
|
||||||
options: {max-size: "50m", max-file: "5"}
|
options: {max-size: "50m", max-file: "5"}
|
||||||
|
|
||||||
|
secrets:
|
||||||
|
redis_api_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_API_PASSWORD
|
||||||
|
redis_safety_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_SAFETY_PASSWORD
|
||||||
|
redis_health_password:
|
||||||
|
file: ${HAN_SECRETS_DIR:-/run/han-chat/secrets}/REDIS_HEALTH_PASSWORD
|
||||||
|
|||||||
@@ -1,17 +1,29 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
for name in REDIS_API_PASSWORD REDIS_SAFETY_PASSWORD REDIS_HEALTH_PASSWORD; do
|
for item in \
|
||||||
eval "value=\${$name:-}"
|
REDIS_API_PASSWORD:redis_api_password \
|
||||||
|
REDIS_SAFETY_PASSWORD:redis_safety_password \
|
||||||
|
REDIS_HEALTH_PASSWORD:redis_health_password
|
||||||
|
do
|
||||||
|
name=${item%%:*}
|
||||||
|
file=/run/secrets/${item#*:}
|
||||||
|
if [ ! -r "$file" ]; then
|
||||||
|
echo "redis bootstrap: missing secret file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
if [ "${#value}" -lt 16 ] || printf '%s' "$value" | grep -Eq '[[:space:]]'; then
|
if [ "${#value}" -lt 16 ] || printf '%s' "$value" | grep -Eq '[[:space:]]'; then
|
||||||
echo "redis bootstrap: $name must be at least 16 characters without whitespace" >&2
|
echo "redis bootstrap: $name must be at least 16 characters without whitespace" >&2
|
||||||
exit 64
|
exit 64
|
||||||
fi
|
fi
|
||||||
|
export "$name=$value"
|
||||||
done
|
done
|
||||||
|
|
||||||
umask 077
|
umask 077
|
||||||
envsubst '${REDIS_API_PASSWORD} ${REDIS_SAFETY_PASSWORD} ${REDIS_HEALTH_PASSWORD}' \
|
envsubst '${REDIS_API_PASSWORD} ${REDIS_SAFETY_PASSWORD} ${REDIS_HEALTH_PASSWORD}' \
|
||||||
< /etc/han-redis/users.acl.template > /tmp/users.acl
|
< /etc/han-redis/users.acl.template > /tmp/users.acl
|
||||||
|
unset REDIS_API_PASSWORD REDIS_SAFETY_PASSWORD REDIS_HEALTH_PASSWORD
|
||||||
|
|
||||||
exec redis-server /etc/han-redis/redis.conf \
|
exec redis-server /etc/han-redis/redis.conf \
|
||||||
--aclfile /tmp/users.acl \
|
--aclfile /tmp/users.acl \
|
||||||
|
|||||||
@@ -1,46 +1,56 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Fail-fast validation for the single HAN Chat deployment environment."""
|
"""Validate public deployment config separately from runtime secrets."""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
import re
|
import re
|
||||||
|
import stat
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from urllib.parse import urlparse
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
REQUIRED = {
|
REQUIRED_CONFIG = {
|
||||||
"APP_ENV", "RELEASE_VERSION", "HAN_PG_HOST", "DATABASE_URL",
|
"SECRETS_SOURCE", "APP_ENV", "RELEASE_VERSION", "HAN_PG_HOST", "HAN_PG_PORT",
|
||||||
"BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL",
|
"HAN_PG_DATABASE", "PG_CA_HOST_PATH", "KEYCLOAK_DB_URL", "KEYCLOAK_DB_USERNAME",
|
||||||
"MESSAGE_SAFETY_DATABASE_URL", "SMS_DATABASE_URL", "KEYCLOAK_DB_URL", "PUBLIC_HOST",
|
"PUBLIC_HOST", "PUBLIC_WEB_URL",
|
||||||
"PUBLIC_WEB_URL", "PUBLIC_API_URL", "PUBLIC_AUTH_URL",
|
"PUBLIC_API_URL", "PUBLIC_AUTH_URL", "KEYCLOAK_PUBLIC_URL",
|
||||||
"KEYCLOAK_PUBLIC_URL", "KEYCLOAK_INTERNAL_URL", "REDIS_URL",
|
"KEYCLOAK_INTERNAL_URL", "KEYCLOAK_REALM", "KEYCLOAK_SMS_SERVICE_URL",
|
||||||
"REDIS_REALTIME_URL", "MESSAGE_SAFETY_REDIS_URL",
|
"IDGTL_SMS_BASE_URL", "IDGTL_SMS_CALLBACK_PUBLIC_URL",
|
||||||
|
}
|
||||||
|
REQUIRED_RUNTIME = {
|
||||||
|
"DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL",
|
||||||
|
"SMS_DATABASE_URL", "KEYCLOAK_DB_PASSWORD", "REDIS_URL",
|
||||||
|
"REDIS_REALTIME_URL", "MESSAGE_SAFETY_REDIS_URL", "REDIS_API_PASSWORD",
|
||||||
|
"REDIS_SAFETY_PASSWORD", "REDIS_HEALTH_PASSWORD",
|
||||||
"MESSAGE_SAFETY_SERVICE_TOKEN", "BITRIX_LOCAL_APP_INTERNAL_TOKEN",
|
"MESSAGE_SAFETY_SERVICE_TOKEN", "BITRIX_LOCAL_APP_INTERNAL_TOKEN",
|
||||||
"BITRIX_INTERNAL_API_TOKEN", "BITRIX_API_FORWARD_TOKEN",
|
"BITRIX_INTERNAL_API_TOKEN", "BITRIX_API_FORWARD_TOKEN",
|
||||||
"BITRIX_API_INBOX_TOKEN", "BITRIX_SYNC_SERVICE_TOKEN",
|
"BITRIX_API_INBOX_TOKEN", "BITRIX_SYNC_SERVICE_TOKEN",
|
||||||
"KEYCLOAK_SETTINGS_BRIDGE_TOKEN", "KEYCLOAK_OTP_HMAC_KEY",
|
"KEYCLOAK_SETTINGS_BRIDGE_TOKEN", "KEYCLOAK_OTP_HMAC_KEY",
|
||||||
"KEYCLOAK_SMS_SERVICE_URL", "KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN",
|
"KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN",
|
||||||
"IDGTL_SMS_BASE_URL", "IDGTL_SMS_API_KEY", "IDGTL_SMS_CALLBACK_PUBLIC_URL",
|
"IDGTL_SMS_API_KEY", "IDGTL_SMS_CALLBACK_USERNAME",
|
||||||
"IDGTL_SMS_CALLBACK_USERNAME", "IDGTL_SMS_CALLBACK_PASSWORD",
|
"IDGTL_SMS_CALLBACK_PASSWORD", "KEYCLOAK_ADMIN_PASSWORD",
|
||||||
"KEYCLOAK_ADMIN", "KEYCLOAK_ADMIN_PASSWORD", "CURSOR_HMAC_SECRET",
|
"CURSOR_HMAC_SECRET", "BITRIX_TOKEN_ENCRYPTION_KEY",
|
||||||
"BITRIX_TOKEN_ENCRYPTION_KEY",
|
|
||||||
"SELECTEL_S3_ENDPOINT_URL",
|
|
||||||
"SELECTEL_S3_ACCESS_KEY", "SELECTEL_S3_SECRET_KEY",
|
"SELECTEL_S3_ACCESS_KEY", "SELECTEL_S3_SECRET_KEY",
|
||||||
"SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY",
|
|
||||||
"SELECTEL_S3_QUARANTINE_READ_SECRET_KEY", "REDIS_API_PASSWORD",
|
|
||||||
"REDIS_SAFETY_PASSWORD", "REDIS_HEALTH_PASSWORD",
|
|
||||||
}
|
}
|
||||||
SECRET_KEYS = {
|
OPTIONAL_SECRET_KEYS = {
|
||||||
key for key in REQUIRED
|
|
||||||
if any(word in key for word in ("TOKEN", "PASSWORD", "SECRET_KEY", "ACCESS_KEY"))
|
|
||||||
} | {
|
|
||||||
"BITRIX_CLIENT_SECRET", "BITRIX_APPLICATION_TOKEN",
|
"BITRIX_CLIENT_SECRET", "BITRIX_APPLICATION_TOKEN",
|
||||||
"IDGTL_SMS_CALLBACK_USERNAME", "IDGTL_SMS_CALLBACK_PASSWORD",
|
"BITRIX_SYNC_CRM_WEBHOOK_URL", "BITRIX_SYNC_WEBHOOK_TOKEN",
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
||||||
|
"NOTIFICATIONS_TOKEN_PRODUCER_TEST",
|
||||||
|
"OTEL_REMOTE_AUTH_HEADER", "KEYCLOAK_OTP_MOCK_CODE",
|
||||||
|
}
|
||||||
|
FORBIDDEN_CONFIG_KEYS = REQUIRED_RUNTIME | OPTIONAL_SECRET_KEYS | {
|
||||||
|
"BITRIX_SYNC_APP_DATABASE_URL", "PG_BACKUP_DSN",
|
||||||
|
}
|
||||||
|
DSN_KEYS = {
|
||||||
|
"DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_APP_DATABASE_URL",
|
||||||
|
"BITRIX_SYNC_DATABASE_URL", "MESSAGE_SAFETY_DATABASE_URL",
|
||||||
|
"SMS_DATABASE_URL", "PG_BACKUP_DSN",
|
||||||
}
|
}
|
||||||
PLACEHOLDER = re.compile(r"(change-me|example\.(com|ru|invalid)|<[^>]+>)", re.I)
|
PLACEHOLDER = re.compile(r"(change-me|example\.(com|ru|invalid)|<[^>]+>)", re.I)
|
||||||
|
|
||||||
|
|
||||||
def load(path: Path) -> tuple[dict[str, str], list[str]]:
|
def load_env(path: Path) -> tuple[dict[str, str], list[str]]:
|
||||||
values: dict[str, str] = {}
|
values: dict[str, str] = {}
|
||||||
errors: list[str] = []
|
errors: list[str] = []
|
||||||
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1):
|
||||||
@@ -58,33 +68,107 @@ def load(path: Path) -> tuple[dict[str, str], list[str]]:
|
|||||||
return values, errors
|
return values, errors
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
def load_manifest(path: Path) -> tuple[dict[str, str], list[str]]:
|
||||||
path = Path(sys.argv[1] if len(sys.argv) > 1 else ".env")
|
files, errors = load_env(path)
|
||||||
if not path.is_file():
|
values: dict[str, str] = {}
|
||||||
print(f"ERROR: файл не найден: {path}", file=sys.stderr)
|
for key, raw_path in files.items():
|
||||||
return 2
|
secret_path = Path(raw_path)
|
||||||
env, errors = load(path)
|
if not secret_path.is_absolute():
|
||||||
for key in sorted(REQUIRED):
|
errors.append(f"{key}: manifest должен содержать абсолютный путь")
|
||||||
if not env.get(key):
|
continue
|
||||||
errors.append(f"{key}: обязательное значение отсутствует")
|
try:
|
||||||
for key in sorted(SECRET_KEYS):
|
metadata = secret_path.lstat()
|
||||||
value = env.get(key, "")
|
mode = stat.S_IMODE(metadata.st_mode)
|
||||||
if value and (len(value) < 16 or PLACEHOLDER.search(value)):
|
parent_mode = stat.S_IMODE(secret_path.parent.stat().st_mode)
|
||||||
errors.append(f"{key}: секрет должен быть непустым, уникальным и длиной >=16")
|
if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
|
||||||
for key in ("SMS_SERVICE_TOKEN", "KEYCLOAK_SMS_SERVICE_TOKEN"):
|
errors.append(f"{key}: secret file должен быть regular file, не symlink")
|
||||||
if env.get(key) and len(env[key]) < 32:
|
continue
|
||||||
errors.append(f"{key}: service token должен иметь длину >=32")
|
if os.name != "nt" and (mode & 0o022 or parent_mode & 0o077):
|
||||||
captcha_enabled = env.get("KEYCLOAK_YANDEX_CAPTCHA_ENABLED", "false").lower()
|
errors.append(
|
||||||
if captcha_enabled not in {"true", "false"}:
|
f"{key}: secret file или runtime directory имеют небезопасные права"
|
||||||
errors.append("KEYCLOAK_YANDEX_CAPTCHA_ENABLED: ожидается true или false")
|
)
|
||||||
if captcha_enabled == "true":
|
continue
|
||||||
for key in (
|
values[key] = secret_path.read_text(encoding="utf-8").rstrip("\r\n")
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY",
|
except OSError:
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
errors.append(f"{key}: secret file недоступен")
|
||||||
):
|
return values, errors
|
||||||
if not env.get(key):
|
|
||||||
errors.append(f"{key}: обязательное значение при включённой CAPTCHA отсутствует")
|
|
||||||
|
|
||||||
|
|
||||||
|
def runtime_from_environment() -> dict[str, str]:
|
||||||
|
keys = REQUIRED_RUNTIME | OPTIONAL_SECRET_KEYS
|
||||||
|
return {key: os.environ[key] for key in keys if key in os.environ}
|
||||||
|
|
||||||
|
|
||||||
|
def validate_config(config: dict[str, str], errors: list[str]) -> None:
|
||||||
|
for key in sorted(REQUIRED_CONFIG):
|
||||||
|
if not config.get(key):
|
||||||
|
errors.append(f"{key}: обязательный config отсутствует")
|
||||||
|
if config.get("SECRETS_SOURCE") not in {"selectel", "file"}:
|
||||||
|
errors.append("SECRETS_SOURCE: ожидается selectel или file")
|
||||||
|
for key in sorted(FORBIDDEN_CONFIG_KEYS & config.keys()):
|
||||||
|
errors.append(f"{key}: секрет или credential-bearing DSN запрещён в обычном .env")
|
||||||
|
for key, value in config.items():
|
||||||
|
if key.endswith("_FILE") and value:
|
||||||
|
errors.append(f"{key}: secret file path задаётся launcher, не обычным .env")
|
||||||
|
if key.endswith("_URL"):
|
||||||
|
parsed = urlparse(value.replace("jdbc:", "", 1))
|
||||||
|
if parsed.username or parsed.password:
|
||||||
|
errors.append(f"{key}: URL с credentials запрещён в обычном .env")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_runtime(config: dict[str, str], runtime: dict[str, str], errors: list[str]) -> None:
|
||||||
|
required = set(REQUIRED_RUNTIME)
|
||||||
|
if config.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "true":
|
||||||
|
required.add("KEYCLOAK_OTP_MOCK_CODE")
|
||||||
|
if config.get("KEYCLOAK_YANDEX_CAPTCHA_ENABLED", "false").lower() == "true":
|
||||||
|
required.add("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY")
|
||||||
|
if not config.get("KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY"):
|
||||||
|
errors.append(
|
||||||
|
"KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: обязательный public config отсутствует"
|
||||||
|
)
|
||||||
|
for key in sorted(required):
|
||||||
|
value = runtime.get(key, "")
|
||||||
|
if not value:
|
||||||
|
errors.append(f"{key}: обязательный runtime secret отсутствует")
|
||||||
|
elif key == "KEYCLOAK_OTP_MOCK_CODE":
|
||||||
|
if not re.fullmatch(r"\d{6,10}", value) or value == "1234":
|
||||||
|
errors.append(
|
||||||
|
f"{key}: требуется нестандартный цифровой код длиной 6–10 цифр"
|
||||||
|
)
|
||||||
|
elif key not in DSN_KEYS and (len(value) < 16 or PLACEHOLDER.search(value)):
|
||||||
|
errors.append(f"{key}: runtime secret слишком короткий или является placeholder")
|
||||||
|
|
||||||
|
for key in DSN_KEYS & runtime.keys():
|
||||||
|
value = runtime[key]
|
||||||
|
if "sslmode=verify-full" not in value or "sslrootcert=" not in value:
|
||||||
|
errors.append(f"{key}: требуется sslmode=verify-full и sslrootcert")
|
||||||
|
if re.search(r"(?:[?&](?:options|currentSchema)=)", value, re.I):
|
||||||
|
errors.append(f"{key}: options/currentSchema запрещены через PgBouncer")
|
||||||
|
|
||||||
|
redis_contract = (
|
||||||
|
("REDIS_URL", "api_backend", "REDIS_API_PASSWORD", "/0"),
|
||||||
|
("REDIS_REALTIME_URL", "api_backend", "REDIS_API_PASSWORD", "/1"),
|
||||||
|
("MESSAGE_SAFETY_REDIS_URL", "message_safety", "REDIS_SAFETY_PASSWORD", "/2"),
|
||||||
|
)
|
||||||
|
for key, username, password_key, database in redis_contract:
|
||||||
|
parsed = urlparse(runtime.get(key, ""))
|
||||||
|
if (
|
||||||
|
parsed.scheme != "redis" or parsed.hostname != "redis"
|
||||||
|
or parsed.username != username or parsed.password != runtime.get(password_key)
|
||||||
|
or parsed.path != database
|
||||||
|
):
|
||||||
|
errors.append(f"{key}: ACL user/password/host/DB не согласованы с {password_key}")
|
||||||
|
|
||||||
|
for left, right in (
|
||||||
|
("BITRIX_LOCAL_APP_INTERNAL_TOKEN", "BITRIX_INTERNAL_API_TOKEN"),
|
||||||
|
("BITRIX_API_FORWARD_TOKEN", "BITRIX_API_INBOX_TOKEN"),
|
||||||
|
("KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN"),
|
||||||
|
):
|
||||||
|
if runtime.get(left) != runtime.get(right):
|
||||||
|
errors.append(f"{left} должен совпадать с {right}")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_shared(env: dict[str, str], errors: list[str]) -> None:
|
||||||
production = env.get("APP_ENV") in {"production-like", "production"}
|
production = env.get("APP_ENV") in {"production-like", "production"}
|
||||||
if production and env.get("FRONTEND_DEV_PROXY_ENABLED", "").lower() != "false":
|
if production and env.get("FRONTEND_DEV_PROXY_ENABLED", "").lower() != "false":
|
||||||
errors.append("FRONTEND_DEV_PROXY_ENABLED: production-like/production требует false")
|
errors.append("FRONTEND_DEV_PROXY_ENABLED: production-like/production требует false")
|
||||||
@@ -105,41 +189,6 @@ def main() -> int:
|
|||||||
errors.append(f"{key}: ожидается http URL с Docker DNS service name")
|
errors.append(f"{key}: ожидается http URL с Docker DNS service name")
|
||||||
if not env.get("KEYCLOAK_INTERNAL_URL", "").rstrip("/").endswith("/auth"):
|
if not env.get("KEYCLOAK_INTERNAL_URL", "").rstrip("/").endswith("/auth"):
|
||||||
errors.append("KEYCLOAK_INTERNAL_URL: внутренний URL должен заканчиваться на /auth")
|
errors.append("KEYCLOAK_INTERNAL_URL: внутренний URL должен заканчиваться на /auth")
|
||||||
redis_contract = (
|
|
||||||
("REDIS_URL", "api_backend", "REDIS_API_PASSWORD", "/0"),
|
|
||||||
("REDIS_REALTIME_URL", "api_backend", "REDIS_API_PASSWORD", "/1"),
|
|
||||||
("MESSAGE_SAFETY_REDIS_URL", "message_safety", "REDIS_SAFETY_PASSWORD", "/2"),
|
|
||||||
)
|
|
||||||
for key, username, password_key, database in redis_contract:
|
|
||||||
parsed = urlparse(env.get(key, ""))
|
|
||||||
if (
|
|
||||||
parsed.scheme != "redis"
|
|
||||||
or parsed.hostname != "redis"
|
|
||||||
or parsed.username != username
|
|
||||||
or parsed.password != env.get(password_key)
|
|
||||||
or parsed.path != database
|
|
||||||
):
|
|
||||||
errors.append(f"{key}: ACL user/password/host/DB не согласованы с {password_key}")
|
|
||||||
for key in (
|
|
||||||
"DATABASE_URL", "BITRIX_DATABASE_URL", "BITRIX_SYNC_DATABASE_URL",
|
|
||||||
"MESSAGE_SAFETY_DATABASE_URL", "SMS_DATABASE_URL", "KEYCLOAK_DB_URL",
|
|
||||||
):
|
|
||||||
value = env.get(key, "")
|
|
||||||
if "sslmode=verify-full" not in value or "sslrootcert=" not in value:
|
|
||||||
errors.append(f"{key}: требуется sslmode=verify-full и sslrootcert")
|
|
||||||
if re.search(r"(?:[?&](?:options|currentSchema)=)", value, re.I):
|
|
||||||
errors.append(
|
|
||||||
f"{key}: options/currentSchema запрещены через PgBouncer; "
|
|
||||||
"используйте database-level search_path роли"
|
|
||||||
)
|
|
||||||
pairs = (
|
|
||||||
("BITRIX_LOCAL_APP_INTERNAL_TOKEN", "BITRIX_INTERNAL_API_TOKEN"),
|
|
||||||
("BITRIX_API_FORWARD_TOKEN", "BITRIX_API_INBOX_TOKEN"),
|
|
||||||
("KEYCLOAK_SMS_SERVICE_TOKEN", "SMS_SERVICE_TOKEN"),
|
|
||||||
)
|
|
||||||
for left, right in pairs:
|
|
||||||
if env.get(left) != env.get(right):
|
|
||||||
errors.append(f"{left} должен совпадать с {right}")
|
|
||||||
try:
|
try:
|
||||||
poll = int(env.get("MESSAGE_SAFETY_TASK_POLL_MAX_SEC", "0"))
|
poll = int(env.get("MESSAGE_SAFETY_TASK_POLL_MAX_SEC", "0"))
|
||||||
nginx = int(env.get("NGINX_MESSAGE_READ_TIMEOUT_SEC", "0"))
|
nginx = int(env.get("NGINX_MESSAGE_READ_TIMEOUT_SEC", "0"))
|
||||||
@@ -153,24 +202,55 @@ def main() -> int:
|
|||||||
if production and env.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "true":
|
if production and env.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "true":
|
||||||
if env.get("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED", "").lower() != "true":
|
if env.get("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED", "").lower() != "true":
|
||||||
errors.append("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true обязателен для mock OTP")
|
errors.append("KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true обязателен для mock OTP")
|
||||||
real_sms = env.get("KEYCLOAK_OTP_MOCK_ENABLED", "").lower() == "false"
|
callback = f"https://{env.get('PUBLIC_HOST', '')}/callbacks/idgtl/sms"
|
||||||
if production and real_sms and (
|
if production and env.get("IDGTL_SMS_CALLBACK_PUBLIC_URL") != callback:
|
||||||
env.get("IDGTL_SMS_BASE_URL", "").rstrip("/") != "https://direct.i-dgtl.ru"
|
errors.append("IDGTL_SMS_CALLBACK_PUBLIC_URL должен совпадать с PUBLIC_HOST")
|
||||||
):
|
|
||||||
errors.append("IDGTL_SMS_BASE_URL: production contract требует https://direct.i-dgtl.ru")
|
|
||||||
expected_callback = f"https://{env.get('PUBLIC_HOST', '')}/callbacks/idgtl/sms"
|
|
||||||
if production and env.get("IDGTL_SMS_CALLBACK_PUBLIC_URL") != expected_callback:
|
|
||||||
errors.append(
|
|
||||||
"IDGTL_SMS_CALLBACK_PUBLIC_URL должен совпадать с публичным host и callback path"
|
|
||||||
)
|
|
||||||
if env.get("HAN_PG_HOST") in {"localhost", "127.0.0.1", "postgres", "db"}:
|
if env.get("HAN_PG_HOST") in {"localhost", "127.0.0.1", "postgres", "db"}:
|
||||||
errors.append("HAN_PG_HOST: PostgreSQL должен быть внешним managed endpoint")
|
errors.append("HAN_PG_HOST: PostgreSQL должен быть внешним managed endpoint")
|
||||||
|
keycloak_dsn = env.get("KEYCLOAK_DB_URL", "")
|
||||||
|
if "sslmode=verify-full" not in keycloak_dsn or "sslrootcert=" not in keycloak_dsn:
|
||||||
|
errors.append("KEYCLOAK_DB_URL: требуется sslmode=verify-full и sslrootcert")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
args = sys.argv[1:]
|
||||||
|
config_path = Path(args.pop(0) if args and not args[0].startswith("--") else ".env")
|
||||||
|
manifest: Path | None = None
|
||||||
|
use_runtime_env = False
|
||||||
|
while args:
|
||||||
|
option = args.pop(0)
|
||||||
|
if option == "--runtime-manifest" and args:
|
||||||
|
manifest = Path(args.pop(0))
|
||||||
|
elif option == "--runtime-env":
|
||||||
|
use_runtime_env = True
|
||||||
|
else:
|
||||||
|
print(f"ERROR: неизвестный параметр {option}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
if not config_path.is_file():
|
||||||
|
print(f"ERROR: config файл не найден: {config_path}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
config, errors = load_env(config_path)
|
||||||
|
validate_config(config, errors)
|
||||||
|
runtime: dict[str, str] = {}
|
||||||
|
if manifest:
|
||||||
|
if not manifest.is_file():
|
||||||
|
errors.append("runtime manifest недоступен")
|
||||||
|
else:
|
||||||
|
runtime, manifest_errors = load_manifest(manifest)
|
||||||
|
errors.extend(manifest_errors)
|
||||||
|
elif use_runtime_env:
|
||||||
|
runtime = runtime_from_environment()
|
||||||
|
if manifest or use_runtime_env:
|
||||||
|
validate_runtime(config, runtime, errors)
|
||||||
|
validate_shared({**config, **runtime}, errors)
|
||||||
|
|
||||||
if errors:
|
if errors:
|
||||||
for error in errors:
|
for error in errors:
|
||||||
print(f"ERROR: {error}", file=sys.stderr)
|
print(f"ERROR: {error}", file=sys.stderr)
|
||||||
return 1
|
return 1
|
||||||
print(f"OK: {path} прошёл проверку ({len(env)} переменных)")
|
scope = "config + runtime secrets" if manifest or use_runtime_env else "non-secret config"
|
||||||
|
print(f"OK: {config_path} прошёл проверку ({scope})")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -13,6 +13,8 @@ COPY --from=builder /build/dist/*.whl /tmp/
|
|||||||
RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl
|
RUN pip install --no-cache-dir /tmp/*.whl && rm -f /tmp/*.whl
|
||||||
COPY alembic.ini ./
|
COPY alembic.ini ./
|
||||||
COPY migrations ./migrations
|
COPY migrations ./migrations
|
||||||
|
COPY --chmod=0555 container-entrypoint.sh /usr/local/bin/han-container-entrypoint
|
||||||
USER 10001:10001
|
USER 10001:10001
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
|
ENTRYPOINT ["/usr/local/bin/han-container-entrypoint"]
|
||||||
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--no-proxy-headers"]
|
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8080", "--no-proxy-headers"]
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from collections.abc import Mapping
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
REDACTED = "[REDACTED]"
|
||||||
|
_SENSITIVE_KEY = re.compile(
|
||||||
|
r"(authorization|cookie|password|passwd|secret|token|api[_-]?key|"
|
||||||
|
r"database[_-]?url|redis[_-]?url|dsn|callback[_-]?url)",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_URI_USERINFO = re.compile(r"(?P<scheme>[a-z][a-z0-9+.-]*://)[^/@\s]+@", re.IGNORECASE)
|
||||||
|
_QUERY_SECRET = re.compile(
|
||||||
|
r"(?P<prefix>[?&](?:token|access_token|api_key|key|secret|password)=)[^&#\s]+",
|
||||||
|
re.IGNORECASE,
|
||||||
|
)
|
||||||
|
_AUTH_VALUE = re.compile(r"\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]+", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_text(value: str) -> str:
|
||||||
|
value = _URI_USERINFO.sub(r"\g<scheme>[REDACTED]@", value)
|
||||||
|
value = _QUERY_SECRET.sub(r"\g<prefix>[REDACTED]", value)
|
||||||
|
return _AUTH_VALUE.sub(r"\1 [REDACTED]", value)
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_value(value: Any) -> Any:
|
||||||
|
if isinstance(value, str):
|
||||||
|
return sanitize_text(value)
|
||||||
|
if isinstance(value, Mapping):
|
||||||
|
return {
|
||||||
|
str(key): REDACTED if _SENSITIVE_KEY.search(str(key)) else sanitize_value(item)
|
||||||
|
for key, item in value.items()
|
||||||
|
}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [sanitize_value(item) for item in value]
|
||||||
|
if isinstance(value, tuple):
|
||||||
|
return tuple(sanitize_value(item) for item in value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def redact_event(
|
||||||
|
_logger: Any,
|
||||||
|
_method_name: str,
|
||||||
|
event_dict: dict[str, Any],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
return sanitize_value(event_dict)
|
||||||
@@ -22,6 +22,7 @@ from starlette.exceptions import HTTPException as StarletteHTTPException
|
|||||||
|
|
||||||
from app.db import Database, SmsTemplate
|
from app.db import Database, SmsTemplate
|
||||||
from app.domain import DomainError
|
from app.domain import DomainError
|
||||||
|
from app.logging_security import redact_event
|
||||||
from app.metrics import CALLBACK_LAG, CALLBACK_TOTAL
|
from app.metrics import CALLBACK_LAG, CALLBACK_TOTAL
|
||||||
from app.schemas import CallbackItem, ErrorEnvelope, MessageResponse, SendRequest, SendResponse
|
from app.schemas import CallbackItem, ErrorEnvelope, MessageResponse, SendRequest, SendResponse
|
||||||
from app.service import (
|
from app.service import (
|
||||||
@@ -42,6 +43,7 @@ def configure_logging(level: str) -> None:
|
|||||||
processors=[
|
processors=[
|
||||||
structlog.contextvars.merge_contextvars,
|
structlog.contextvars.merge_contextvars,
|
||||||
add_trace_context,
|
add_trace_context,
|
||||||
|
redact_event,
|
||||||
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
||||||
structlog.stdlib.add_log_level,
|
structlog.stdlib.add_log_level,
|
||||||
structlog.processors.JSONRenderer(),
|
structlog.processors.JSONRenderer(),
|
||||||
@@ -147,7 +149,11 @@ async def http_error(request: Request, exc: StarletteHTTPException) -> JSONRespo
|
|||||||
|
|
||||||
@app.exception_handler(Exception)
|
@app.exception_handler(Exception)
|
||||||
async def unhandled_error(request: Request, exc: Exception) -> JSONResponse:
|
async def unhandled_error(request: Request, exc: Exception) -> JSONResponse:
|
||||||
log.exception("request.failed", error_code="internal_error")
|
log.error(
|
||||||
|
"request.failed",
|
||||||
|
error_code="internal_error",
|
||||||
|
error_type=type(exc).__name__,
|
||||||
|
)
|
||||||
return error_response(request, "internal_error", "Internal server error", 500)
|
return error_response(request, "internal_error", "Internal server error", 500)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ from prometheus_client import start_http_server
|
|||||||
from sqlalchemy import and_, func, or_, select, update
|
from sqlalchemy import and_, func, or_, select, update
|
||||||
|
|
||||||
from app.db import Database, SendStatus, SmsOutboundMessage
|
from app.db import Database, SendStatus, SmsOutboundMessage
|
||||||
|
from app.logging_security import redact_event
|
||||||
from app.metrics import (
|
from app.metrics import (
|
||||||
JOURNAL_ROWS,
|
JOURNAL_ROWS,
|
||||||
PENDING_AGE,
|
PENDING_AGE,
|
||||||
@@ -37,6 +38,7 @@ def configure_logging(level: str) -> None:
|
|||||||
processors=[
|
processors=[
|
||||||
structlog.contextvars.merge_contextvars,
|
structlog.contextvars.merge_contextvars,
|
||||||
add_trace_context,
|
add_trace_context,
|
||||||
|
redact_event,
|
||||||
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
structlog.processors.TimeStamper(fmt="iso", utc=True, key="timestamp"),
|
||||||
structlog.stdlib.add_log_level,
|
structlog.stdlib.add_log_level,
|
||||||
structlog.processors.JSONRenderer(),
|
structlog.processors.JSONRenderer(),
|
||||||
@@ -217,9 +219,12 @@ async def worker_loop(stop: asyncio.Event) -> None:
|
|||||||
await save_result(db, message.id, result, message.attempt_count)
|
await save_result(db, message.id, result, message.attempt_count)
|
||||||
except TimeoutError:
|
except TimeoutError:
|
||||||
continue
|
continue
|
||||||
except Exception:
|
except Exception as exc:
|
||||||
SETTINGS_VALID.set(0)
|
SETTINGS_VALID.set(0)
|
||||||
log.exception("worker.iteration_failed")
|
log.error(
|
||||||
|
"worker.iteration_failed",
|
||||||
|
error_type=type(exc).__name__,
|
||||||
|
)
|
||||||
try:
|
try:
|
||||||
await asyncio.wait_for(stop.wait(), timeout=5)
|
await asyncio.wait_for(stop.wait(), timeout=5)
|
||||||
except TimeoutError:
|
except TimeoutError:
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
for name in ${HAN_SECRET_VARS:-}; do
|
||||||
|
case "$name" in
|
||||||
|
""|[0-9]*|*[!A-Z0-9_]*)
|
||||||
|
echo "container secrets: invalid variable name" >&2
|
||||||
|
exit 64
|
||||||
|
;;
|
||||||
|
*) ;;
|
||||||
|
esac
|
||||||
|
eval "file=\${${name}_FILE:-}"
|
||||||
|
if [ -z "$file" ] || [ ! -r "$file" ]; then
|
||||||
|
echo "container secrets: missing file for $name" >&2
|
||||||
|
exit 66
|
||||||
|
fi
|
||||||
|
value=$(cat "$file")
|
||||||
|
export "$name=$value"
|
||||||
|
unset "${name}_FILE"
|
||||||
|
done
|
||||||
|
|
||||||
|
unset HAN_SECRET_VARS
|
||||||
|
exec "$@"
|
||||||
@@ -1,17 +1,17 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import os
|
||||||
from logging.config import fileConfig
|
from logging.config import fileConfig
|
||||||
|
|
||||||
from alembic import context
|
from alembic import context
|
||||||
|
|
||||||
from app.db import Base, create_postgres_engine
|
from app.db import Base, create_postgres_engine
|
||||||
from app.settings import get_settings
|
|
||||||
|
|
||||||
config = context.config
|
config = context.config
|
||||||
if config.config_file_name:
|
if config.config_file_name:
|
||||||
fileConfig(config.config_file_name)
|
fileConfig(config.config_file_name)
|
||||||
database_url = get_settings().database_url
|
database_url = os.environ["SMS_DATABASE_URL"]
|
||||||
if database_url.startswith("postgresql://"):
|
if database_url.startswith("postgresql://"):
|
||||||
database_url = database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
database_url = database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
|
||||||
config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
|
config.set_main_option("sqlalchemy.url", database_url.replace("%", "%%"))
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
from app.logging_security import REDACTED, redact_event, sanitize_text
|
||||||
|
|
||||||
|
|
||||||
|
def test_redacts_sensitive_fields_recursively() -> None:
|
||||||
|
event = {
|
||||||
|
"api_key": "provider-key",
|
||||||
|
"nested": {
|
||||||
|
"callback_url": "https://user:password@example.test/callback",
|
||||||
|
"status": "accepted",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
redacted = redact_event(None, "info", event)
|
||||||
|
|
||||||
|
assert redacted["api_key"] == REDACTED
|
||||||
|
assert redacted["nested"]["callback_url"] == REDACTED
|
||||||
|
assert redacted["nested"]["status"] == "accepted"
|
||||||
|
|
||||||
|
|
||||||
|
def test_redacts_credentials_embedded_in_text() -> None:
|
||||||
|
value = (
|
||||||
|
"POST https://callback-user:callback-password@example.test/cb"
|
||||||
|
"?api_key=query-secret Authorization=Basic header-secret"
|
||||||
|
)
|
||||||
|
|
||||||
|
redacted = sanitize_text(value)
|
||||||
|
|
||||||
|
assert "callback-password" not in redacted
|
||||||
|
assert "query-secret" not in redacted
|
||||||
|
assert "header-secret" not in redacted
|
||||||
|
assert redacted.count(REDACTED) == 3
|
||||||
@@ -44,11 +44,13 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
application = (ROOT / "infra/compose/application.yml").read_text(encoding="utf-8")
|
application = (ROOT / "infra/compose/application.yml").read_text(encoding="utf-8")
|
||||||
self.assertIn("networks: [backend, observability, egress]", application)
|
self.assertIn("networks: [backend, observability, egress]", application)
|
||||||
self.assertIn("networks: [public, backend, observability, egress]", application)
|
self.assertIn("networks: [public, backend, observability, egress]", application)
|
||||||
self.assertIn('KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY: ""', application)
|
self.assertIn(
|
||||||
|
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY_FILE: "
|
||||||
|
"/run/secrets/keycloak_yandex_captcha_server_key",
|
||||||
|
application,
|
||||||
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
application.count(
|
application.count("IDGTL_SMS_API_KEY_FILE: /run/secrets/idgtl_sms_api_key"),
|
||||||
"IDGTL_SMS_API_KEY: ${IDGTL_SMS_API_KEY:?IDGTL_SMS_API_KEY is required}"
|
|
||||||
),
|
|
||||||
1,
|
1,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -56,7 +58,7 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
self.assertEqual(jobs.count("networks: [backend, egress]"), 5)
|
self.assertEqual(jobs.count("networks: [backend, egress]"), 5)
|
||||||
|
|
||||||
observability = (ROOT / "observability/docker-compose.yml").read_text(encoding="utf-8")
|
observability = (ROOT / "observability/docker-compose.yml").read_text(encoding="utf-8")
|
||||||
self.assertIn("networks: [observability, backend, egress]", observability)
|
self.assertIn("networks: [observability, egress]", observability)
|
||||||
|
|
||||||
redis = (ROOT / "redis/docker-compose.yml").read_text(encoding="utf-8")
|
redis = (ROOT / "redis/docker-compose.yml").read_text(encoding="utf-8")
|
||||||
self.assertNotIn("egress", redis)
|
self.assertNotIn("egress", redis)
|
||||||
@@ -64,6 +66,8 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
def test_only_nginx_fragment_publishes_ports(self) -> None:
|
def test_only_nginx_fragment_publishes_ports(self) -> None:
|
||||||
forbidden = (
|
forbidden = (
|
||||||
ROOT / "infra/compose/application.yml",
|
ROOT / "infra/compose/application.yml",
|
||||||
|
ROOT / "api-backend/docker-compose.yml",
|
||||||
|
ROOT / "keycloak/docker-compose.yml",
|
||||||
ROOT / "redis/docker-compose.yml",
|
ROOT / "redis/docker-compose.yml",
|
||||||
ROOT / "observability/docker-compose.yml",
|
ROOT / "observability/docker-compose.yml",
|
||||||
ROOT / "deployment/docker-compose.jobs.yml",
|
ROOT / "deployment/docker-compose.jobs.yml",
|
||||||
@@ -177,6 +181,10 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
self.assertIn("storage: file_storage", config)
|
self.assertIn("storage: file_storage", config)
|
||||||
self.assertIn("retry_on_failure:", config)
|
self.assertIn("retry_on_failure:", config)
|
||||||
self.assertIn('insecure: "${env:OTEL_REMOTE_TLS_INSECURE}"', config)
|
self.assertIn('insecure: "${env:OTEL_REMOTE_TLS_INSECURE}"', config)
|
||||||
|
self.assertIn(
|
||||||
|
"authorization: ${file:/run/secrets/otel_remote_auth_header}", config
|
||||||
|
)
|
||||||
|
self.assertNotIn("OTEL_REMOTE_AUTH_HEADER", config)
|
||||||
self.assertIn("tail_sampling:", config)
|
self.assertIn("tail_sampling:", config)
|
||||||
self.assertNotIn("probabilistic_sampler:", config)
|
self.assertNotIn("probabilistic_sampler:", config)
|
||||||
self.assertIn('targets: ["sms-service:8080"]', config)
|
self.assertIn('targets: ["sms-service:8080"]', config)
|
||||||
@@ -231,7 +239,7 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
|
|
||||||
def test_smoke_script_does_not_source_env_as_shell(self) -> None:
|
def test_smoke_script_does_not_source_env_as_shell(self) -> None:
|
||||||
smoke = (ROOT / "deployment/scripts/smoke.sh").read_text(encoding="utf-8")
|
smoke = (ROOT / "deployment/scripts/smoke.sh").read_text(encoding="utf-8")
|
||||||
self.assertNotIn('. "./$ENV_FILE"', smoke)
|
self.assertNotIn('. "./$CONFIG_FILE"', smoke)
|
||||||
for variable in ("PUBLIC_HOST", "PUBLIC_WEB_URL", "KEYCLOAK_REALM"):
|
for variable in ("PUBLIC_HOST", "PUBLIC_WEB_URL", "KEYCLOAK_REALM"):
|
||||||
self.assertIn(f"{variable}=$(env_value {variable})", smoke)
|
self.assertIn(f"{variable}=$(env_value {variable})", smoke)
|
||||||
|
|
||||||
@@ -298,18 +306,64 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
for variable in (
|
for variable in (
|
||||||
"KC_DB_SCHEMA",
|
"KC_DB_SCHEMA",
|
||||||
"KEYCLOAK_OTP_MOCK_ENABLED",
|
"KEYCLOAK_OTP_MOCK_ENABLED",
|
||||||
"KEYCLOAK_OTP_MOCK_CODE",
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED",
|
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED",
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY",
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
|
||||||
"KEYCLOAK_OTP_HMAC_KEY",
|
|
||||||
"KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC",
|
"KEYCLOAK_OTP_SETTINGS_MAX_STALE_SEC",
|
||||||
"KEYCLOAK_SETTINGS_BRIDGE_URL",
|
"KEYCLOAK_SETTINGS_BRIDGE_URL",
|
||||||
"KEYCLOAK_SETTINGS_BRIDGE_TOKEN",
|
|
||||||
"KEYCLOAK_SMS_SERVICE_URL",
|
"KEYCLOAK_SMS_SERVICE_URL",
|
||||||
"KEYCLOAK_SMS_SERVICE_TOKEN",
|
|
||||||
):
|
):
|
||||||
self.assertIn(f" {variable}:", application)
|
self.assertIn(f" {variable}:", application)
|
||||||
|
for variable in (
|
||||||
|
"KEYCLOAK_OTP_MOCK_CODE",
|
||||||
|
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY",
|
||||||
|
"KEYCLOAK_OTP_HMAC_KEY",
|
||||||
|
"KEYCLOAK_SETTINGS_BRIDGE_TOKEN",
|
||||||
|
"KEYCLOAK_SMS_SERVICE_TOKEN",
|
||||||
|
):
|
||||||
|
self.assertIn(f" {variable}_FILE:", application)
|
||||||
|
self.assertIn(
|
||||||
|
" KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY: "
|
||||||
|
"${KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY:-}",
|
||||||
|
application,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_compose_secrets_do_not_enter_config_environment(self) -> None:
|
||||||
|
compose_paths = (
|
||||||
|
ROOT / "infra/compose/application.yml",
|
||||||
|
ROOT / "redis/docker-compose.yml",
|
||||||
|
ROOT / "observability/docker-compose.yml",
|
||||||
|
ROOT / "deployment/docker-compose.jobs.yml",
|
||||||
|
)
|
||||||
|
combined = "\n".join(path.read_text(encoding="utf-8") for path in compose_paths)
|
||||||
|
self.assertNotIn("env_file:", combined)
|
||||||
|
for path in compose_paths:
|
||||||
|
self.assertIn(
|
||||||
|
"core: {soft: 0, hard: 0}",
|
||||||
|
path.read_text(encoding="utf-8"),
|
||||||
|
path,
|
||||||
|
)
|
||||||
|
for variable in (
|
||||||
|
"DATABASE_URL",
|
||||||
|
"REDIS_URL",
|
||||||
|
"SMS_SERVICE_TOKEN",
|
||||||
|
"KEYCLOAK_DB_PASSWORD",
|
||||||
|
"OTEL_REMOTE_AUTH_HEADER",
|
||||||
|
):
|
||||||
|
self.assertNotIn(f"${{{variable}", combined)
|
||||||
|
self.assertIn(
|
||||||
|
"${HAN_SECRETS_DIR:-/run/han-chat/secrets}/DATABASE_URL", combined
|
||||||
|
)
|
||||||
|
self.assertNotIn("entrypoint: []", combined)
|
||||||
|
|
||||||
|
for service in (
|
||||||
|
"api-backend",
|
||||||
|
"sms-service",
|
||||||
|
"message-safety",
|
||||||
|
"bitrix-local-app",
|
||||||
|
"bitrix-sync",
|
||||||
|
"keycloak",
|
||||||
|
):
|
||||||
|
dockerfile = (ROOT / service / "Dockerfile").read_text(encoding="utf-8")
|
||||||
|
self.assertIn("han-container-entrypoint", dockerfile, service)
|
||||||
|
|
||||||
def test_env_validator_accepts_materialized_example(self) -> None:
|
def test_env_validator_accepts_materialized_example(self) -> None:
|
||||||
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
||||||
@@ -317,15 +371,10 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
self.assertNotIn("currentSchema=", example)
|
self.assertNotIn("currentSchema=", example)
|
||||||
self.assertIn("KEYCLOAK_DB_SCHEMA=keycloak", example)
|
self.assertIn("KEYCLOAK_DB_SCHEMA=keycloak", example)
|
||||||
self.assertIn("HAN_PG_PORT=5433", example)
|
self.assertIn("HAN_PG_PORT=5433", example)
|
||||||
for required in (
|
self.assertIn("SECRETS_SOURCE=file", example)
|
||||||
"CURSOR_HMAC_SECRET=",
|
self.assertNotIn("CURSOR_HMAC_SECRET=", example)
|
||||||
"BITRIX_TOKEN_ENCRYPTION_KEY=",
|
self.assertNotIn("BITRIX_TOKEN_ENCRYPTION_KEY=", example)
|
||||||
"KEYCLOAK_INTERNAL_URL=http://keycloak:8080/auth",
|
materialized = example
|
||||||
"KEYCLOAK_SMS_SERVICE_URL=http://sms-service:8080",
|
|
||||||
"IDGTL_SMS_CALLBACK_PUBLIC_URL=https://chat.example.ru/callbacks/idgtl/sms",
|
|
||||||
):
|
|
||||||
self.assertIn(required, example)
|
|
||||||
materialized = example.replace("change-me", "0123456789abcdef0123456789abcdef")
|
|
||||||
materialized = materialized.replace(
|
materialized = materialized.replace(
|
||||||
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
||||||
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
||||||
@@ -341,11 +390,9 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.assertEqual(result.returncode, 0, result.stderr)
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
|
||||||
def test_env_validator_requires_captcha_keys_only_when_enabled(self) -> None:
|
def test_env_validator_keeps_captcha_server_key_out_of_config(self) -> None:
|
||||||
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
||||||
materialized = example.replace(
|
materialized = example.replace(
|
||||||
"change-me", "0123456789abcdef0123456789abcdef"
|
|
||||||
).replace(
|
|
||||||
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
||||||
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
||||||
)
|
)
|
||||||
@@ -353,33 +400,17 @@ class InfrastructureConfigTests(unittest.TestCase):
|
|||||||
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false",
|
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED=false",
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED=true",
|
"KEYCLOAK_YANDEX_CAPTCHA_ENABLED=true",
|
||||||
)
|
)
|
||||||
enabled_with_keys = enabled_without_keys.replace(
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=",
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY=client-key",
|
|
||||||
).replace(
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=",
|
|
||||||
"KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=server-key-0123456789",
|
|
||||||
)
|
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
env_file = Path(directory) / ".env"
|
env_file = Path(directory) / ".env"
|
||||||
env_file.write_text(enabled_without_keys, encoding="utf-8")
|
env_file.write_text(enabled_without_keys, encoding="utf-8")
|
||||||
missing = subprocess.run(
|
result = subprocess.run(
|
||||||
[sys.executable, str(ROOT / "scripts/validate-env"), str(env_file)],
|
[sys.executable, str(ROOT / "scripts/validate-env"), str(env_file)],
|
||||||
text=True,
|
text=True,
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
check=False,
|
check=False,
|
||||||
)
|
)
|
||||||
env_file.write_text(enabled_with_keys, encoding="utf-8")
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
configured = subprocess.run(
|
self.assertNotIn("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY=", example)
|
||||||
[sys.executable, str(ROOT / "scripts/validate-env"), str(env_file)],
|
|
||||||
text=True,
|
|
||||||
capture_output=True,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
self.assertNotEqual(missing.returncode, 0)
|
|
||||||
self.assertIn("KEYCLOAK_YANDEX_CAPTCHA_CLIENT_KEY", missing.stderr)
|
|
||||||
self.assertIn("KEYCLOAK_YANDEX_CAPTCHA_SERVER_KEY", missing.stderr)
|
|
||||||
self.assertEqual(configured.returncode, 0, configured.stderr)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import runpy
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
VALIDATOR = ROOT / "scripts/validate-env"
|
||||||
|
|
||||||
|
|
||||||
|
def runtime_values() -> dict[str, str]:
|
||||||
|
symbols = runpy.run_path(str(VALIDATOR))
|
||||||
|
values = {key: "a" * 32 for key in symbols["REQUIRED_RUNTIME"]}
|
||||||
|
pg_tail = "?sslmode=verify-full&sslrootcert=/run/secrets/pg-ca.pem"
|
||||||
|
values.update(
|
||||||
|
{
|
||||||
|
"DATABASE_URL": f"postgresql+asyncpg://han_app:password@pg:5433/han_chat{pg_tail}",
|
||||||
|
"BITRIX_DATABASE_URL": f"postgresql://bitrix:password@pg:5433/han_chat{pg_tail}",
|
||||||
|
"BITRIX_SYNC_DATABASE_URL": f"postgresql://sync:password@pg:5433/han_chat{pg_tail}",
|
||||||
|
"MESSAGE_SAFETY_DATABASE_URL": f"postgresql://safety:password@pg:5433/han_chat{pg_tail}",
|
||||||
|
"SMS_DATABASE_URL": f"postgresql+asyncpg://sms:password@pg:5433/han_chat{pg_tail}",
|
||||||
|
"KEYCLOAK_DB_URL": f"jdbc:postgresql://pg:5433/han_chat{pg_tail}",
|
||||||
|
"REDIS_API_PASSWORD": "r" * 32,
|
||||||
|
"REDIS_SAFETY_PASSWORD": "s" * 32,
|
||||||
|
"REDIS_HEALTH_PASSWORD": "h" * 32,
|
||||||
|
"REDIS_URL": f"redis://api_backend:{'r' * 32}@redis:6379/0",
|
||||||
|
"REDIS_REALTIME_URL": f"redis://api_backend:{'r' * 32}@redis:6379/1",
|
||||||
|
"MESSAGE_SAFETY_REDIS_URL": f"redis://message_safety:{'s' * 32}@redis:6379/2",
|
||||||
|
"BITRIX_LOCAL_APP_INTERNAL_TOKEN": "b" * 32,
|
||||||
|
"BITRIX_INTERNAL_API_TOKEN": "b" * 32,
|
||||||
|
"BITRIX_API_FORWARD_TOKEN": "f" * 32,
|
||||||
|
"BITRIX_API_INBOX_TOKEN": "f" * 32,
|
||||||
|
"KEYCLOAK_SMS_SERVICE_TOKEN": "k" * 32,
|
||||||
|
"SMS_SERVICE_TOKEN": "k" * 32,
|
||||||
|
"KEYCLOAK_OTP_MOCK_CODE": "846271",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return values
|
||||||
|
|
||||||
|
|
||||||
|
class SecretHygieneTests(unittest.TestCase):
|
||||||
|
def run_validator(
|
||||||
|
self, config: Path, *args: str, environment: dict[str, str] | None = None
|
||||||
|
) -> subprocess.CompletedProcess[str]:
|
||||||
|
return subprocess.run(
|
||||||
|
[sys.executable, str(VALIDATOR), str(config), *args],
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
env=environment,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_example_is_non_secret_config(self) -> None:
|
||||||
|
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
||||||
|
symbols = runpy.run_path(str(VALIDATOR))
|
||||||
|
for key in symbols["FORBIDDEN_CONFIG_KEYS"]:
|
||||||
|
self.assertNotRegex(example, rf"(?m)^{key}=")
|
||||||
|
self.assertIn("SECRETS_SOURCE=file", example)
|
||||||
|
|
||||||
|
def test_config_rejects_secret_key_and_credential_url(self) -> None:
|
||||||
|
example = (ROOT / ".env.example").read_text(encoding="utf-8")
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
config = Path(directory) / ".env"
|
||||||
|
config.write_text(
|
||||||
|
example
|
||||||
|
+ "\nSMS_SERVICE_TOKEN=not-for-config\n"
|
||||||
|
+ "EXTERNAL_URL=https://user:password@example.net/path\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
result = self.run_validator(config)
|
||||||
|
self.assertNotEqual(result.returncode, 0)
|
||||||
|
self.assertIn("SMS_SERVICE_TOKEN", result.stderr)
|
||||||
|
self.assertIn("URL с credentials", result.stderr)
|
||||||
|
self.assertNotIn("not-for-config", result.stderr)
|
||||||
|
|
||||||
|
def test_runtime_environment_is_validated_without_value_disclosure(self) -> None:
|
||||||
|
values = runtime_values()
|
||||||
|
secret_canary = "canary-never-print-" + "x" * 20
|
||||||
|
values["CURSOR_HMAC_SECRET"] = secret_canary
|
||||||
|
environment = {**os.environ, **values}
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
config = Path(directory) / ".env"
|
||||||
|
config.write_text(
|
||||||
|
(ROOT / ".env.example").read_text(encoding="utf-8").replace(
|
||||||
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
||||||
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
result = self.run_validator(config, "--runtime-env", environment=environment)
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertNotIn(secret_canary, result.stdout + result.stderr)
|
||||||
|
|
||||||
|
def test_runtime_manifest_reads_protected_files_without_value_disclosure(self) -> None:
|
||||||
|
values = runtime_values()
|
||||||
|
secret_canary = "manifest-canary-" + "z" * 20
|
||||||
|
values["CURSOR_HMAC_SECRET"] = secret_canary
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
config = root / ".env"
|
||||||
|
config.write_text(
|
||||||
|
(ROOT / ".env.example").read_text(encoding="utf-8").replace(
|
||||||
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=false",
|
||||||
|
"KEYCLOAK_OTP_MOCK_RISK_ACCEPTED=true",
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
entries = []
|
||||||
|
for index, (key, value) in enumerate(sorted(values.items())):
|
||||||
|
path = root / f"secret-{index}"
|
||||||
|
path.write_text(value + "\n", encoding="utf-8")
|
||||||
|
path.chmod(0o600)
|
||||||
|
entries.append(f"{key}={path.resolve()}")
|
||||||
|
manifest = root / "manifest"
|
||||||
|
manifest.write_text("\n".join(entries) + "\n", encoding="utf-8")
|
||||||
|
result = self.run_validator(config, "--runtime-manifest", str(manifest))
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
self.assertNotIn(secret_canary, result.stdout + result.stderr)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,471 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import importlib.util
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import stat
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
import urllib.error
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
LOADER_PATH = ROOT / "deployment" / "secrets" / "secrets_loader.py"
|
||||||
|
LAUNCHER_PATH = ROOT / "deployment" / "secrets" / "han-secrets"
|
||||||
|
SPEC = importlib.util.spec_from_file_location("han_secrets_loader", LOADER_PATH)
|
||||||
|
assert SPEC and SPEC.loader
|
||||||
|
loader = importlib.util.module_from_spec(SPEC)
|
||||||
|
sys.modules[SPEC.name] = loader
|
||||||
|
SPEC.loader.exec_module(loader)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeResponse:
|
||||||
|
def __init__(
|
||||||
|
self, status: int, body: bytes, headers: dict[str, str] | None = None
|
||||||
|
) -> None:
|
||||||
|
self.status = status
|
||||||
|
self._body = body
|
||||||
|
self.headers = headers or {}
|
||||||
|
|
||||||
|
def __enter__(self) -> "FakeResponse":
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __exit__(self, *args: Any) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def read(self, amount: int) -> bytes:
|
||||||
|
return self._body[:amount]
|
||||||
|
|
||||||
|
|
||||||
|
class FakeOpener:
|
||||||
|
def __init__(self, outcomes: list[Any]) -> None:
|
||||||
|
self.outcomes = outcomes
|
||||||
|
self.requests: list[Any] = []
|
||||||
|
|
||||||
|
def open(self, request: Any, timeout: float) -> FakeResponse:
|
||||||
|
self.requests.append((request, timeout))
|
||||||
|
outcome = self.outcomes.pop(0)
|
||||||
|
if isinstance(outcome, BaseException):
|
||||||
|
raise outcome
|
||||||
|
return outcome
|
||||||
|
|
||||||
|
|
||||||
|
def identity_response(*, project_scoped: bool = True) -> FakeResponse:
|
||||||
|
token: dict[str, Any] = {
|
||||||
|
"catalog": [
|
||||||
|
{
|
||||||
|
"type": "secrets-manager",
|
||||||
|
"endpoints": [
|
||||||
|
{
|
||||||
|
"region": "ru-test",
|
||||||
|
"interface": "public",
|
||||||
|
"url": "https://secrets.example",
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
if project_scoped:
|
||||||
|
token["project"] = {"id": "project-id"}
|
||||||
|
return FakeResponse(
|
||||||
|
201,
|
||||||
|
json.dumps({"token": token}).encode(),
|
||||||
|
{"X-Subject-Token": "iam-token-do-not-log"},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def secret_response(value: bytes) -> FakeResponse:
|
||||||
|
return FakeResponse(
|
||||||
|
200, json.dumps({"value": base64.b64encode(value).decode()}).encode()
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def selectel_config(runtime_dir: Path) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"version": 1,
|
||||||
|
"mode": "selectel",
|
||||||
|
"runtime_dir": str(runtime_dir),
|
||||||
|
"http": {"timeout_seconds": 2, "retries": 1, "max_response_bytes": 4096},
|
||||||
|
"selectel": {
|
||||||
|
"account_id": "123456",
|
||||||
|
"username": "reader",
|
||||||
|
"project_name": "production",
|
||||||
|
"region": "ru-test",
|
||||||
|
"password_file": "selectel-password",
|
||||||
|
},
|
||||||
|
"secrets": {
|
||||||
|
"DATABASE_URL": {
|
||||||
|
"remote": "han/database-url",
|
||||||
|
"consumers": ["api", "migration"],
|
||||||
|
"max_bytes": 512,
|
||||||
|
},
|
||||||
|
"SHARED_TOKEN": {
|
||||||
|
"remote": "han/shared-token",
|
||||||
|
"consumers": ["api"],
|
||||||
|
"max_bytes": 128,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def write_json(path: Path, value: dict[str, Any]) -> None:
|
||||||
|
path.write_text(json.dumps(value), encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def make_private(path: Path) -> None:
|
||||||
|
os.chmod(path, 0o600)
|
||||||
|
|
||||||
|
|
||||||
|
class SecretsLoaderTests(unittest.TestCase):
|
||||||
|
def test_decodes_current_secret_from_nested_selectel_version(self) -> None:
|
||||||
|
encoded = base64.b64encode(b"current-secret").decode()
|
||||||
|
|
||||||
|
value = loader.decode_secret(
|
||||||
|
{"name": "DATABASE_URL", "version": {"version_id": 1, "value": encoded}},
|
||||||
|
"DATABASE_URL",
|
||||||
|
1024,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(value, b"current-secret")
|
||||||
|
|
||||||
|
def test_empty_literal_never_requires_provider_or_fallback_value(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
fallback = root / "fallback.env"
|
||||||
|
fallback.write_text("REQUIRED_TOKEN=required-value\n", encoding="utf-8")
|
||||||
|
make_private(fallback)
|
||||||
|
config = {
|
||||||
|
"version": 1,
|
||||||
|
"mode": "file",
|
||||||
|
"runtime_dir": str(root / "run"),
|
||||||
|
"file": {"path": str(fallback)},
|
||||||
|
"secrets": {
|
||||||
|
"REQUIRED_TOKEN": {
|
||||||
|
"remote": "provider-token",
|
||||||
|
"consumers": ["service"],
|
||||||
|
},
|
||||||
|
"OPTIONAL_HEADER": {
|
||||||
|
"literal": "",
|
||||||
|
"consumers": ["service"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
config_path = root / "file.json"
|
||||||
|
write_json(config_path, config)
|
||||||
|
|
||||||
|
loader.run(config_path, environ={})
|
||||||
|
|
||||||
|
self.assertEqual((root / "run" / "OPTIONAL_HEADER").read_bytes(), b"")
|
||||||
|
self.assertIn(
|
||||||
|
'OPTIONAL_HEADER=""',
|
||||||
|
(root / "run" / "service.env").read_text(encoding="utf-8"),
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_launcher_uses_public_source_switch_without_exporting_values(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
runtime = root / "runtime"
|
||||||
|
fallback = root / "fallback.env"
|
||||||
|
fallback.write_text("TEST_SECRET=canary-secret-value\n", encoding="utf-8")
|
||||||
|
make_private(fallback)
|
||||||
|
public = root / ".env"
|
||||||
|
public.write_text(
|
||||||
|
"SECRETS_SOURCE=file\nAPP_ENV=production\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
loader_config = root / "production.file.json"
|
||||||
|
write_json(
|
||||||
|
loader_config,
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"mode": "file",
|
||||||
|
"runtime_dir": str(runtime),
|
||||||
|
"file": {"path": str(fallback)},
|
||||||
|
"secrets": {
|
||||||
|
"TEST_SECRET": {
|
||||||
|
"remote": "unused-in-file-mode",
|
||||||
|
"consumers": ["test-service"],
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
probe = (
|
||||||
|
"import json,os,pathlib;"
|
||||||
|
"p=os.environ['TEST_SECRET_FILE'];"
|
||||||
|
"print(json.dumps({'raw':os.environ.get('TEST_SECRET'),"
|
||||||
|
"'value':pathlib.Path(p).read_text()}))"
|
||||||
|
)
|
||||||
|
result = subprocess.run(
|
||||||
|
[
|
||||||
|
sys.executable,
|
||||||
|
str(LAUNCHER_PATH),
|
||||||
|
"run",
|
||||||
|
"--config",
|
||||||
|
str(public),
|
||||||
|
"--loader-config",
|
||||||
|
str(loader_config),
|
||||||
|
"--",
|
||||||
|
sys.executable,
|
||||||
|
"-c",
|
||||||
|
probe,
|
||||||
|
],
|
||||||
|
text=True,
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(result.returncode, 0, result.stderr)
|
||||||
|
payload = json.loads(result.stdout)
|
||||||
|
self.assertIsNone(payload["raw"])
|
||||||
|
self.assertEqual(payload["value"], "canary-secret-value")
|
||||||
|
self.assertEqual(
|
||||||
|
(runtime / "manifest").read_text(encoding="utf-8").strip(),
|
||||||
|
f"TEST_SECRET={(runtime / 'TEST_SECRET').resolve()}",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_selectel_flow_uses_project_scope_catalog_and_per_service_files(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
credentials = root / "credentials"
|
||||||
|
credentials.mkdir()
|
||||||
|
(credentials / "selectel-password").write_text(
|
||||||
|
"service-user-password\n", encoding="utf-8"
|
||||||
|
)
|
||||||
|
make_private(credentials / "selectel-password")
|
||||||
|
runtime = root / "run"
|
||||||
|
config_path = root / "config.json"
|
||||||
|
write_json(config_path, selectel_config(runtime))
|
||||||
|
opener = FakeOpener(
|
||||||
|
[
|
||||||
|
identity_response(),
|
||||||
|
secret_response(b"postgresql://user:password@db/app"),
|
||||||
|
secret_response(b'token with spaces and "quotes"'),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
def factory(**kwargs: Any) -> Any:
|
||||||
|
return loader.HTTPClient(
|
||||||
|
**kwargs, opener=opener, sleeper=lambda _: None, jitter=lambda: 0
|
||||||
|
)
|
||||||
|
|
||||||
|
consumers = loader.run(
|
||||||
|
config_path,
|
||||||
|
environ={"CREDENTIALS_DIRECTORY": str(credentials)},
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(consumers, ["api", "migration"])
|
||||||
|
auth_request = opener.requests[0][0]
|
||||||
|
auth_payload = json.loads(auth_request.data)
|
||||||
|
self.assertEqual(
|
||||||
|
auth_payload["auth"]["scope"]["project"]["name"], "production"
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
auth_payload["auth"]["identity"]["password"]["user"]["domain"]["name"],
|
||||||
|
"123456",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
opener.requests[1][0].get_header("X-auth-token"),
|
||||||
|
"iam-token-do-not-log",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
opener.requests[1][0].full_url,
|
||||||
|
"https://secrets.example/v1/han%2Fdatabase-url",
|
||||||
|
)
|
||||||
|
api_text = (runtime / "api.env").read_text(encoding="utf-8")
|
||||||
|
migration_text = (runtime / "migration.env").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('DATABASE_URL="postgresql://user:password@db/app"', api_text)
|
||||||
|
self.assertIn(
|
||||||
|
'SHARED_TOKEN="token with spaces and \\"quotes\\""', api_text
|
||||||
|
)
|
||||||
|
self.assertNotIn("SHARED_TOKEN", migration_text)
|
||||||
|
if os.name != "nt":
|
||||||
|
self.assertEqual(stat.S_IMODE((runtime / "api.env").stat().st_mode), 0o600)
|
||||||
|
self.assertEqual(stat.S_IMODE(runtime.stat().st_mode), 0o700)
|
||||||
|
|
||||||
|
def test_retry_uses_jitter_and_then_succeeds(self) -> None:
|
||||||
|
opener = FakeOpener(
|
||||||
|
[urllib.error.URLError("temporary"), FakeResponse(200, b"{}")]
|
||||||
|
)
|
||||||
|
sleeps: list[float] = []
|
||||||
|
client = loader.HTTPClient(
|
||||||
|
timeout=1,
|
||||||
|
retries=1,
|
||||||
|
max_response_bytes=1024,
|
||||||
|
opener=opener,
|
||||||
|
sleeper=sleeps.append,
|
||||||
|
jitter=lambda: 0.25,
|
||||||
|
)
|
||||||
|
result = client.request(
|
||||||
|
"GET", "https://provider.example/v1/key", expected=frozenset({200})
|
||||||
|
)
|
||||||
|
self.assertEqual(result.status, 200)
|
||||||
|
self.assertEqual(sleeps, [0.1875])
|
||||||
|
|
||||||
|
def test_paired_canonical_names_fetch_provider_version_once(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
credentials = root / "credentials"
|
||||||
|
credentials.mkdir()
|
||||||
|
credential = credentials / "selectel-password"
|
||||||
|
credential.write_text("password", encoding="utf-8")
|
||||||
|
make_private(credential)
|
||||||
|
config = selectel_config(root / "run")
|
||||||
|
config["secrets"]["SHARED_TOKEN"]["remote"] = "han/database-url"
|
||||||
|
config_path = root / "config.json"
|
||||||
|
write_json(config_path, config)
|
||||||
|
opener = FakeOpener([identity_response(), secret_response(b"same-value")])
|
||||||
|
|
||||||
|
def factory(**kwargs: Any) -> Any:
|
||||||
|
return loader.HTTPClient(**kwargs, opener=opener)
|
||||||
|
|
||||||
|
loader.run(
|
||||||
|
config_path,
|
||||||
|
environ={"CREDENTIALS_DIRECTORY": str(credentials)},
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
self.assertEqual(len(opener.requests), 2)
|
||||||
|
output = (root / "run" / "api.env").read_text(encoding="utf-8")
|
||||||
|
self.assertIn('DATABASE_URL="same-value"', output)
|
||||||
|
self.assertIn('SHARED_TOKEN="same-value"', output)
|
||||||
|
|
||||||
|
def test_http_error_is_redacted_and_body_is_not_read(self) -> None:
|
||||||
|
leaked = b"postgresql://admin:secret@db/app iam-token response-body"
|
||||||
|
|
||||||
|
class ExplodingBody(io.BytesIO):
|
||||||
|
def read(self, *args: Any, **kwargs: Any) -> bytes:
|
||||||
|
raise AssertionError("HTTP error body must not be read")
|
||||||
|
|
||||||
|
error = urllib.error.HTTPError(
|
||||||
|
"https://provider.example/v1/key",
|
||||||
|
403,
|
||||||
|
"body contains a secret",
|
||||||
|
{},
|
||||||
|
ExplodingBody(leaked),
|
||||||
|
)
|
||||||
|
client = loader.HTTPClient(
|
||||||
|
timeout=1,
|
||||||
|
retries=0,
|
||||||
|
max_response_bytes=1024,
|
||||||
|
opener=FakeOpener([error]),
|
||||||
|
)
|
||||||
|
with self.assertRaises(loader.LoaderError) as caught:
|
||||||
|
client.request(
|
||||||
|
"GET",
|
||||||
|
"https://provider.example/v1/key",
|
||||||
|
headers={"X-Auth-Token": "iam-token"},
|
||||||
|
expected=frozenset({200}),
|
||||||
|
)
|
||||||
|
message = str(caught.exception)
|
||||||
|
self.assertEqual(message, "provider request failed with HTTP 403")
|
||||||
|
for forbidden in ("secret", "iam-token", "response-body", "postgresql://"):
|
||||||
|
self.assertNotIn(forbidden, message)
|
||||||
|
|
||||||
|
def test_unscoped_token_fails_without_replacing_existing_output(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
credentials = root / "credentials"
|
||||||
|
credentials.mkdir()
|
||||||
|
(credentials / "selectel-password").write_text("password", encoding="utf-8")
|
||||||
|
make_private(credentials / "selectel-password")
|
||||||
|
runtime = root / "run"
|
||||||
|
runtime.mkdir()
|
||||||
|
existing = runtime / "api.env"
|
||||||
|
existing.write_text('DATABASE_URL="old-value"\n', encoding="utf-8")
|
||||||
|
config_path = root / "config.json"
|
||||||
|
write_json(config_path, selectel_config(runtime))
|
||||||
|
opener = FakeOpener([identity_response(project_scoped=False)])
|
||||||
|
|
||||||
|
def factory(**kwargs: Any) -> Any:
|
||||||
|
return loader.HTTPClient(**kwargs, opener=opener)
|
||||||
|
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "not project-scoped"):
|
||||||
|
loader.run(
|
||||||
|
config_path,
|
||||||
|
environ={"CREDENTIALS_DIRECTORY": str(credentials)},
|
||||||
|
client_factory=factory,
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
existing.read_text(encoding="utf-8"),
|
||||||
|
'DATABASE_URL="old-value"\n',
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_invalid_base64_and_size_limit_fail_closed(self) -> None:
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "invalid base64"):
|
||||||
|
loader.decode_secret({"value": "not-base64!"}, "TOKEN", 128)
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "configured limit"):
|
||||||
|
loader.decode_secret(
|
||||||
|
{"value": base64.b64encode(b"too-long").decode()}, "TOKEN", 3
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "dotenv"):
|
||||||
|
loader.decode_secret(
|
||||||
|
{"value": base64.b64encode(b"line1\nline2").decode()}, "TOKEN", 128
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_file_mode_is_explicit_strict_and_narrow(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
source = root / "fallback.env"
|
||||||
|
source.write_text(
|
||||||
|
"# exact recovery set\n"
|
||||||
|
'DATABASE_URL="postgresql://user:pass@db/app"\n'
|
||||||
|
"SHARED_TOKEN='literal value'\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
make_private(source)
|
||||||
|
runtime = root / "run"
|
||||||
|
config = selectel_config(runtime)
|
||||||
|
config["mode"] = "file"
|
||||||
|
config.pop("selectel")
|
||||||
|
config.pop("http")
|
||||||
|
config["file"] = {"path": str(source)}
|
||||||
|
config_path = root / "config.json"
|
||||||
|
write_json(config_path, config)
|
||||||
|
loader.run(config_path, environ={})
|
||||||
|
output = (runtime / "api.env").read_text(encoding="utf-8")
|
||||||
|
self.assertIn("literal value", output)
|
||||||
|
self.assertNotIn("selectel", output)
|
||||||
|
|
||||||
|
source.write_text(
|
||||||
|
"DATABASE_URL=ok\nSHARED_TOKEN=ok\nUNDECLARED=leak\n",
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "undeclared key"):
|
||||||
|
loader.run(config_path, environ={})
|
||||||
|
|
||||||
|
def test_selectel_failure_never_falls_back_to_file_section(self) -> None:
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
config = selectel_config(root / "run")
|
||||||
|
config["file"] = {"path": str(root / "fallback.env")}
|
||||||
|
config_path = root / "config.json"
|
||||||
|
write_json(config_path, config)
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "forbidden"):
|
||||||
|
loader.run(config_path, environ={})
|
||||||
|
|
||||||
|
def test_relative_credential_requires_systemd_directory(self) -> None:
|
||||||
|
selectel = {"password_file": "credential"}
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "CREDENTIALS_DIRECTORY"):
|
||||||
|
loader.credential_value(selectel, {})
|
||||||
|
|
||||||
|
def test_response_content_length_limit_is_enforced(self) -> None:
|
||||||
|
client = loader.HTTPClient(
|
||||||
|
timeout=1,
|
||||||
|
retries=0,
|
||||||
|
max_response_bytes=10,
|
||||||
|
opener=FakeOpener(
|
||||||
|
[FakeResponse(200, b"{}", {"Content-Length": "100"})]
|
||||||
|
),
|
||||||
|
)
|
||||||
|
with self.assertRaisesRegex(loader.LoaderError, "exceeds"):
|
||||||
|
client.request(
|
||||||
|
"GET", "https://provider.example/v1/key", expected=frozenset({200})
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -344,7 +344,7 @@ git status --short
|
|||||||
- [ ] SBOM/vulnerability scan без unresolved critical/high.
|
- [ ] SBOM/vulnerability scan без unresolved critical/high.
|
||||||
- [ ] Root Compose — единственный production entrypoint.
|
- [ ] Root Compose — единственный production entrypoint.
|
||||||
|
|
||||||
## 9. Stage 6 — `.env` и secrets
|
## 9. Stage 6 — несекретный `.env` и runtime secrets
|
||||||
|
|
||||||
### 9.1. Создание
|
### 9.1. Создание
|
||||||
|
|
||||||
@@ -355,19 +355,24 @@ cp .env.example .env
|
|||||||
chmod 600 .env
|
chmod 600 .env
|
||||||
```
|
```
|
||||||
|
|
||||||
Генерировать минимум 256-bit:
|
`.env` содержит только несекретный config и `SECRETS_SOURCE=file|selectel`.
|
||||||
|
Пароли, токены, access keys, credential-bearing DSN и `*_FILE` пути в нём
|
||||||
|
запрещены. Секреты выдаёт единый интерфейс:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
openssl rand -hex 32
|
deployment/secrets/han-secrets run --config .env -- <command>
|
||||||
```
|
```
|
||||||
|
|
||||||
Не выполнять `export SECRET=...` в shared shell history. Использовать editor с restricted permissions или secret manager/secret files.
|
Launcher устанавливает `HAN_SECRETS_ACTIVE=1`, не выводит значения и может
|
||||||
|
передать paths-only manifest через `HAN_RUNTIME_SECRET_MANIFEST`. Файлы manifest
|
||||||
|
должны быть абсолютными, недоступными group/other. Генерацию выполняет secret
|
||||||
|
manager; не выполнять `export SECRET=...` и не вставлять значения в history.
|
||||||
|
|
||||||
### 9.2. Обязательные группы
|
### 9.2. Обязательные группы
|
||||||
|
|
||||||
- `APP_ENV`, release/version, log level;
|
- `APP_ENV`, release/version, log level;
|
||||||
- private PG host/port/database, TLS CA и runtime DSN;
|
- private PG host/port/database и TLS CA в config; runtime DSN в secret backend;
|
||||||
- Redis ACL credentials/URLs DB0/1/2;
|
- Redis ACL credentials/URLs DB0/1/2 только в secret backend;
|
||||||
- public web/API/auth URLs;
|
- public web/API/auth URLs;
|
||||||
- Keycloak realm/audience/hostname/bootstrap/provider technical secrets;
|
- Keycloak realm/audience/hostname/bootstrap/provider technical secrets;
|
||||||
- SMS DB URL, парные Keycloak↔SMS tokens, Direct `TOKEN_1`, callback URL и отдельные callback credentials;
|
- SMS DB URL, парные Keycloak↔SMS tokens, Direct `TOKEN_1`, callback URL и отдельные callback credentials;
|
||||||
@@ -390,9 +395,10 @@ Service token и webhook token — разные secrets.
|
|||||||
|
|
||||||
### 9.3. Validation
|
### 9.3. Validation
|
||||||
|
|
||||||
Добавить/запустить `scripts/validate-env`:
|
Запустить `scripts/validate-env` в двух независимых режимах:
|
||||||
|
|
||||||
- mandatory not empty;
|
- config: mandatory not empty, `SECRETS_SOURCE`, запрет secret keys/DSN credentials;
|
||||||
|
- runtime: manifest/files или child environment без печати значений;
|
||||||
- нет `change-me`, example IP/domain, default OTP;
|
- нет `change-me`, example IP/domain, default OTP;
|
||||||
- URLs have correct schemes;
|
- URLs have correct schemes;
|
||||||
- public URLs HTTPS, internal URLs service DNS;
|
- public URLs HTTPS, internal URLs service DNS;
|
||||||
@@ -409,14 +415,18 @@ Service token и webhook token — разные secrets.
|
|||||||
```bash
|
```bash
|
||||||
cd <BACKEND_ROOT>
|
cd <BACKEND_ROOT>
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
docker compose config --quiet
|
sudo systemctl restart han-secrets@production.service
|
||||||
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
```
|
```
|
||||||
|
|
||||||
`docker compose config` может раскрыть resolved secrets; не сохранять/публиковать его stdout.
|
`docker compose config` может раскрыть resolved secrets; не сохранять/публиковать его stdout.
|
||||||
|
|
||||||
### Gate 6
|
### Gate 6
|
||||||
|
|
||||||
- [ ] `.env` mode 0600, отсутствует в git.
|
- [ ] `.env` отсутствует в git и содержит только несекретный config.
|
||||||
|
- [ ] Secret launcher и permissions manifest/files проверены.
|
||||||
- [ ] Все placeholder/default secrets отклонены.
|
- [ ] Все placeholder/default secrets отклонены.
|
||||||
- [ ] Paired tokens совпадают.
|
- [ ] Paired tokens совпадают.
|
||||||
- [ ] DSN private/TLS; URLs/issuer согласованы.
|
- [ ] DSN private/TLS; URLs/issuer согласованы.
|
||||||
@@ -1000,10 +1010,13 @@ DB backup включает realm/users/signing keys/provider data. Secret-free r
|
|||||||
4. остановить новые claims/send при возможности;
|
4. остановить новые claims/send при возможности;
|
||||||
5. переключить image tags на previous digests;
|
5. переключить image tags на previous digests;
|
||||||
6. не выполнять Alembic downgrade;
|
6. не выполнять Alembic downgrade;
|
||||||
7. `docker compose up -d`;
|
7. `SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true deployment/scripts/rollback.sh <PREVIOUS_IMMUTABLE_RELEASE>`;
|
||||||
8. health/smoke/idempotency;
|
8. health/smoke/idempotency;
|
||||||
9. проверить outbox/inbox/SMS pending/uncertain/recovery.
|
9. проверить outbox/inbox/SMS pending/uncertain/recovery.
|
||||||
|
|
||||||
|
Используется текущий runtime secret set и текущий несекретный config. Snapshot
|
||||||
|
старого `.env` не создаётся и не применяется.
|
||||||
|
|
||||||
### После backward-incompatible migration
|
### После backward-incompatible migration
|
||||||
|
|
||||||
Обычный rollback запрещён. Выбор:
|
Обычный rollback запрещён. Выбор:
|
||||||
|
|||||||
@@ -2,13 +2,23 @@
|
|||||||
# diagnose-han-chat.sh — запускать из /opt/han-chat/backend
|
# diagnose-han-chat.sh — запускать из /opt/han-chat/backend
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
cd /opt/han-chat/backend
|
cd /opt/han-chat/backend
|
||||||
|
CONFIG_FILE="${CONFIG_FILE:-.env}"
|
||||||
|
SECRETS_LAUNCHER="${SECRETS_LAUNCHER:-deployment/secrets/han-secrets}"
|
||||||
|
if [[ "${HAN_SECRETS_ACTIVE:-0}" != 1 ]]; then
|
||||||
|
[[ -x "$SECRETS_LAUNCHER" ]] || {
|
||||||
|
echo "Secret launcher is required: $SECRETS_LAUNCHER" >&2
|
||||||
|
exit 66
|
||||||
|
}
|
||||||
|
exec "$SECRETS_LAUNCHER" run --config "$CONFIG_FILE" -- "$0" "$@"
|
||||||
|
fi
|
||||||
|
compose() { docker compose --env-file "$CONFIG_FILE" "$@"; }
|
||||||
|
|
||||||
echo "=== 1. Статус сервисов ==="
|
echo "=== 1. Статус сервисов ==="
|
||||||
docker compose --env-file .env ps api-backend keycloak redis message-safety
|
compose ps api-backend keycloak redis message-safety
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "=== 2. Healthcheck api-backend (что именно падает) ==="
|
echo "=== 2. Healthcheck api-backend (что именно падает) ==="
|
||||||
docker compose --env-file .env exec -T api-backend \
|
compose exec -T api-backend \
|
||||||
python -c "
|
python -c "
|
||||||
import json, urllib.request
|
import json, urllib.request
|
||||||
try:
|
try:
|
||||||
@@ -22,13 +32,13 @@ except urllib.error.HTTPError as e:
|
|||||||
|
|
||||||
echo
|
echo
|
||||||
echo "=== 3. Keycloak health (management port 9000) ==="
|
echo "=== 3. Keycloak health (management port 9000) ==="
|
||||||
docker compose --env-file .env exec -T keycloak bash -c \
|
compose exec -T keycloak bash -c \
|
||||||
"exec 3<>/dev/tcp/127.0.0.1/9000 && printf 'GET /auth/health/ready HTTP/1.0\r\n\r\n' >&3 && cat <&3" \
|
"exec 3<>/dev/tcp/127.0.0.1/9000 && printf 'GET /auth/health/ready HTTP/1.0\r\n\r\n' >&3 && cat <&3" \
|
||||||
2>/dev/null | head -20 || echo "Keycloak health FAILED"
|
2>/dev/null | head -20 || echo "Keycloak health FAILED"
|
||||||
|
|
||||||
echo
|
echo
|
||||||
echo "=== 4. api-backend -> Keycloak (JWKS) ==="
|
echo "=== 4. api-backend -> Keycloak (JWKS) ==="
|
||||||
docker compose --env-file .env exec -T api-backend \
|
compose exec -T api-backend \
|
||||||
python -c "
|
python -c "
|
||||||
import os, urllib.request, json
|
import os, urllib.request, json
|
||||||
base = os.environ.get('KEYCLOAK_INTERNAL_URL','http://keycloak:8080/auth').rstrip('/')
|
base = os.environ.get('KEYCLOAK_INTERNAL_URL','http://keycloak:8080/auth').rstrip('/')
|
||||||
|
|||||||
+30
-59
@@ -5,41 +5,25 @@
|
|||||||
cd /opt/han-chat/backend
|
cd /opt/han-chat/backend
|
||||||
umask 077
|
umask 077
|
||||||
|
|
||||||
read -r -p "Тестовый номер в E.164 (+79...): " TEST_PHONE
|
|
||||||
|
|
||||||
CHALLENGE_ID=$(python3 -c 'import uuid; print(uuid.uuid4())')
|
|
||||||
OTP_CODE=$(python3 -c 'import secrets; print(f"{secrets.randbelow(1000000):06d}")')
|
|
||||||
|
|
||||||
SMS_TOKEN=$(python3 - <<'PY'
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
for line in Path(".env").read_text().splitlines():
|
|
||||||
if line.startswith("SMS_SERVICE_TOKEN="):
|
|
||||||
print(line.split("=", 1)[1].strip().strip("\"'"))
|
|
||||||
break
|
|
||||||
else:
|
|
||||||
raise SystemExit("SMS_SERVICE_TOKEN отсутствует")
|
|
||||||
PY
|
|
||||||
)
|
|
||||||
|
|
||||||
export TEST_PHONE CHALLENGE_ID OTP_CODE SMS_TOKEN
|
|
||||||
REQUEST_FILE=$(mktemp)
|
REQUEST_FILE=$(mktemp)
|
||||||
|
read -r -p "Тестовый номер в E.164 (+79...): " TEST_PHONE
|
||||||
python3 - "$REQUEST_FILE" <<'PY'
|
python3 - "$REQUEST_FILE" "$TEST_PHONE" <<'PY'
|
||||||
import json
|
import json
|
||||||
import os
|
import secrets
|
||||||
import sys
|
import sys
|
||||||
|
import uuid
|
||||||
|
|
||||||
|
challenge_id = str(uuid.uuid4())
|
||||||
payload = {
|
payload = {
|
||||||
"idempotency_key": f"ops:smoke:{os.environ['CHALLENGE_ID']}",
|
"idempotency_key": f"ops:smoke:{challenge_id}",
|
||||||
"template_code": "auth_otp",
|
"template_code": "auth_otp",
|
||||||
"locale": "ru",
|
"locale": "ru",
|
||||||
"phone_e164": os.environ["TEST_PHONE"],
|
"phone_e164": sys.argv[2],
|
||||||
"substitutions": {
|
"substitutions": {
|
||||||
"code": os.environ["OTP_CODE"],
|
"code": f"{secrets.randbelow(1000000):06d}",
|
||||||
"ttl_min": "1",
|
"ttl_min": "1",
|
||||||
},
|
},
|
||||||
"customer_ref": os.environ["CHALLENGE_ID"],
|
"customer_ref": challenge_id,
|
||||||
"message_ttl_sec": 60,
|
"message_ttl_sec": 60,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -51,16 +35,20 @@ PY
|
|||||||
## Создайте функцию отправки:
|
## Создайте функцию отправки:
|
||||||
|
|
||||||
send_sms_smoke() {
|
send_sms_smoke() {
|
||||||
|
deployment/secrets/han-secrets run --config .env -- \
|
||||||
docker compose --env-file .env --profile ops run --rm --no-deps \
|
docker compose --env-file .env --profile ops run --rm --no-deps \
|
||||||
--user 0:0 \
|
--user 0:0 \
|
||||||
--entrypoint sh \
|
--entrypoint sh \
|
||||||
-e SMS_TOKEN \
|
-v /run/han-chat/secrets/SMS_SERVICE_TOKEN:/run/secrets/sms_service_token:ro \
|
||||||
-v "$REQUEST_FILE:/tmp/sms-request.json:ro" \
|
-v "$REQUEST_FILE:/tmp/sms-request.json:ro" \
|
||||||
toolbox -ec '
|
toolbox -ec '
|
||||||
|
umask 077
|
||||||
|
printf "Authorization: Bearer %s\n" \
|
||||||
|
"$(cat /run/secrets/sms_service_token)" > /tmp/auth-header
|
||||||
curl -sS \
|
curl -sS \
|
||||||
-w "\nHTTP %{http_code}\n" \
|
-w "\nHTTP %{http_code}\n" \
|
||||||
-X POST \
|
-X POST \
|
||||||
-H "Authorization: Bearer $SMS_TOKEN" \
|
-H @/tmp/auth-header \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-H "X-Request-ID: ops-sms-smoke" \
|
-H "X-Request-ID: ops-sms-smoke" \
|
||||||
--data-binary @/tmp/sms-request.json \
|
--data-binary @/tmp/sms-request.json \
|
||||||
@@ -90,7 +78,7 @@ LIMIT 5;
|
|||||||
|
|
||||||
## После проверки удалите секретные данные:
|
## После проверки удалите секретные данные:
|
||||||
shred -u "$REQUEST_FILE" 2>/dev/null || rm -f "$REQUEST_FILE"
|
shred -u "$REQUEST_FILE" 2>/dev/null || rm -f "$REQUEST_FILE"
|
||||||
unset SMS_TOKEN OTP_CODE TEST_PHONE CHALLENGE_ID REQUEST_FILE
|
unset TEST_PHONE REQUEST_FILE
|
||||||
|
|
||||||
# Тесты
|
# Тесты
|
||||||
|
|
||||||
@@ -138,33 +126,12 @@ HTTP 403
|
|||||||
|
|
||||||
### 3. Проверить Basic auth внутри Docker-сети
|
### 3. Проверить Basic auth внутри Docker-сети
|
||||||
|
|
||||||
Получите credentials из `.env`:
|
Credentials получает только дочерний процесс через secret launcher.
|
||||||
|
|
||||||
```bash
|
|
||||||
CB_USER=$(python3 - <<'PY'
|
|
||||||
from pathlib import Path
|
|
||||||
for line in Path(".env").read_text().splitlines():
|
|
||||||
if line.startswith("IDGTL_SMS_CALLBACK_USERNAME="):
|
|
||||||
print(line.split("=", 1)[1].strip().strip("\"'"))
|
|
||||||
break
|
|
||||||
PY
|
|
||||||
)
|
|
||||||
|
|
||||||
CB_PASS=$(python3 - <<'PY'
|
|
||||||
from pathlib import Path
|
|
||||||
for line in Path(".env").read_text().splitlines():
|
|
||||||
if line.startswith("IDGTL_SMS_CALLBACK_PASSWORD="):
|
|
||||||
print(line.split("=", 1)[1].strip().strip("\"'"))
|
|
||||||
break
|
|
||||||
PY
|
|
||||||
)
|
|
||||||
|
|
||||||
export CB_USER CB_PASS
|
|
||||||
```
|
|
||||||
|
|
||||||
Неверные credentials:
|
Неверные credentials:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
deployment/secrets/han-secrets run --config .env -- \
|
||||||
docker compose --env-file .env --profile ops run --rm --no-deps \
|
docker compose --env-file .env --profile ops run --rm --no-deps \
|
||||||
--entrypoint sh toolbox -ec '
|
--entrypoint sh toolbox -ec '
|
||||||
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
|
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
|
||||||
@@ -180,10 +147,20 @@ docker compose --env-file .env --profile ops run --rm --no-deps \
|
|||||||
Правильные credentials:
|
Правильные credentials:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
deployment/secrets/han-secrets run --config .env -- \
|
||||||
docker compose --env-file .env --profile ops run --rm --no-deps \
|
docker compose --env-file .env --profile ops run --rm --no-deps \
|
||||||
--entrypoint sh -e CB_USER -e CB_PASS toolbox -ec '
|
--entrypoint sh \
|
||||||
|
-v /run/han-chat/secrets/IDGTL_SMS_CALLBACK_USERNAME:/run/secrets/callback_user:ro \
|
||||||
|
-v /run/han-chat/secrets/IDGTL_SMS_CALLBACK_PASSWORD:/run/secrets/callback_password:ro \
|
||||||
|
toolbox -ec '
|
||||||
|
umask 077
|
||||||
|
{
|
||||||
|
printf "user = \"%s:%s\"\n" \
|
||||||
|
"$(cat /run/secrets/callback_user)" \
|
||||||
|
"$(cat /run/secrets/callback_password)"
|
||||||
|
} > /tmp/curl-auth.conf
|
||||||
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
|
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
|
||||||
-u "$CB_USER:$CB_PASS" \
|
--config /tmp/curl-auth.conf \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
--data "[]" \
|
--data "[]" \
|
||||||
http://sms-service:8080/callbacks/idgtl/sms
|
http://sms-service:8080/callbacks/idgtl/sms
|
||||||
@@ -192,12 +169,6 @@ docker compose --env-file .env --profile ops run --rm --no-deps \
|
|||||||
|
|
||||||
Ожидается `HTTP 422`: авторизация прошла, но пустой callback-массив невалиден.
|
Ожидается `HTTP 422`: авторизация прошла, но пустой callback-массив невалиден.
|
||||||
|
|
||||||
После проверки:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
unset CB_USER CB_PASS
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Проверить реальный callback Direct
|
### 4. Проверить реальный callback Direct
|
||||||
|
|
||||||
После тестовой SMS:
|
После тестовой SMS:
|
||||||
|
|||||||
+92
-530
@@ -1,555 +1,117 @@
|
|||||||
# usefull commands
|
# Release #0: безопасный порядок развертывания
|
||||||
- Убрать переносы строк: sed -i 's/\r$//' name-file.sh
|
|
||||||
- find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} +
|
|
||||||
- docker compose stop # мягко останавливает контейнеры (не удаляет)
|
|
||||||
- sudo shutdown -h now # выглючить ВМ
|
|
||||||
- Генерация паролей
|
|
||||||
- (со спец.символами) openssl rand -base64 32
|
|
||||||
- (без спец.символов) openssl rand -hex 16
|
|
||||||
- openssl rand -base64 16 | xclip -selection clipboard # Linux
|
|
||||||
|
|
||||||
Посмотреть состояние контейнеров
|
Этот файл больше не является журналом реальных адресов, SSH-ключей и локальных
|
||||||
docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}"
|
путей. Фактические значения инфраструктуры хранятся в защищённой CMDB/ops wiki,
|
||||||
|
а не в Git.
|
||||||
|
|
||||||
Туннель до БД: ssh -i C:\Users\MI\.ssh\hansel -L 5433:192.168.0.211:5432 root@135.106.164.58 -N
|
## 1. Подготовка ВМ
|
||||||
|
|
||||||
#Обновление проекта
|
Разрешите извне только 80/443 и SSH из административной сети. PostgreSQL
|
||||||
mkdir -p ~/.ssh
|
доступен ВМ только через приватную сеть.
|
||||||
cp /mnt/c/Users/MI/.ssh/hansel ~/.ssh/hansel
|
|
||||||
chmod 600 ~/.ssh/hansel
|
|
||||||
|
|
||||||
1. Скопировать и автоматически почистить артефакты
|
```sh
|
||||||
Когда вы убедитесь, что вывод предыдущей команды вас устраивает, просто уберите флаг -n:
|
scp -i <DEPLOY_SSH_KEY> deployment/scripts/setup-vm.sh \
|
||||||
rsync -rltD --no-perms --no-owner --no-group -ivc --delete \
|
<CLOUD_USER>@<VM_IP>:/tmp/setup-vm.sh
|
||||||
--exclude='.env' \
|
ssh -i <DEPLOY_SSH_KEY> <CLOUD_USER>@<VM_IP>
|
||||||
--exclude='*.crt' \
|
sudo chmod 0755 /tmp/setup-vm.sh
|
||||||
--exclude='*.pem' \
|
|
||||||
--exclude='*.key' \
|
|
||||||
--exclude='secrets/' \
|
|
||||||
--exclude='node_modules/' \
|
|
||||||
--exclude='dist/' \
|
|
||||||
--exclude='.expo/' \
|
|
||||||
--exclude='playwright-report/' \
|
|
||||||
--exclude='test-results/' \
|
|
||||||
--exclude='*.log' \
|
|
||||||
-e "ssh -i ~/.ssh/hansel" \
|
|
||||||
/mnt/c/Users/MI/Documents/Assistent/HAN_chat_specification/codebase/backend/ \
|
|
||||||
root@135.106.164.58:/opt/han-chat/backend/
|
|
||||||
|
|
||||||
-r — рекурсивно.
|
|
||||||
-l — сохранять символические ссылки.
|
|
||||||
-t — сохранять время модификации (важно для будущих проверок).
|
|
||||||
-D — сохранять устройства (на всякий случай, как в -a).
|
|
||||||
--no-perms --no-owner --no-group — главное исправление: не пытаться копировать права, владельца и группу с Windows на Linux. Это избавит от ложных срабатываний.
|
|
||||||
-i — покажет только реально измененные файлы (можно заменить на -v, если хотите просто список).
|
|
||||||
-a (archive) — сохраняет права, время и рекурсивно копирует.
|
|
||||||
-v (verbose) — выводит список файлов.
|
|
||||||
-n (dry-run) — главный флаг, показывает, что бы произошло, но не делает этого.
|
|
||||||
--delete — решение вашей проблемы. Говорит rsync удалять на приемнике (ВМ) файлы, которых нет в источнике (локально).
|
|
||||||
Важно: не забудьте поставить слэш / в конце пути к локальному проекту, иначе rsync скопирует саму папку внутрь папки на ВМ.
|
|
||||||
|
|
||||||
2. Копирование env (опционально)
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/backend
|
|
||||||
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} +
|
|
||||||
chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh
|
|
||||||
|
|
||||||
docker compose --env-file .env build frontend-static keycloak
|
|
||||||
docker compose --env-file .env up -d \
|
|
||||||
--no-deps \
|
|
||||||
--force-recreate frontend-static keycloak
|
|
||||||
|
|
||||||
Посмотреть состояние контейнеров
|
|
||||||
docker compose ps --format "table {{.Service}}\t{{.Status}}"
|
|
||||||
docker compose ps --format "table {{.Service}}\t{{.Status}}\t{{.Ports}}"
|
|
||||||
|
|
||||||
# Архивный способ копирования:
|
|
||||||
|
|
||||||
cd /tmp
|
|
||||||
rm han-chat-backend.tar.gz
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
|
|
||||||
#команда складывает архив в ту папку, из которой запускается команда
|
|
||||||
cd C:\Users\MI\Documents\Assistent\
|
|
||||||
rm C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz
|
|
||||||
tar -C C:\Users\MI\Documents\Assistent\HAN_chat_specification\codebase\backend -czf han-chat-backend.tar.gz .
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz root@135.106.164.58:/tmp/han-chat-backend.tar.gz
|
|
||||||
|
|
||||||
tar -xzf /tmp/han-chat-backend.tar.gz
|
|
||||||
find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} +
|
|
||||||
chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh
|
|
||||||
|
|
||||||
docker compose --env-file .env build frontend-static keycloak
|
|
||||||
docker compose --env-file .env up -d \
|
|
||||||
--no-deps \
|
|
||||||
--force-recreate frontend-static keycloak
|
|
||||||
|
|
||||||
# Разворачиваем инфраструктуру в Селектел ч1
|
|
||||||
|
|
||||||
## Создание сети
|
|
||||||
|
|
||||||
## Создание групп безопасности для сети и для БД
|
|
||||||
Для сети открываем порты 80, 22, 443
|
|
||||||
Для БД открываем порты 5432, 5433 но только из CIDR (192.168.0.211\24)
|
|
||||||
|
|
||||||
## Создаем ВМ, подключаем к новой сети (через порт) и перезагружаем (либо руками вводить новые сетевые настройки - надо выбрать)
|
|
||||||
|
|
||||||
# Подключение к ВМ и первичный скрипт
|
|
||||||
++ ssh -i C:\Users\MI\.ssh\hansel root@135.106.164.58
|
|
||||||
++ scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\Assistent\HAN_chat_specification\codebase\backend\deployment\scripts\setup-vm.sh" root@135.106.164.58:/tmp/setup-vm.sh
|
|
||||||
chmod +x /tmp/setup-vm.sh
|
|
||||||
sudo /tmp/setup-vm.sh
|
sudo /tmp/setup-vm.sh
|
||||||
|
```
|
||||||
|
|
||||||
# Скрипт создал деплой пользователя. Нужно Перенести туда публичный ключ и дать права на файл и папку
|
После создания пользователя `deploy` проверьте отдельную SSH-сессию до
|
||||||
|
отключения root login. Не копируйте приватный ключ на ВМ.
|
||||||
|
|
||||||
## Вставляем публичный ключ новой строкой Далее даем права
|
## 2. Доставка приложения
|
||||||
sudo nano /home/deploy/.ssh/authorized_keys
|
|
||||||
sudo chmod 700 /home/deploy/.ssh
|
|
||||||
sudo chmod 600 /home/deploy/.ssh/authorized_keys
|
|
||||||
sudo chown -R deploy:deploy /home/deploy/.ssh
|
|
||||||
|
|
||||||
## Пароль деплою для использования sudo
|
Предпочтительно использовать Git либо архив без runtime-данных:
|
||||||
sudo passwd deploy
|
|
||||||
|
|
||||||
## После этого можно подключаться под деплой пользователем
|
```sh
|
||||||
ssh -i C:\Users\MI\.ssh\han_chat_deploy deploy@135.106.164.58
|
tar -C HAN_chat_specification/codebase/backend \
|
||||||
|
--exclude='.env' \
|
||||||
|
--exclude='secrets' \
|
||||||
|
--exclude='backups' \
|
||||||
|
-czf han-chat-backend.tar.gz .
|
||||||
|
scp -i <DEPLOY_SSH_KEY> han-chat-backend.tar.gz deploy@<VM_IP>:/tmp/
|
||||||
|
```
|
||||||
|
|
||||||
## GIT не работал. Проект пришлось копировать.
|
На ВМ:
|
||||||
tar -C HAN_chat_specification/codebase/backend -czf han-chat-backend.tar.gz .
|
|
||||||
scp -i C:\Users\MI\.ssh\han_chat_deploy C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz deploy@135.106.164.58:/tmp/
|
|
||||||
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel C:\Users\MI\Documents\Assistent\han-chat-backend.tar.gz root@135.106.164.58:/tmp/
|
```sh
|
||||||
|
sudo install -d -o deploy -g deploy -m 0755 /opt/han-chat/backend
|
||||||
|
sudo -u deploy tar -C /opt/han-chat/backend \
|
||||||
|
-xzf /tmp/han-chat-backend.tar.gz
|
||||||
|
|
||||||
## Дальше распаковка
|
|
||||||
cd /opt/han-chat/backend
|
cd /opt/han-chat/backend
|
||||||
tar -xzf /tmp/han-chat-backend.tar.gz
|
sudo find . -type f \( -name '*.sh' -o -name 'validate-env' \
|
||||||
|
-o -name 'han-secrets' -o -name 'han-compose' \) \
|
||||||
|
-exec dos2unix {} +
|
||||||
|
sudo chmod 0755 scripts/validate-env deployment/scripts/*.sh \
|
||||||
|
deployment/secrets/han-secrets deployment/secrets/han-compose \
|
||||||
|
redis/scripts/*.sh nginx/scripts/*.sh
|
||||||
|
sudo deployment/scripts/setup-vm.sh
|
||||||
|
```
|
||||||
|
|
||||||
## Правим переносы строк и даем права на исполнение
|
## 3. Несекретная конфигурация
|
||||||
find . -type f \( -name '*.sh' -o -name 'validate-env' \) -exec dos2unix {} +
|
|
||||||
chmod +x scripts/validate-env deployment/scripts/*.sh redis/scripts/*.sh nginx/scripts/*.sh
|
|
||||||
|
|
||||||
# Разворачиваем инфраструктуру в Селектел ч2
|
`.env` создаётся на самой ВМ из `.env.example` и содержит только URL, resource
|
||||||
|
names, feature flags и `SECRETS_SOURCE`. Его разрешено передавать как обычный
|
||||||
|
config, но запрещено добавлять credential-bearing DSN, password, token и key.
|
||||||
|
|
||||||
## Создаем БД
|
```sh
|
||||||
В базе подключаем pgcrypto
|
cp .env.example .env
|
||||||
Чтобы корректно работал PgBouncer выбираем session pooling
|
chmod 0600 .env
|
||||||
|
nano .env
|
||||||
### Копируем СА сертификат
|
|
||||||
|
|
||||||
Вариант Деплоя:
|
|
||||||
mkdir -p /opt/han-chat/backend/secrets/pg
|
|
||||||
scp -i C:\Users\MI\.ssh\han_chat_deploy -r C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\sertificates\CA.pem deploy@135.106.164.58:/opt/han-chat/backend/secrets/pg/ca.pem
|
|
||||||
chmod 644 /opt/han-chat/backend/secrets/pg/ca.pem
|
|
||||||
|
|
||||||
Вариант Селектела:
|
|
||||||
mkdir -p ~/.postgresql/
|
|
||||||
wget https://storage.dbaas.selcloud.ru/CA.pem -O ~/.postgresql/root.crt
|
|
||||||
chmod 0600 ~/.postgresql/root.crt
|
|
||||||
|
|
||||||
### Правки DNS-маршрутизации (в прошлый раз, какие-то трабблы были с базовой маршрутизацией) - скрипт ниже создаст
|
|
||||||
sudo tee /etc/netplan/99-han-dns.yaml <<'EOF'
|
|
||||||
network:
|
|
||||||
version: 2
|
|
||||||
ethernets:
|
|
||||||
eth0:
|
|
||||||
nameservers:
|
|
||||||
addresses:
|
|
||||||
- 8.8.8.8
|
|
||||||
- 1.1.1.1
|
|
||||||
search:
|
|
||||||
- selcloud.ru
|
|
||||||
eth1:
|
|
||||||
dhcp4-overrides:
|
|
||||||
use-dns: false
|
|
||||||
use-domains: false
|
|
||||||
EOF
|
|
||||||
|
|
||||||
sudo chmod 600 /etc/netplan/99-han-dns.yaml
|
|
||||||
sudo netplan apply
|
|
||||||
resolvectl flush-caches
|
|
||||||
|
|
||||||
### Проверка подключения к Postgers в ВМ
|
|
||||||
nc -zv 192.168.0.211 5433
|
|
||||||
|
|
||||||
sudo apt update
|
|
||||||
sudo apt install postgresql-client
|
|
||||||
|
|
||||||
psql "host=master.ef54e3e4-ad3d-4b80-a6af-d63269e0895a.c.dbaas.selcloud.ru \
|
|
||||||
port=5432 \
|
|
||||||
dbname=han_chat \
|
|
||||||
user=dbAdmin \
|
|
||||||
sslmode=verify-ca"
|
|
||||||
|
|
||||||
(psql "host=master.ef54e3e4-ad3d-4b80-a6af-d63269e0895a.c.dbaas.selcloud.ru port=5432 dbname=han_chat user=dbAdmin sslmode=verify-ca")
|
|
||||||
|
|
||||||
### Заводим пользователей и схемы
|
|
||||||
|
|
||||||
Пользователей создаем через интерфейс селектела.
|
|
||||||
Под админом даем права на создание в БД:
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO han_app;
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO bitrix_local_app;
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO bitrix_sync_user;
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO message_safety_app;
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO keycloak_user;
|
|
||||||
GRANT CREATE ON DATABASE han_chat TO sms_user
|
|
||||||
|
|
||||||
Если создаем пользователей после того как отозвали права from public, надо давать гранты на коннект:
|
|
||||||
GRANT CONNECT ON DATABASE han_chat TO sms_user
|
|
||||||
|
|
||||||
Схемы создаем от лица пользователей, заходя каждым из них в БД.
|
|
||||||
+ Запрещаем всем посторонним входить в схему han_app и др.
|
|
||||||
CREATE SCHEMA IF NOT EXISTS han_app;
|
|
||||||
REVOKE ALL ON SCHEMA han_app FROM PUBLIC;
|
|
||||||
ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO han_app;
|
|
||||||
SHOW search_path; --чтобы заработало надо переподключиться (должно быть han_app)
|
|
||||||
|
|
||||||
CREATE SCHEMA IF NOT EXISTS bitrix_local;
|
|
||||||
REVOKE ALL ON SCHEMA bitrix_local FROM PUBLIC;
|
|
||||||
ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO bitrix_local;
|
|
||||||
SHOW search_path; --чтобы заработало надо переподключиться
|
|
||||||
|
|
||||||
CREATE SCHEMA IF NOT EXISTS bitrix_sync;
|
|
||||||
REVOKE ALL ON SCHEMA bitrix_sync FROM PUBLIC;
|
|
||||||
ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO bitrix_sync;
|
|
||||||
SHOW search_path; --чтобы заработало надо переподключиться
|
|
||||||
|
|
||||||
CREATE SCHEMA IF NOT EXISTS message_safety;
|
|
||||||
REVOKE ALL ON SCHEMA message_safety FROM PUBLIC;
|
|
||||||
ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO message_safety;
|
|
||||||
SHOW search_path; --чтобы заработало надо переподключиться
|
|
||||||
|
|
||||||
CREATE SCHEMA IF NOT EXISTS keycloak;
|
|
||||||
REVOKE ALL ON SCHEMA keycloak FROM PUBLIC;
|
|
||||||
ALTER ROLE CURRENT_USER IN DATABASE han_chat SET search_path TO keycloak;
|
|
||||||
SHOW search_path; --чтобы заработало надо переподключиться
|
|
||||||
|
|
||||||
CREATE SCHEMA IF NOT EXISTS sms AUTHORIZATION sms_user;
|
|
||||||
REVOKE ALL ON SCHEMA sms FROM PUBLIC;
|
|
||||||
ALTER ROLE sms_user IN DATABASE han_chat SET search_path TO sms, public;
|
|
||||||
SHOW search_path;
|
|
||||||
|
|
||||||
|
|
||||||
Проверка search_path
|
|
||||||
SELECT r.rolname, d.datname, s.setconfig
|
|
||||||
FROM pg_db_role_setting s
|
|
||||||
JOIN pg_roles r ON r.oid = s.setrole
|
|
||||||
JOIN pg_database d ON d.oid = s.setdatabase
|
|
||||||
WHERE r.rolname in ('han_app',
|
|
||||||
'bitrix_local_app',
|
|
||||||
'bitrix_sync_user',
|
|
||||||
'message_safety_app',
|
|
||||||
'keycloak_user');
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
После реализации bitrix_sync (проверить, вероятно не на все таблицы права нужны):
|
|
||||||
-- Даем право на чтение (SELECT) всех СУЩЕСТВУЮЩИХ таблиц в схеме
|
|
||||||
GRANT SELECT, INSERT ON ALL TABLES IN SCHEMA han_app TO bitrix_sync_user;
|
|
||||||
-- Настраиваем права по умолчанию для новых таблиц
|
|
||||||
ALTER DEFAULT PRIVILEGES IN SCHEMA han_app
|
|
||||||
GRANT SELECT ON TABLES TO bitrix_sync_user;
|
|
||||||
-- Если нужно дать право на чтение и для новых последовательностей (sequences):
|
|
||||||
ALTER DEFAULT PRIVILEGES IN SCHEMA han_app
|
|
||||||
GRANT USAGE, SELECT ON SEQUENCES TO bitrix_sync_user;
|
|
||||||
|
|
||||||
## Заводим S3 хранилища:
|
|
||||||
|
|
||||||
Создаем 2 сервисных пользователя и заводим им ключи:
|
|
||||||
- API backend SELECTEL_S3_ACCESS_KEY / SELECTEL_S3_SECRET_KEY
|
|
||||||
- Message Safety SELECTEL_S3_QUARANTINE_READ_ACCESS_KEY / SELECTEL_S3_QUARANTINE_READ_SECRET_KEY
|
|
||||||
|
|
||||||
Создаем 3 приватных бакета
|
|
||||||
- han-chat-quarantine (политика доступа API backend RW, Message Safety R)
|
|
||||||
- han-chat-attachments (политика доступа API backend RW)
|
|
||||||
- han-chat-documents (политика доступа API backend RW)
|
|
||||||
|
|
||||||
RW:
|
|
||||||
ListBucket
|
|
||||||
GetBucketLocation
|
|
||||||
PutObject
|
|
||||||
GetObject
|
|
||||||
DeleteObject
|
|
||||||
ListBucketMultipartUploads
|
|
||||||
ListMultipartUploadParts
|
|
||||||
AbortMultipartUpload
|
|
||||||
|
|
||||||
R:
|
|
||||||
ListBucket
|
|
||||||
GetBucketLocation
|
|
||||||
GetObject
|
|
||||||
ListBucketMultipartUploads
|
|
||||||
ListMultipartUploadParts
|
|
||||||
|
|
||||||
Политика CORS (бакет han-chat-quarantine, vHosted обязателен):
|
|
||||||
Allowed origin: https://chat.han0107.ru
|
|
||||||
Methods: POST, PUT, GET, HEAD
|
|
||||||
Headers: * (или явно content-type; wildcard x-amz-* в Selectel не работает)
|
|
||||||
Expose headers: ETag
|
|
||||||
|
|
||||||
# Тест инфраструктуры
|
|
||||||
|
|
||||||
mkdir -p /opt/han-chat/infratest
|
|
||||||
cd /opt/han-chat/infratest
|
|
||||||
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\Assistent\HAN_chat_specification\infratest\*" root@135.106.164.58:/opt/han-chat/infratest
|
|
||||||
|
|
||||||
--настраиваю env на локальной машине и копирую на диск
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/infratest/
|
|
||||||
|
|
||||||
|
|
||||||
Создаем изолированное окружение:
|
|
||||||
python3 -m venv .venv
|
|
||||||
. .venv/bin/activate
|
|
||||||
python3 -m pip install --upgrade pip
|
|
||||||
python3 -m pip install -r requirements.txt
|
|
||||||
|
|
||||||
Если не работает, то надо DNS переправить на eth0
|
|
||||||
resolvectl dns eth0 1.1.1.1 8.8.8.8
|
|
||||||
resolvectl domain eth0 '~.'
|
|
||||||
resolvectl flush-caches
|
|
||||||
resolvectl query pypi.org
|
|
||||||
|
|
||||||
# Подготовка к запуску
|
|
||||||
|
|
||||||
Сертификат рекомендуется скопировать в папку с секретами
|
|
||||||
cp /root/.postgresql/root.crt /opt/han-chat/backend/secrets/pg/root.crt
|
|
||||||
chmod 644 /opt/han-chat/backend/secrets/pg/root.crt
|
|
||||||
|
|
||||||
В битрикс регистрируем локальное приложение:
|
|
||||||
Установка: https://chat.example.ru/bitrix/install
|
|
||||||
Обработчик: https://chat.example.ru/bitrix/handler
|
|
||||||
??? (не понадобилось) Placement: https://chat.example.ru/bitrix/placement
|
|
||||||
|
|
||||||
--настраиваю env на локальной машине и копирую на диск
|
|
||||||
scp -i C:\Users\MI\.ssh\hansel -r "C:\Users\MI\Documents\job\HAN_new_life\HANapp\Production\.env" root@135.106.164.58:/opt/han-chat/backend
|
|
||||||
|
|
||||||
--Проверка .env
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
chmod 600 .env
|
|
||||||
./scripts/validate-env .env
|
./scripts/validate-env .env
|
||||||
docker compose --env-file .env config --quiet
|
```
|
||||||
docker compose --env-file .env config --services
|
|
||||||
|
## 4. Selectel Secrets Manager
|
||||||
|
|
||||||
|
Выполните `deployment/secrets/SELECTEL_RUNBOOK.ru.md`:
|
||||||
|
|
||||||
|
1. отдельный проект и service user;
|
||||||
|
2. provider secrets и audit alerts;
|
||||||
|
3. root-only JSON-карта;
|
||||||
|
4. encrypted systemd credential;
|
||||||
|
5. `SECRETS_SOURCE=selectel`.
|
||||||
|
|
||||||
|
Не передавайте значение секрета аргументом команды, через `export`, тикет или
|
||||||
|
shell history. Старые значения из прежнего `.env` после cutover ротируются.
|
||||||
|
|
||||||
|
## 5. Проверка и запуск
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable han-secrets@production.service
|
||||||
|
sudo systemctl restart han-secrets@production.service
|
||||||
|
sudo ./scripts/validate-env .env \
|
||||||
|
--runtime-manifest /run/han-chat/secrets/manifest
|
||||||
|
sudo deployment/secrets/han-compose config --quiet
|
||||||
python3 -m unittest discover -s tests -v
|
python3 -m unittest discover -s tests -v
|
||||||
|
sudo deployment/secrets/han-compose build --pull
|
||||||
--Проверка портов (Публиковаться должны только 80 и 443 у nginx)
|
sudo deployment/scripts/migrate.sh
|
||||||
docker compose --env-file .env config | grep -n 'published:'
|
sudo deployment/secrets/han-compose up -d --wait
|
||||||
|
sudo deployment/scripts/smoke.sh
|
||||||
--Собираем локальные образы:
|
|
||||||
docker compose --env-file .env build --pull
|
|
||||||
|
|
||||||
docker compose --env-file .env images
|
|
||||||
|
|
||||||
## Миграции БД и начальные настройки
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
PITR_MARKER_CONFIRMED=true deployment/scripts/migrate.sh
|
|
||||||
deployment/scripts/seed.sh
|
|
||||||
|
|
||||||
## Запуск внутренних сервисов
|
|
||||||
|
|
||||||
Сначала запустите Redis:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env up -d redis
|
|
||||||
docker compose --env-file .env ps redis
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Затем Keycloak и OpenTelemetry:
|
Для диагностики используйте `han-compose ps` и ограниченные logs. Не выводите
|
||||||
|
resolved Compose config, `docker inspect` environment, полный `env` или secret
|
||||||
|
files.
|
||||||
|
|
||||||
|
## 6. Откат
|
||||||
|
|
||||||
|
Откат приложения использует immutable image/release ID и подтверждённую
|
||||||
|
совместимость схемы:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
docker compose --env-file .env up -d keycloak otel-collector
|
sudo SCHEMA_BACKWARD_COMPATIBLE_CONFIRMED=true \
|
||||||
docker compose --env-file .env ps keycloak otel-collector
|
deployment/scripts/rollback.sh <PREVIOUS_RELEASE>
|
||||||
```
|
```
|
||||||
|
|
||||||
Первый запуск Keycloak может занять несколько минут: он создаст свои таблицы и
|
Откат секрета выполняется активацией предыдущей версии в Selectel, повторным
|
||||||
импортирует realm `han-chat`.
|
sync и пересозданием только затронутых сервисов. Snapshot старого `.env` не
|
||||||
|
создаётся.
|
||||||
|
|
||||||
После готовности Keycloak:
|
## 7. Break-glass
|
||||||
|
|
||||||
```sh
|
Только при подтверждённом инциденте доставьте root-only recovery file из
|
||||||
docker compose --env-file .env up -d message-safety
|
защищённой офлайн-копии, установите `SECRETS_SOURCE=file`, выполните
|
||||||
docker compose --env-file .env up -d api-backend
|
sync/validate/recreate и зафиксируйте событие. Автоматический fallback запрещён.
|
||||||
docker compose --env-file .env up -d bitrix-local-app bitrix-sync
|
После восстановления Selectel верните штатный режим и удалите recovery file.
|
||||||
docker compose --env-file .env up -d \
|
|
||||||
delivery-worker safety-recovery-worker cleanup-worker
|
|
||||||
docker compose --env-file .env ps
|
|
||||||
```
|
|
||||||
|
|
||||||
Если сервис не становится healthy:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env logs --tail=200 <SERVICE_NAME>
|
|
||||||
docker inspect "$(docker compose --env-file .env ps -q <SERVICE_NAME>)"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Первоначальный выпуск TLS-сертификата
|
|
||||||
|
|
||||||
Для ACME требуется работающий nginx по HTTP. В `.env` оставьте
|
|
||||||
`NGINX_TLS_ENABLED=true`, но первый nginx запустите с временным переопределением:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
NGINX_TLS_ENABLED=false \
|
|
||||||
docker compose --env-file .env up -d frontend-static nginx
|
|
||||||
```
|
|
||||||
|
|
||||||
Проверьте HTTP:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
curl -I http://chat.han0107.ru/
|
|
||||||
```
|
|
||||||
|
|
||||||
Сначала рекомендуется проверить Certbot через staging:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env --profile certbot run --rm certbot certonly \
|
|
||||||
--staging \
|
|
||||||
--webroot -w /var/www/certbot \
|
|
||||||
-d chat.han0107.ru \
|
|
||||||
--cert-name chat.han0107.ru-staging \
|
|
||||||
--email ap@han.ru \
|
|
||||||
--agree-tos --no-eff-email --non-interactive
|
|
||||||
```
|
|
||||||
|
|
||||||
После успешного staging-теста выпустите рабочий сертификат с основным cert-name
|
|
||||||
без `--staging`:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env --profile certbot run --rm certbot certonly \
|
|
||||||
--webroot -w /var/www/certbot \
|
|
||||||
-d chat.han0107.ru \
|
|
||||||
--cert-name chat.han0107.ru \
|
|
||||||
--email ap@han.ru \
|
|
||||||
--agree-tos --no-eff-email --non-interactive
|
|
||||||
```
|
|
||||||
|
|
||||||
Пересоздайте nginx уже с TLS:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env up -d --force-recreate nginx
|
|
||||||
docker compose --env-file .env exec -T nginx nginx -t -c /tmp/nginx.conf
|
|
||||||
curl -I https://chat.han0107.ru/
|
|
||||||
```
|
|
||||||
|
|
||||||
Повторно запустите VM setup, чтобы он обнаружил проект и установил systemd-таймер
|
|
||||||
продления сертификата:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
sudo /opt/han-chat/backend/deployment/scripts/setup-vm.sh
|
|
||||||
systemctl status han-chat-ssl-renew.timer
|
|
||||||
|
|
||||||
## Запуск всего контура
|
|
||||||
|
|
||||||
Теперь можно привести весь проект к состоянию, описанному Compose:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
docker compose --env-file .env up -d
|
|
||||||
docker compose --env-file .env up -d --wait \
|
|
||||||
api-backend keycloak sms-service bitrix-local-app
|
|
||||||
|
|
||||||
# Nginx разрешает имена upstream при загрузке конфигурации. Если upstream
|
|
||||||
# был пересоздан и получил новый Docker IP, без reload edge продолжит ходить
|
|
||||||
# на старый адрес и вернёт 502.
|
|
||||||
docker compose --env-file .env exec -T nginx nginx -t -c /tmp/nginx.conf
|
|
||||||
docker compose --env-file .env kill -s HUP nginx
|
|
||||||
|
|
||||||
docker compose --env-file .env ps
|
|
||||||
```
|
|
||||||
|
|
||||||
Проверьте, что контейнеры не перезапускаются:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
docker compose --env-file .env ps
|
|
||||||
docker compose --env-file .env logs --since=10m
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Публичная проверка
|
|
||||||
|
|
||||||
Запустите smoke-тест:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cd /opt/han-chat/backend
|
|
||||||
deployment/scripts/smoke.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Также вручную проверьте:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
curl -fsS https://chat.han0107.ru/api/v1/public/app-config | jq
|
|
||||||
curl -fsS https://chat.han0107.ru/api/v1/public/content | jq
|
|
||||||
curl -fsS \
|
|
||||||
https://chat.han0107.ru/auth/realms/han-chat/.well-known/openid-configuration | jq
|
|
||||||
```
|
|
||||||
|
|
||||||
Если после `up`, `build`, `pull`, rollback или `--force-recreate` менялись
|
|
||||||
`api-backend`, `keycloak`, `sms-service` либо `bitrix-local-app`, всегда
|
|
||||||
повторяйте проверку `/tmp/nginx.conf` и `docker compose kill -s HUP nginx`.
|
|
||||||
Команды `nginx -t`/`nginx -s reload` без `-c /tmp/nginx.conf` здесь неверны:
|
|
||||||
контейнер read-only и рабочий PID расположен в `/tmp/nginx.pid`.
|
|
||||||
|
|
||||||
Внутренний API не должен быть опубликован:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
curl -i https://chat.han0107.ru/internal/safety/v1/messages/check
|
|
||||||
```
|
|
||||||
|
|
||||||
Ожидаемый статус — `404`.
|
|
||||||
|
|
||||||
Откройте в браузере:
|
|
||||||
|
|
||||||
```text
|
|
||||||
https://chat.example.ru/
|
|
||||||
```
|
|
||||||
|
|
||||||
Для тестовой авторизации используйте значение `KEYCLOAK_OTP_MOCK_CODE` из `.env`.
|
|
||||||
|
|
||||||
|
|
||||||
Настройки keykcloack:
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Для битрикса код установки приложения:
|
|
||||||
docker compose --env-file .env exec -T api-backend python - <<'PY'
|
|
||||||
import os
|
|
||||||
import urllib.request
|
|
||||||
|
|
||||||
base = os.environ["BITRIX_LOCAL_APP_BASE_URL"]
|
|
||||||
token = os.environ["BITRIX_LOCAL_APP_INTERNAL_TOKEN"]
|
|
||||||
|
|
||||||
request = urllib.request.Request(
|
|
||||||
base + "/internal/openlines/v1/setup/retry",
|
|
||||||
method="POST",
|
|
||||||
headers={"Authorization": "Bearer " + token},
|
|
||||||
)
|
|
||||||
print(urllib.request.urlopen(request).read().decode())
|
|
||||||
PY
|
|
||||||
|
|
||||||
## План включения реальной SMS-авторизации
|
|
||||||
|
|
||||||
Этот раздел — чек-лист будущего release из `modules/module-11-idgtl-sms.md`, а не подтверждение готовности текущего Compose. Пока отсутствуют реализованные `sms-service`/worker, migrations, callback route и env validation, оставлять `KEYCLOAK_OTP_MOCK_ENABLED=true`.
|
|
||||||
|
|
||||||
Prerequisites без placeholders:
|
|
||||||
|
|
||||||
- согласованные i-Digital sender и active approved template `auth_otp` с placeholders `code`, `ttl_min`;
|
|
||||||
- выданный Direct `TOKEN_1` (`IDGTL_SMS_API_KEY`, без повторного Base64);
|
|
||||||
- отдельные random callback username/password и публичный HTTPS URL;
|
|
||||||
- повторно подтверждённый source IP callback Direct;
|
|
||||||
- фактический статический egress IP, измеренный из `sms-worker`, записанный в inventory и переданный Direct для allowlist; при динамическом IP сначала настроить NAT/static IP.
|
|
||||||
|
|
||||||
Rollout:
|
|
||||||
|
|
||||||
1. Seed новых `otp.phone.*` в App DB.
|
|
||||||
2. Создать schema/role `sms`, применить versioned migrations и seed template/settings.
|
|
||||||
3. Проверить Keycloak→`sms-service`→локальный mock Direct в test environment.
|
|
||||||
4. Развернуть production `sms-service`/worker и nginx callback route, не выключая mock.
|
|
||||||
5. Применить Keycloak expand migration/SPI; прежние незавершённые challenges истечь по module-11.
|
|
||||||
6. Выполнить provider smoke на контролируемом номере; проверить `sms_message_id`, journal, callback и redaction.
|
|
||||||
7. Переключить `KEYCLOAK_OTP_MOCK_ENABLED=false`.
|
|
||||||
8. Проверить resend→`superseded`, expiry snapshot, limits и то, что Direct reject/timeout после durable order не меняет verify.
|
|
||||||
|
|
||||||
Rollback: вернуть Keycloak в mock mode; не удалять schema/journal. Остановить новые real orders, дать worker завершить либо зафиксировать in-flight/`uncertain`. Schema downgrade только при доказанной backward compatibility, иначе forward-fix.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user