Feature/1175 add minos support

README.md

@@ -2,7 +2,14 @@
 
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)
 
-> A [Django](https://docs.djangoproject.com) project template ready for continuous delivery.
+> A [Django](https://docs.djangoproject.com) service template aligned to the 20tab **Minos** platform model: per-env Vault-driven secrets, GitLab Components OpenTofu deploys, Terraform Cloud workspaces managed by the parent platform.
+
+The generated service is meant to live as a sibling sub-repo of a platform produced by [talos](https://github.com/20tab/talos), and ships with:
+
+-   `Dockerfile` multi-stage on `uv` + Python 3.14
+-   `.gitlab-ci.yml` using `${CI_SERVER_FQDN}/components/opentofu/apply` and `registry.gitlab.com/20tab-open/minos/service:latest`
+-   `minos/{development,staging,production}/this.tfvars` + `common.tfvars` per-env configs
+-   Vault secret consumption at `{project}/envs/${CI_ENVIRONMENT_SLUG}/{service}/...`
 
 ## 🧩 Requirements
 
@@ -33,15 +40,20 @@ python3 -m pip install -r requirements/common.txt
 
 The `terraform` cli package is required, unless you want to generate a project only locally. To install it we suggest to use the official [install guide](https://learn.hashicorp.com/tutorials/terraform/install-cli).
 
-## 🔑 Credentials (optional)
+## 🔑 Prerequisites
 
-### 🦊 GitLab
+### 🗝️ Vault project (one-time, admin)
+
+The Minos pipeline assumes a shared Vault auth backbone is already provisioned by the [vault-project](https://github.com/20tab/vault-project) admin repo: KV mount, GitLab JWT auth backend, JWT roles `service-gitlab-job` and `platform-gitlab-job`, identity entity, admin policy. Run that **once per Vault cluster, before** bootstrapping any platform/service.
+
+This sub-bootstrapper only seeds **service-scoped** secrets at `{project_slug}/envs/{env}/{service_slug}/...`. Vault prompts are optional: skip them if Vault is not used for this project.
 
-If the GitLab integration is enabled, a Personal Access Token with _api_ permission is required.<br/>
-It can be generated in the GitLab User Settings panel.
+### 🦊 GitLab (optional)
 
-**Note:** the token can be generated in the Access Tokens section of the GitLab User Settings panel.<br/>
-⚠️ Beware that the token is shown only once after creation.
+If the GitLab integration is enabled, a Personal Access Token with _api_ scope is required.<br/>
+It can be generated in the GitLab User Settings → Access Tokens panel.
+
+⚠️ The token is shown only once after creation.
 
 ## 🚀️ Quickstart
 
@@ -70,41 +82,34 @@ source talos-django/.venv/bin/activate
 Project name: My Project Name
 Project slug [my-project-name]:
 Service slug [backend]:
-Project dirname (backend, myprojectname) [backend]: myprojectname
-Deploy type (digitalocean-k8s, other-k8s) [digitalocean-k8s]:
-Terraform backend (gitlab, terraform-cloud) [terraform-cloud]:
-Terraform host name [app.terraform.io]:
-Terraform Cloud User token:
-Terraform Organization: my-organization-name
-Do you want to create Terraform Cloud Organization 'my-organization-name'? [y/N]:
-Choose the environments distribution:
-  1 - All environments share the same stack (Default)
-  2 - Dev and Stage environments share the same stack, Prod has its own
-  3 - Each environment has its own stack
- (1, 2, 3) [1]:
+Project dirname (backend, myprojectname) [backend]:
+Do you want to use Redis? [y/N]:
+Do you want to use Postgres? [Y/n]:
+Create a database inside the Postgres cluster? [Y/n]:
+Terraform Cloud organization: my-tfc-org
+Do you want to use Vault for secrets management? [y/N]: y
+Vault token (leave blank to perform a browser-based OIDC authentication):
+Make sure your Vault permissions allow to enable the project secrets backends and manage the project secrets. Continue? [y/N]: y
+Vault address: https://vault.example.com
+Cluster slug hosting the 'development' environment [dev]:
+Cluster slug hosting the 'staging' environment [dev]:
+Cluster slug hosting the 'production' environment [main]:
 Development environment complete URL [https://dev.my-project-name.com]:
 Staging environment complete URL [https://stage.my-project-name.com]:
 Production environment complete URL [https://www.my-project-name.com]:
-Media storage (digitalocean-s3, aws-s3, local, none) [digitalocean-s3]:
-Do you want to configure Redis? [y/N]:
+Do you want to use Sentry? [y/N]:
 Do you want to use GitLab? [Y/n]:
-GitLab group slug [my-project-name]:
-Make sure the GitLab "my-project-name" group exists before proceeding. Continue? [y/N]: y
-GitLab private token (with API scope enabled):
-Sentry DSN (leave blank if unused) []:
+GitLab URL [https://gitlab.com]:
+GitLab access token (with API scope enabled):
+GitLab parent group path: 20tab/my-project-name
+Media storage (digitalocean-s3, aws-s3, local, none) [digitalocean-s3]:
 Initializing the backend service:
 ...cookiecutting the service
 ...generating the .env file
 ...formatting the cookiecut python code
-...compiling the requirements files
-	- common.txt
-	- test.txt
-	- local.txt
-	- remote.txt
-	- base.txt
 ...creating the '/static' directory
 ...creating the GitLab repository and associated resources
-...creating the Terraform Cloud resources
+...creating the Vault resources with Terraform
 ```
 
 ## 🗒️ Arguments
@@ -147,43 +152,22 @@ The following arguments can be appended to the Docker and shell commands
 
 ### 📐 Architecture
 
-#### Deploy type
+#### Terraform Cloud organization
 
-| Description             | Argument                             |
-| ----------------------- | ------------------------------------ |
-| DigitalOcean Kubernetes | `--deployment-type=digitalocean-k8s` |
-| Other Kubernetes        | `--deployment-type=other-k8s`        |
+The TFC organization that owns the service workspaces. The workspaces themselves (`{project}_{service}_{env}`) are created by the parent platform via [talos](https://github.com/20tab/talos), not here.
 
-#### Terraform backend
+`--terraform-cloud-organization=my-tfc-org`
 
-| Name            | Argument                              |
-| --------------- | ------------------------------------- |
-| Terraform Cloud | `--terraform-backend=terraform-cloud` |
-| GitLab          | `--terraform-backend=gitlab`          |
+#### Cluster mapping per environment
 
-##### Terraform Cloud required argument
+Each environment is deployed to one cluster. Cluster slugs are prompted interactively per env (defaults: `development → dev`, `staging → dev`, `production → main`). There is no CLI flag for this mapping; pass them via prompt or `--quiet` with the defaults.
 
-`--terraform-cloud-hostname=app.terraform.io`<br/>
-`--terraform-cloud-token={{terraform-cloud-token}}`<br/>
-`--terraform-cloud-organization`
+#### 🗝️ Vault
 
-##### Terraform Cloud create organization
+`--vault-url=https://vault.example.com`<br/>
+`--vault-token={{vault-token}}` (env var: `VAULT_TOKEN`; leave blank for browser-based OIDC)
 
-`--terraform-cloud-organization-create`<br/>
-`--terraform-cloud-admin-email={{terraform-cloud-admin-email}}`
-
-Disabled args
-`--terraform-cloud-organization-create-skip`
-
-#### Environment distribution
-
-Choose the environments distribution:
-
-| Value | Description                                                       | Argument                       |
-| ----- | ----------------------------------------------------------------- | ------------------------------ |
-| 1     | All environments share the same stack (Default)                   | `--environment-distribution=1` |
-| 2     | Dev and Stage environments share the same stack, Prod has its own | `--environment-distribution=2` |
-| 3     | Each environment has its own stack                                | `--environment-distribution=3` |
+Omit `--vault-url` to disable Vault integration (in that case GitLab CI vars are used as a fallback for sensitive values).
 
 #### Project Domain
 
@@ -213,21 +197,37 @@ Disabled args
 
 ### 🦊 GitLab
 
-> **⚠️ Important: Make sure the GitLab group exists before creating.** > https://gitlab.com/gitlab-org/gitlab/-/issues/244345
-
 For enabling gitlab integration the following arguments are needed:
 
-`--gitlab-private-token={{gitlab-private-token}}`<br/>
-`--gitlab-group-path={{gitlab-group-path}}`
+`--gitlab-url=https://gitlab.com`<br/>
+`--gitlab-token={{gitlab-token}}` (env var: `GITLAB_PRIVATE_TOKEN`)<br/>
+`--gitlab-namespace-path=20tab/my-project-name`
+
+The namespace path can be nested (e.g. `20tab/my-project-name`). When invoked from talos, this is set automatically to `{parent-group}/{project-slug}`.
 
 #### 🪖 Sentry
 
 For enabling sentry integration the following arguments are needed:
 
-`--sentry-dsn={{frontend-sentry-dsn}}`
+`--sentry-org={{sentry-org}}`<br/>
+`--sentry-url=https://sentry.io/`<br/>
+`--sentry-dsn={{sentry-dsn}}`
 
 #### 🔇 Quiet
 
 No confirmations shown.
 
 `--quiet`
+
+### 🧰 Toolchain version overrides
+
+The generated service pins specific versions of Python, OpenTofu and the Minos image. Defaults match the current 20tab platform; override only if needed.
+
+| Field                        | Default                                                  | Where it lands                              |
+| ---------------------------- | -------------------------------------------------------- | ------------------------------------------- |
+| `python_version`             | `3.14`                                                   | `Dockerfile`, `pyproject.toml` (ruff/mypy)  |
+| `minos_service_image`        | `registry.gitlab.com/20tab-open/minos/service:latest`    | `.gitlab-ci.yml` deploy image               |
+| `opentofu_component_version` | `3.11.0`                                                 | GitLab Component pin in `.gitlab-ci.yml`    |
+| `opentofu_version`           | `1.10.6`                                                 | OpenTofu binary version in `.gitlab-ci.yml` |
+
+These are not exposed as CLI flags; pass them as kwargs when invoking the `Runner` directly (e.g. from talos).

bootstrap/collector.py

@@ -9,15 +9,15 @@
 from slugify import slugify
 
 from bootstrap.constants import (
-    DEPLOYMENT_TYPE_CHOICES,
-    DEPLOYMENT_TYPE_DIGITALOCEAN,
-    DEPLOYMENT_TYPE_OTHER,
-    ENVIRONMENTS_DISTRIBUTION_CHOICES,
-    ENVIRONMENTS_DISTRIBUTION_DEFAULT,
-    ENVIRONMENTS_DISTRIBUTION_PROMPT,
+    ENV_NAMES,
+    ENV_TO_CLUSTER_DEFAULT,
     GITLAB_URL_DEFAULT,
     MEDIA_STORAGE_CHOICES,
     MEDIA_STORAGE_DIGITALOCEAN_S3,
+    MINOS_SERVICE_IMAGE,
+    OPENTOFU_COMPONENT_VERSION,
+    OPENTOFU_VERSION,
+    PYTHON_VERSION_DEFAULT,
     TERRAFORM_BACKEND_CHOICES,
     TERRAFORM_BACKEND_TFC,
 )
@@ -43,7 +43,6 @@ class Collector:
     project_dirname: str | None = None
     service_slug: str | None = None
     internal_service_port: int | None = None
-    deployment_type: str | None = None
     terraform_backend: str | None = None
     terraform_cloud_hostname: str | None = None
     terraform_cloud_token: str | None = None
@@ -52,7 +51,9 @@ class Collector:
     terraform_cloud_admin_email: str | None = None
     vault_token: str | None = None
     vault_url: str | None = None
-    environments_distribution: str | None = None
+    use_postgres: bool | None = None
+    postgres_create_database: bool | None = None
+    env_to_cluster: dict[str, str] | None = None
     project_url_dev: str | None = None
     project_url_stage: str | None = None
     project_url_prod: str | None = None
@@ -64,6 +65,10 @@ class Collector:
     gitlab_url: str | None = None
     gitlab_token: str | None = None
     gitlab_namespace_path: str | None = None
+    python_version: str | None = None
+    minos_service_image: str | None = None
+    opentofu_component_version: str | None = None
+    opentofu_version: str | None = None
     uid: int | None = None
     gid: int | None = None
     terraform_dir: Path | None = None
@@ -81,14 +86,15 @@ def collect(self):
         self.set_project_dirname()
         self.set_service_dir()
         self.set_use_redis()
+        self.set_postgres()
         self.set_terraform()
         self.set_vault()
-        self.set_deployment_type()
-        self.set_environments_distribution()
+        self.set_env_to_cluster()
         self.set_project_urls()
         self.set_sentry()
         self.set_gitlab()
         self.set_media_storage()
+        self.set_versions()
 
     def set_project_slug(self):
         """Set the project slug option."""
@@ -133,6 +139,18 @@ def set_use_redis(self):
                 warning("Do you want to use Redis?"), default=False
             )
 
+    def set_postgres(self):
+        """Set the Postgres options."""
+        if self.use_postgres is None:
+            self.use_postgres = click.confirm(
+                warning("Do you want to use Postgres?"), default=True
+            )
+        if self.use_postgres and self.postgres_create_database is None:
+            self.postgres_create_database = click.confirm(
+                warning("Create a database inside the Postgres cluster?"),
+                default=True,
+            )
+
     def set_terraform(self):
         """Set the Terraform options."""
         if self.terraform_backend not in TERRAFORM_BACKEND_CHOICES:
@@ -192,27 +210,15 @@ def set_vault(self):
             )
             self.vault_url = validate_or_prompt_url("Vault address", self.vault_url)
 
-    def set_deployment_type(self):
-        """Set the deployment type option."""
-        if self.deployment_type not in DEPLOYMENT_TYPE_CHOICES:
-            self.deployment_type = click.prompt(
-                "Deploy type",
-                default=DEPLOYMENT_TYPE_DIGITALOCEAN,
-                type=click.Choice(DEPLOYMENT_TYPE_CHOICES, case_sensitive=False),
-            ).lower()
-
-    def set_environments_distribution(self):
-        """Set the environments distribution option."""
-        # TODO: forcing a single stack when deployment is `k8s-other` should be removed,
-        #       and `set_deployment_type` merged with `set_deployment`
-        if self.deployment_type == DEPLOYMENT_TYPE_OTHER:
-            self.environments_distribution = "1"
-        elif self.environments_distribution not in ENVIRONMENTS_DISTRIBUTION_CHOICES:
-            self.environments_distribution = click.prompt(
-                ENVIRONMENTS_DISTRIBUTION_PROMPT,
-                default=ENVIRONMENTS_DISTRIBUTION_DEFAULT,
-                type=click.Choice(ENVIRONMENTS_DISTRIBUTION_CHOICES),
-            )
+    def set_env_to_cluster(self):
+        """Set the environment-to-cluster mapping (one cluster slug per environment)."""
+        self.env_to_cluster = self.env_to_cluster or {}
+        for env_name in ENV_NAMES:
+            if env_name not in self.env_to_cluster:
+                self.env_to_cluster[env_name] = click.prompt(
+                    f"Cluster slug hosting the '{env_name}' environment",
+                    default=ENV_TO_CLUSTER_DEFAULT[env_name],
+                )
 
     def set_project_urls(self):
         """Set the project urls options."""
@@ -286,6 +292,21 @@ def set_media_storage(self):
                 type=click.Choice(MEDIA_STORAGE_CHOICES, case_sensitive=False),
             ).lower()
 
+    def set_versions(self):
+        """Set the toolchain versions."""
+        self.python_version = self.python_version or click.prompt(
+            "Python version", default=PYTHON_VERSION_DEFAULT
+        )
+        self.minos_service_image = self.minos_service_image or click.prompt(
+            "Minos service image", default=MINOS_SERVICE_IMAGE
+        )
+        self.opentofu_component_version = self.opentofu_component_version or click.prompt(
+            "OpenTofu CI component version", default=OPENTOFU_COMPONENT_VERSION
+        )
+        self.opentofu_version = self.opentofu_version or click.prompt(
+            "OpenTofu version", default=OPENTOFU_VERSION
+        )
+
     def get_runner(self):
         """Get the bootstrap runner instance."""
         return Runner(
@@ -298,7 +319,6 @@ def get_runner(self):
             service_dir=self._service_dir,
             service_slug=self.service_slug,
             internal_service_port=self.internal_service_port,
-            deployment_type=self.deployment_type,
             terraform_backend=self.terraform_backend,
             terraform_cloud_hostname=self.terraform_cloud_hostname,
             terraform_cloud_token=self.terraform_cloud_token,
@@ -307,7 +327,9 @@ def get_runner(self):
             terraform_cloud_admin_email=self.terraform_cloud_admin_email,
             vault_token=self.vault_token,
             vault_url=self.vault_url,
-            environments_distribution=self.environments_distribution,
+            use_postgres=self.use_postgres,
+            postgres_create_database=self.postgres_create_database,
+            env_to_cluster=self.env_to_cluster,
             project_url_dev=self.project_url_dev,
             project_url_stage=self.project_url_stage,
             project_url_prod=self.project_url_prod,
@@ -319,6 +341,10 @@ def get_runner(self):
             gitlab_url=self.gitlab_url,
             gitlab_token=self.gitlab_token,
             gitlab_namespace_path=self.gitlab_namespace_path,
+            python_version=self.python_version,
+            minos_service_image=self.minos_service_image,
+            opentofu_component_version=self.opentofu_component_version,
+            opentofu_version=self.opentofu_version,
             terraform_dir=self.terraform_dir,
             logs_dir=self.logs_dir,
         )