whatwedo iac-shell

shell in a container with tools for managing our infrastructure:

• ansible
• molecule (ansible testing)
• borg (inspect and restore from our remote backups)
• prettier, yamllint, ...

Requirements

• podman
• the 1Password desktop app and the 1Password CLI on the host

Setup

Clone the repository, then source source.sh from your local clone:

git clone https://github.com/whatwedo/iac-shell.git ~/git/whatwedo/iac-shell.git
source ~/git/whatwedo/iac-shell.git/source.sh

Any path works — ~/git/whatwedo/iac-shell.git as an example

To get the iac command in every shell, add the source line to your ~/.bashrc:

echo 'source ~/git/whatwedo/iac-shell.git/source.sh' >> ~/.bashrc

Keep it up to date with:

git -C ~/git/whatwedo/iac-shell.git pull

1Password

The shell gets both its SSH keys and its secrets from 1Password, so iac refuses to start until it is set up.

In the 1Password app, enable Settings > Developer > Use the SSH agent. Set Ask approval for each new to application, so a playbook across many hosts asks once rather than per host.

Secrets come from a service account:

1. Create a vault (Infra in the examples below) for the shared secrets. Service accounts cannot read Private, Personal, Employee or the default Shared vault.
2. Under Developer > Service Accounts, create one named iac-shell-sa and grant it read_items on that vault only. Vault permissions are fixed at creation and cannot be changed afterwards.
3. Save the generated token straight into your own Private vault from the same dialog — the item is named after the service account, giving op://Private/iac-shell-sa/credential, which is the reference iac expects.

iac reads that token at launch and passes it in, so nothing is stored on disk.

Usage

iac opens the shell. Your current directory is mounted at /workspace inside it, so run it from the repository you want to work on:

cd ~/git/whatwedo/some-infra-repo
iac

Pass --pull to fetch the latest image before starting:

iac --pull

Arguments after -- are passed straight through to podman run. They land just before the image name, so they can also override the defaults iac sets:

iac -- -v /srv/backups:/srv/backups:ro   # mount something extra
iac -- -w /workspace/roles               # start somewhere other than /workspace

Tools

Containers (podman / docker)

The host's rootless podman socket is forwarded into the container, so both the podman and docker CLIs work inside the shell — they are thin clients driving the host's podman (which serves a Docker-API-compatible endpoint on the same socket). Nothing runs a container engine daemon inside the shell.

podman ps          # host podman
docker ps          # same host podman, via the Docker API
docker compose up  # docker-compose-plugin is installed

DOCKER_HOST / CONTAINER_HOST both point at the forwarded socket, and it is also symlinked to /var/run/docker.sock for tools that hardcode that path.

Secrets (1Password)

op is authenticated inside the shell:

op read op://Infra/some-item/password
op run --env-file=.env.tpl -- ./some-script

In playbooks, look secrets up from 1Password:

- name: Fetch the secrets this play needs, once
  ansible.builtin.set_fact:
    pihole_password: "{{ lookup('community.general.onepassword', 'pihole', field='password', vault='Infra') }}"
  run_once: true
  no_log: true

Read values once into facts, as above. Every lookup shells out to op and counts against the service account's rate limit.

SSH keys

Keys stored in 1Password are used by default; the host's agent socket is mounted in and applied to every host via /etc/ssh/ssh_config.d/10-1password.conf. Approval prompts appear on the host.

A key added by hand lands in the shell's own agent, which that IdentityAgent shadows. Point a host back at it explicitly:

ssh-add ~/.ssh/some-key
ssh -o IdentityAgent=SSH_AUTH_SOCK my-server

To make that the default for a host, add to ~/.ssh/config:

Host legacy-box
  IdentityAgent SSH_AUTH_SOCK

On Too many authentication failures, the agent is offering more keys than the server's MaxAuthTries allows. Pin one:

Host *
  IdentityFile ~/.ssh/iac.pub    # the PUBLIC key
  IdentitiesOnly yes

Open Source ❤️

We love whatwedo — a software studio in Bern, Switzerland, fighting the good fight against bad software. But here's the thing: we also love open source.

Curious what else we've been building in the open? → github.com/whatwedo