Add support for `COLLECTION_DIRS` in Docker

[grantfitzsimmons]

Source: Speciforum

I am not a developer, but from what I gather in our Dockerfile on GitHub, it does not appear that the docker-compose.yml file supports setting up COLLECTION_DIRS at this time. This may be possible with some modification on our end.

See these lines in the Dockerfile:

web-asset-server/Dockerfile

Lines 28 to 35 in ae6c660

	RUN echo \
	"import os" \
	"\nSERVER = 'paste'" \
	"\nSERVER_NAME = os.environ['SERVER_NAME']" \
	"\nSERVER_PORT = int(os.getenv('SERVER_PORT', 8080))" \
	"\nKEY = os.environ['ATTACHMENT_KEY']" \
	"\nDEBUG = os.getenv('DEBUG_MODE', 'false').lower() == 'true'" \
	>> settings.py

While the SERVER_NAME, SERVER_PORT, ATTACHMENT_KEY, and DEBUG_MODE environmental variables are entered, there is no support in our current docker deployment method for the user to submit the collection directories through the compose file.

The RUN command could likely be modified to passthrough the you are setting in the compose file, but it is currently structured as it needs to be stored in the settings.py file rather than the environmental variable format the docker-compose.yml and Dockerfile are set up to expect.

Requested By: Fedor at NHMD

[grantfitzsimmons]

Tried to start here:
https://github.com/specify/web-asset-server/tree/collection_dirs

[specifysoftware]

This issue has been mentioned on Specify Community Forum. There might be relevant details there:

https://discourse.specifysoftware.org/t/specify7-in-docker-keeps-on-failing-on-port-8080/1361/12

[maxpatiiuk]

I remember that Ben was hesitant to add complex props (like arrays and dictionaries) as env variables.
Instead, it might be better to provide those as a mounted python config file

[realVinayak]

I don't know if we should support very complicated props like that even in the first place, given that docker allows mounting files. So, if someone wants, they can just mount an entire settings.py file. Given that the workaround is pretty simple, and doesn't even require a separate docker image, I don't think it is worth the dev time.

[melton-jason]

The most complicated structure I think we should reasonably support through environment variables are single-delimiter (e.g., comma only) separated arrays coupled with strict precedence regarding resolution of defaults.

There is already a precedent for this within the Specify 7 repository for the ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS variables:

https://github.com/specify/specify7/blob/a69f68cecb895028f32a9709b44abb771dfa4515/Dockerfile#L176-L183

        # Resolve ALLOWED_HOSTS in the following precedence:
        # - Use the ALLOWED_HOSTS environment variable (if present)
        # - Otherwise, fallback to the default specified in settings/specify_settings.py
        # - If still not defined, use the hard-coded default ['*']
        # See https://github.com/specify/specify7/pull/6831
        "\n_env_allowed_hosts = os.getenv('ALLOWED_HOSTS', None)" \
        "\n_default_allowed_hosts = getattr(specify_defaults,'ALLOWED_HOSTS', ['*'])" \
        "\nALLOWED_HOSTS = _default_allowed_hosts if _env_allowed_hosts is None else [host.strip() for host in _env_allowed_hosts.split(',')]" \

Anything more complicated will likely require a custom structure and associated parser: which will be significantly more complex to implement and definitely require an additional burden on the user to learn and match the corresponding spec--- overall harder to implement/maintain and more difficult to learn/work-with as a user compared with simpler environment variables.

If anything, can we maybe modify our docker-compose.yml templates to include a bind mount to a settings file?

  asset-server:
    restart: unless-stopped
    image: specifyconsortium/specify-asset-service
    init: true
    volumes:
      # Store all attachments outside the container, in a separate
      # volume
      - "attachments:/home/specify/attachments"
+      # Mount an asset_server configuration file to the asset server
+      # - ./asset_server_settings.py:/home/specify/settings.py:ro