Add plugin config to Docker image

Goobi workflow
#1

The Docker image intranda/goobi-workflow:default-plugins contains a set of default plugins.

However, it does not contain the configuration files required by the plugins. (Correct me, if I’m wrong.)

The configuration files must be downloaded manually from Github and bind mounted into the Docker container running the image. This step must be repeated every time the plugins and/or plugin configuration files change. Which entails that one must continously observe all plugins for changes.

Suggestion: add default configuration files for plugins into intranda/goobi-worflow:default-plugins, to make the image self-contained. Overwritting the default plugin configuration will still be possible by mounting configuration files during startup.

#2

Hey,

this is a good suggestion. However I am not that involved into the docker infrastructure development, so that others might jump in here. Just one quick comment from my side: I would not really say that there are that many default plugins. Lots of users use the fileUpload and the imageQA plugins. But a lot of the other 210 plugins are quite individual and not helpful for everybody (e.g. Barcode Generator plugin). But regarding at lease these two most commonly used ones (fileUpload and imageQA) this might be helpful indeed.

@oliver.paetzel and @matthias Can you maybe help me out here with some more details?

All the best,

Steffen

#3

So I just checked and the image contains the configuration files for the plugins. Here is what I did to verify this:

(base) oliver@oliver-desktop:~$ docker pull intranda/goobi-workflow:default-plugins
default-plugins: Pulling from intranda/goobi-workflow
# left out some docker pull output
Digest: sha256:c4c2c92995b8d5e1eaa858e12b3e9d33d37ac02d0a5d171f85de334be65fee2e
Status: Downloaded newer image for intranda/goobi-workflow:default-plugins
docker.io/intranda/goobi-workflow:default-plugins
(base) oliver@oliver-desktop:~$ docker run -it intranda/goobi-workflow:default-plugins /bin/bash
root@95dad2016f27:/usr/local/tomcat# ls /opt/digiverso/goobi/config/ | grep plugin_
plugin_intranda_dashboard_extended.xml
plugin_intranda_dashboard_simple.xml
plugin_intranda_step_fileUpload.xml
plugin_intranda_step_imageQA.xml
plugin_intranda_validation_filename.xml
plugin_metadata-dante.xml
#4

Thanks for the replies.

You are right @oliver.paetzel, the config files are included already!

I found out what the problem on my end is: obviously, we need to mount a local directory containing configuration files into /opt/digiverso/goobi/config to be able to configure goobi workflow.

The problem is: the files in /opt/digiverso/goobi/config coming with the container, such as the plugin configuration files, are lost. Only the files from the mounted (host) directory will remain.

Is it possible to use multiple directories containing configuration files? Or a place configuration in a subfolder of /opt/digiverso/goobi/config? This way, one could mount a directory containing custom configuration files, while leaving goobi’s plugin configuration untouched.

#5

What we do for other production docker installations is to copy the configuration from another configuration source to the Goobi configuration folders. Currently, we only do this with S3, but a mounted folder could be used in the same way.
I think this would help you. You could just add your configuration files to the said folder and these would overwrite the default ones. What do you think?

#6

That doesn’t solve the problem, see original post.

If the configuration files for plug-ins shipping with the container change, we will have to manually update our copies of these files. This requires to constantly monitor all plug-ins used for changes.

If an image ships with a set of default plug-ins, the required configuration files should follow along. Otherwise, the purpose and usefulness of such a container are getting undermined.

#7

The image will still ship with all configuration files. What would change is the following:

Before starting Goobi, the run.sh script would copy configuration files from a mounted folder into the regular Goobi configuration folder. If the mounted folder only contains general Goobi configuration files, the plugin configuration in the image would still be there, unaltered.

#8

Thanks @oliver.paetzel, I see where run.sh fetches configuration from s3, if configured.

If not using s3, though, no configuration will be fetched. So were you saying that you were going to adapt run.sh to fetch configuration from a mounted folder during startup? Or did you only suggest that this would be one possibility of solving this problem?

I could implement this and send a PR. I just would like to use an official Docker image (from intranda), instead of creating my own.

#9

Hi,

sorry… chiming in late.
Actually I had already implemented that, but obviously did not put in it into the goobi workflow master. That way custom configs could be used to overwrite the ones coming within the image. For the locally modified ones, one will however have to keep track of configuration changes that might be required for the update. But for this more limited number of files this would be less involved of course.
If this would help, we could definitely put it into the images that are available on docker hub.

#10

Perfect @matthias, exactly what I was looking for.

I’m aware that I still have to keep track of modifications made to default config files.

Much appreciated, if you could put that into the images on Dockerhub! Could you let me know when the changes have been published?

#11

Just a heads-up @sepastian that I finally got around to put this into the develop branch of goobi workflow (link). It will be available with the next release. @oliver.paetzel might know more about any timelines there.

1 Like