Merge pull request #10 from gammasim/gammasim-tools-production

Production container

README.md

@@ -1,6 +1,6 @@
 # Introduction
 
-Experimental Docker files (and later singularity configurations) are provided for gammasim applications.
+Docker files for [gammasim-tools](https://github.com/gammasim/gammasim-tools) applications.
 
 # Docker Containers
 
@@ -12,20 +12,8 @@ The following Docker files are available:
 
 ## Installation
 
-## Running a gammasim-tools Container
+see [./dev](./dev) or [./prod](./prod) directories for docker images used for 
+gammasim-tools development or production. Main difference is:
+- [prod](prod/README.md): includes gammasim-tools and corsika/sim_telarray installation (**expermental - not working yet**)
+- [dev](dev/README.md): includes corsika/sim_telarray, a full latex installation and expects the gammasim-tools installation in the `./external` directory
 
-```
-docker run --rm -it -v "$(pwd)/external:/workdir/external" gammasim-tools-dev bash
-```
-
-# Building corsika/sim_telarray
-
-```
-build_all prod5 qgs2 gsl
-```
-
-or use the following Makefile options:
-
-```
-EXTRA_DEFINES="-DCTA_PROD4 -DMAXIMUM_TELESCOPES=180 -DWITH_GSL_RNG"
-```

dev/Dockerfile

@@ -41,12 +41,6 @@ RUN wget https://raw.githubusercontent.com/gammasim/gammasim-tools/master/enviro
     mamba env update --file  environment.yml && \
     conda clean -tipsy
 
-# Make sure krb token is delegated
-RUN mkdir ~/.ssh && \
-    echo "GSSAPIDelegateCredentials yes" > ~/.ssh/config && \
-    ssh-keyscan -H warp1.zeuthen.desy.de >> ~/.ssh/known_hosts
-
-ENV KRB5CCNAME="FILE:/tmp/token.krb"
 SHELL ["/bin/bash", "-c"]
 
 RUN source /root/.bashrc && \

dev/README.md

@@ -44,7 +44,7 @@ Building expects that a tar ball of corsika/sim\_telarray (named corsika7.7\_sim
 Download the tar package from the MPIK website (password applies) with
 
 ```
-$ ./download_simulationsoftware.sh
+$ ../tools/download_simulationsoftware.sh
 ```
 
 Build the container with:
@@ -60,11 +60,9 @@ Building will take a while and the image is large (3.9 GB; mostly due to the >2
 To run the container in bash 
 
 ```
-$ docker run --rm -it -v "$(pwd)/external:/workdir/external" -v "/tmp/token.krb:/tmp/token.krb" gammasim-tools-dev bash
+$ docker run --rm -it -v "$(pwd)/external:/workdir/external" gammasim-tools-dev bash
 ```
 
-Notice that we assume a kerberos token file "token.krb" is present in /tmp. This allows smooth connection to the DB. The docker does not fail if the token does not exist (this is a temporary solution.)
-
 Any output will be written to the `$(pwd)/external` directory.
 
 ## Docker Hub

dev/external/.gitignore

@@ -3,3 +3,4 @@ verification-process
 output
 simulation-model-description
 data-from-instrument-teams
+*.yml

prod/Dockerfile

@@ -0,0 +1,54 @@
+From ubuntu:latest
+WORKDIR workdir
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && apt-get install -y \
+    bash \
+    build-essential \
+    bzip2 \
+    csh \
+    gfortran \
+    gcc \ 
+    g++ \
+    git \
+    libgsl-dev \
+    krb5-user \
+    libpam-krb5 \
+    make \
+    unzip \
+    vim \
+    wget && \
+    apt-get clean && rm -rf /var/lib/apt/lists/*
+
+# corsika and sim_telarray
+RUN mkdir sim_telarray
+COPY corsika7.7_simtelarray.tar.gz sim_telarray
+RUN cd sim_telarray && \
+    tar -xvzf corsika7.7_simtelarray.tar.gz && \
+    ./build_all prod5 qgs2 gsl && \
+    find . -name "*.tar.gz" -exec rm -f {} \; && \
+    cd ..
+ENV SIMTELPATH="/workdir/sim_telarray/"
+
+RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-py38_4.10.3-Linux-x86_64.sh -O ~/miniconda.sh && \
+    /bin/bash ~/miniconda.sh -b -p ${WORKDIR}/conda && \
+    rm ~/miniconda.sh && \
+    ${WORKDIR}/conda/bin/conda clean -tipsy && \
+    ${WORKDIR}/conda/bin/conda install -c conda-forge mamba
+ENV PATH /${WORKDIR}/conda/bin:$PATH
+
+RUN wget --quiet https://github.com/gammasim/gammasim-tools/archive/refs/heads/master.zip && \
+    unzip master.zip && mv gammasim-tools-master gammasim-tools && \
+    rm -f master.zip && cd gammasim-tools && \
+    mv environment.yml environment.yml.tmp && \
+    grep -v "name: gammasim-tools-dev" environment.yml.tmp > environment.yml && \
+    mamba env update -n base --file environment.yml && \
+    conda clean -tipsy
+
+SHELL ["/bin/bash", "-c"]
+
+RUN source /root/.bashrc && \
+    conda init bash
+ENV PYTHONPATH=$PYTHONPATH:"/workdir/external/gammasim-tools"
+
+COPY config.yml /workdir/gammasim-tools

prod/README.md

@@ -0,0 +1,47 @@
+# Docker container for gammasim-tools production
+
+**this is work in progress and not working yet**
+
+## Introduction
+
+Provide a container for using gammasim-tools.
+
+Container contains installation of:
+
+- corsika and sim\_telarray
+- miniconda
+- packages required by gammasim-tools (from environment.yml)
+- gammasim-tools (master)
+
+## Building
+
+Building expects that a tar ball of corsika/sim\_telarray (corsika7.7\_simtelarray.tar.gz) is available in the building directory.
+Download the tar package from the MPIK website (password applies) with
+
+```
+$ ../tools/download_simulationsoftware.sh
+```
+
+Build the container with:
+
+```
+$ docker build -t gammasim-tools-prod .
+```
+
+Building will take a while and the image is large (3.9 GB; mostly due to the >2 GB conda stuff).
+
+## Running
+
+To run the container in bash 
+
+```
+$  docker run --rm -it -v "$(pwd)/external:/workdir/external" gammasim-tools-prod bash
+```
+
+In the container, find the gammasim-tools directory:
+```
+$ cd ./gammasim-tools/
+```
+
+Expect any external software (e.g., gammasim-tools) in ./external directory (see [README.md](external/README.md))
+

dev/external/config.yml → prod/config.yml

@@ -1,20 +1,22 @@
+---
 # configuration file for simtools
 
-useMongoDB: true
+useMongoDB: True
 mongoDBConfigFile: ./dbDetails.yml
 
 modelFilesLocations: [
   /workdir/external/simulation-model-description/configReports/,
   /workdir/external/simulation-model-description/datFiles/,
-  /workdir/external/data-from-instrument-teams/,
-  /workdir/external/gammasim-tools/data/test-data/
+  /workdir/external/verification-process/dataFromInstrumentTeams/
 ]
 
 outputLocation: /workdir/external/output/
 
-dataLocation: '/workdir/external/gammasim-tools/data/'
+dataLocation: './data/'
 
-simtelPath: /workdir/sim_telarray/
+simtelPath: '$SIMTELPATH'
 
 # Not used yet
 submissionCommand: 'qsub '
+
+extraCommands: []

dev/download_simulationsoftware.sh → tools/download_simulationsoftware.sh

@@ -3,7 +3,7 @@
 # download CORSIKA / sim_telarray package from 
 # MPIK page
 # 
-# requies CTA user name / password
+# requires CTA user name / password
 #
 
 VERSION="7.7"