Dockerfiles for Monasca

This repository contains resources for building and deploying a full Monasca stack in Docker environments.

Current Status

Documentation

Full documentation is available here.

Continuous Integration (CI)

The CI is carried out automatically by Github Actions. The configuration of the Workflow and Job are available here.

Automated CI Process

Every time a user sends a pull request (PR) Github-Actions will do the following:

1. A virtual machine (VM) in the cloud will be created using Ubuntu as the operating system. Docker, Docker-Compose, Python, Python-pip and the other necessary libraries will be installed in the virtual machine.

2. The CI will determine which images are affected by the PR and will build them, the rest of images are not built but downloaded from FEST Docker Hub.

3. Once all the necessary images are available in the VM, the containers will be created based on the parameters from the .env file and the docker-compose files.

   3.1. First the Keystone MySql-Keystone and Horizon containers will be created, these containers are not part of Monasca but are necessary for Monasca to run. The Keystone container have already all needed OpenStack's Users, Projects and Roles.

   3.2. Depending on the Pipeline (Metrics or Logs) the CI Script will create a set of specific containers.

4. The CI will wait until all the init containers have finished with value 0, that is, they have finished without error.

   • grafana-init
   • kafka-init
   • influx-init
   • etc...

5. Once all init containers have finished successfully their run, a series of Smoke-Tests will be executed.

2022-12-04T17:08:01.900230910Z TEST MEASUREMENTS FLOWING
2022-12-04T17:08:01.913007626Z SUCCESS
2022-12-04T17:08:01.913035772Z
2022-12-04T17:08:01.913040759Z TEST NOTIFICATION CREATION
2022-12-04T17:08:01.923244678Z SUCCESS
2022-12-04T17:08:01.923386784Z
2022-12-04T17:08:01.923449746Z TEST ALARM DEFINITION CREATION
2022-12-04T17:08:02.390359009Z SUCCESS
2022-12-04T17:08:02.390384484Z
2022-12-04T17:08:07.390497048Z TEST METRIC CREATION
2022-12-04T17:08:08.107221489Z SUCCESS
2022-12-04T17:08:08.107306802Z
2022-12-04T17:08:08.107316943Z TEST WEBHOOK TRIGGERED
2022-12-04T17:08:11.111524878Z SUCCESS
2022-12-04T17:08:11.111553571Z
2022-12-04T17:08:11.111557960Z TEST CLEANUP
2022-12-04T17:08:11.940878764Z SUCCESS
2022-12-04T17:08:11.940900658Z
2022-12-04T17:08:11.940904837Z All smoke tests passed successfully!!!

6. After all Smoke-Tests have passed successfully, an extensive series of integration tests (called Tempest-Tests) will be executed.

2022-12-04T17:08:27.798384647Z +---------------+---------------------------------------------------+
2022-12-04T17:08:27.798415804Z |      Name     |                     EntryPoint                    |
2022-12-04T17:08:27.798432387Z +---------------+---------------------------------------------------+
2022-12-04T17:08:27.798436282Z | monasca_tests | monasca_tempest_tests.plugin:MonascaTempestPlugin |
2022-12-04T17:08:27.798439671Z +---------------+---------------------------------------------------+
2022-12-04T17:08:27.881756238Z + tempest run -r monasca_tempest_tests.tests.api
2022-12-04T17:08:33.673202328Z {0} monasca_tempest_tests.tests.api.test_alarm_definitions.TestAlarmDefinitions.test_create_alarm_definition [0.296985s] ... ok
2022-12-04T17:08:33.683525287Z {0} monasca_tempest_tests.tests.api.test_alarm_definitions.TestAlarmDefinitions.test_create_alarm_definition_with_alarm_actions_exceeds_max_length [0.008862s] ... ok
2022-12-04T17:08:33.699498929Z {0} monasca_tempest_tests.tests.api.test_alarm_definitions.TestAlarmDefinitions.test_create_alarm_definition_with_description_exceeds_max_length [0.008322s] ... ok

  etc...

2022-12-04T17:15:31.901675418Z ======
2022-12-04T17:15:31.901678448Z Totals
2022-12-04T17:15:31.901681485Z ======
2022-12-04T17:15:31.901684523Z Ran: 249 tests in 418.1196 sec.
2022-12-04T17:15:31.901687639Z  - Passed: 249
2022-12-04T17:15:31.901690698Z  - Skipped: 0
2022-12-04T17:15:31.901693775Z  - Expected Fail: 0
2022-12-04T17:15:31.901696852Z  - Unexpected Success: 0
2022-12-04T17:15:31.901699929Z  - Failed: 0
2022-12-04T17:15:31.901702992Z Sum of execute time for each test: 411.3009 sec.
2022-12-04T17:15:31.901706124Z
2022-12-04T17:15:31.901709118Z ==============
2022-12-04T17:15:31.901712148Z Worker Balance
2022-12-04T17:15:31.901715198Z ==============
2022-12-04T17:15:31.901718842Z  - Worker 0 (184 tests) => 0:06:58.119552
2022-12-04T17:15:31.901725170Z  - Worker 1 (65 tests) => 0:02:53.341307

7. If all the Tempest-Tests finish successfully the Workflow will be marked in green as passed. Here you can see the status of the Workflows..

8. At this point the CI has marked the PR as passed or failed. In the event that it has been marked as passed, a user with Merge permissions can accept the PR.

9. Once the CI and at least one user with Merge permissions accept the change, a new VM will be created and the images affected by the PR will be built as in step 1. and 2.

10. Finally the new images will be published in FEST Docker Hub.

Manual CI Process

In order to execute the CI locally in an specific host follow the next steps:

Requirements

• Python and Python-PIP
• Docker and Docker-Compose
• Six and DBuild Libraries

  sudo pip install git+https://github.com/monasca/dbuild.git
  sudo pip install "six>=1.13.0"
  

Execution

1. Clone the repository

   git clone https://github.com/FujitsuEnablingSoftwareTechnologyGmbH/monasca-docker.git -b master
   cd monasca-docker
   

2. Choose metrics or logs for the pipeline and run the CI:

   python ci.py --pipeline metrics --print-logs --verbose --CI_EVENT_TYPE cron
   

3. Once the CI finished successfully, you can create a tar file with the pipeline's images.

   docker save $(docker images --format '{{.Repository}}:{{.Tag}}') -o images.tar
   

For more CI options see the CI help:

python ci.py --help
usage: ci.py [-h] -p PIPELINE [-nv] [-pl] [-v] [--CI_BRANCH CI_BRANCH]
             [--CI_EVENT_TYPE CI_EVENT_TYPE]
             [--CI_COMMIT_RANGE CI_COMMIT_RANGE]

CI command

optional arguments:
  -h, --help            show this help message and exit
  -p PIPELINE, --pipeline PIPELINE
                        Select the pipeline [metrics|logs]
  -nv, --non-voting     Set the check as non-voting
  -pl, --print-logs     Print containers logs
  -v, --verbose         Increment verbosity
  --CI_BRANCH CI_BRANCH
  --CI_EVENT_TYPE CI_EVENT_TYPE
  --CI_COMMIT_RANGE CI_COMMIT_RANGE

Execution on Vagrant's VM

In order to simplify the process a vagrant file is provided.

Requirements

• Virtualbox 6.x
• Vagrant 2.x
• vagrant plugin: vagrant-docker-compose

$ vagrant plugin install vagrant-docker-compose

Execution

1. Clone the repository

   git clone https://github.com/FujitsuEnablingSoftwareTechnologyGmbH/monasca-docker.git -b master
   cd monasca-docker
   

2. Start the Vagrant VM

   vagrant up
   

3. After finished you can see the logs inside the Vagrant VM:

   $ vagrant ssh
   $ ls monasca-logs/<yyyy-mm-dd_hh:mm:ss>/