Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NU-1740] preparing quickstart-setup to be extracted as an external image #187

Merged
merged 18 commits into from
Aug 23, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
@@ -1 +1 @@
NUSSKNACKER_VERSION=1.16.0
NUSSKNACKER_VERSION=preview_NU-1740_scenario-types-reload_testing_1-latest
mk-software-pl marked this conversation as resolved.
Show resolved Hide resolved
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@

## About

This repo contains quick start showcases for [Nussknacker](https://nussknacker.io), a visual tool to define and run real-time decision algorithms. The Nussknacker repo can be found [here](https://github.com/TouK/nussknacker).
This repo contains quick start showcases for [Nussknacker](https://nussknacker.io), a visual tool to define and run real-time decision algorithms. The Nussknacker repository can be found [here](https://github.com/TouK/nussknacker).

 
## Quickstart documentation

This Quickstart documentation can be found at [Nussknacker's documentation site](https://nussknacker.io/documentation/quickstart/docker/).
If you don't want to run it on your machine you can check our [Online Demo](https://nussknacker.io/documentation/quickstart/demo/) (but it's read-only, so rather limited) or try our [Nusskacker in our Cloud](https://nussknacker.io/documentation/quickstart/cloud/).
To figure out what's in the Quickstart and how to run it, please see its documentation at the [Nussknacker's site](https://nussknacker.io/documentation/quickstart/docker/).

You may also be interested in our [Online Demo](https://nussknacker.io/documentation/quickstart/demo/) (it's read-only, so rather limited) or trying out [Nusskacker in our Cloud](https://nussknacker.io/documentation/quickstart/cloud/).

 
## What's next?
Expand Down
29 changes: 29 additions & 0 deletions designer/additional-configuration.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
## To customize Nu Designer configuration see https://nussknacker.io/documentation/docs/configuration/Common/#configuration-file

scenarioTypes {
"streaming" {
# customize Flink streaming scenario type
}
"streaming-lite-embedded" {
# customize Lite streaming scenario type
}
"request-response-embedded": {
# customize Lite request-response scenario type
}
}

## Additional configuration of Nu Designer Upper Menu
tabs: ${tabs} [
{
id: "data",
title: "Data",
url: "/akhq/ui/nussknacker-kafka/topic",
type: "IFrame"
},
{
id: "flink",
title: "Flink",
url: "/flink/#/overview",
type: "IFrame"
}
]
54 changes: 0 additions & 54 deletions designer/application-customizations.conf

This file was deleted.

29 changes: 11 additions & 18 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,23 +10,15 @@ services:
depends_on:
nginx:
condition: service_healthy
designer:
condition: service_healthy
volumes:
- nussknacker_designer_shared_configuration:/opt/nussknacker/conf/
deploy:
resources:
limits:
memory: 256M

### Quickstart mocks
mocks:
build:
context: mocks/
dockerfile: Dockerfile
# todo added temporary
ports:
- 18080:8080
deploy:
resources:
limits:
memory: 256M
cpus: '0.5'

### Nussknacker-related services

Expand Down Expand Up @@ -59,7 +51,7 @@ services:
image: touk/nussknacker:${NUSSKNACKER_VERSION:?NUSSKNACKER_VERSION must be defined}_scala-2.12
restart: unless-stopped
environment:
CONFIG_FILE: "/opt/nussknacker/conf/application.conf,/opt/nussknacker/conf/application-customizations.conf"
CONFIG_FILE: "/opt/nussknacker/conf/application.conf,/opt/nussknacker/conf/additional-configuration.conf,/opt/nussknacker/conf/application-customizations.conf"
DB_URL: "jdbc:postgresql://postgres:5432/nu-db"
DB_DRIVER: "org.postgresql.Driver"
DB_USER: "nu"
Expand All @@ -84,16 +76,15 @@ services:
condition: service_healthy
flink-taskmanager:
condition: service_started
mocks:
condition: service_healthy
expose:
- 8181
healthcheck:
test: [ "CMD-SHELL", "curl localhost:8080/api/app/healthCheck" ]
interval: 10s
retries: 10
volumes:
- ./designer/application-customizations.conf:/opt/nussknacker/conf/application-customizations.conf
- nussknacker_designer_shared_configuration:/opt/nussknacker/conf
- ./designer/additional-configuration.conf:/opt/nussknacker/conf/additional-configuration.conf
volumes_from:
- flink-jobmanager
deploy:
Expand Down Expand Up @@ -222,7 +213,7 @@ services:
deploy:
resources:
limits:
memory: 256M
memory: 512M

### FLINK-related services

Expand Down Expand Up @@ -280,5 +271,7 @@ services:
volumes:
nussknacker_designer_data:
name: nussknacker_designer_data
nussknacker_designer_shared_configuration:
name: nussknacker_designer_shared_configuration
nussknacker_flink_data:
name: nussknacker_flink_data
2 changes: 1 addition & 1 deletion flink/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG FLINK_VERSION
ARG FLINK_VERSION="FLINK_VERSION_IS_UNDEFINED"

FROM curlimages/curl:8.9.1 AS lib_provider

Expand Down
41 changes: 0 additions & 41 deletions mocks/Dockerfile

This file was deleted.

6 changes: 0 additions & 6 deletions mocks/db/scripts/common.sh

This file was deleted.

42 changes: 0 additions & 42 deletions mocks/db/scripts/configure.sh

This file was deleted.

52 changes: 42 additions & 10 deletions quickstart-setup/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,18 +1,50 @@
FROM bitnami/kafka:3.7.0
FROM holomekc/wiremock-gui:3.8.1 AS wiremock

USER root
RUN apt-get update && \
apt-get install -y wget && \
wget -P /var/wiremock/extensions https://repo1.maven.org/maven2/org/wiremock/extensions/wiremock-faker-extension-standalone/0.2.0/wiremock-faker-extension-standalone-0.2.0.jar

RUN apt update && \
apt install -y curl jq kafkacat less && \
rm -rf /var/lib/apt/lists/*
FROM phusion/baseimage:noble-1.0.0

# Use baseimage-docker's init system.
CMD ["/sbin/my_init"]

WORKDIR /app

COPY entrypoint.sh /
COPY scripts/ /app/scripts/
COPY data/ /app/data/
COPY run.sh /etc/service/setup/run

COPY setup/ /app/setup/
COPY mocks/ /app/mocks/
COPY data/ /app/data/
COPY utils/ /app/utils/

# WIREMOCK & POSTGRES
COPY --from=wiremock /var/wiremock /var/wiremock
COPY --from=wiremock /home/wiremock /home/wiremock

USER root

RUN apt update && \
apt install -y --no-install-recommends curl ca-certificates jq less && \
install -d /usr/share/postgresql-common/pgdg && \
curl -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc --fail https://www.postgresql.org/media/keys/ACCC4CF8.asc && \
echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] https://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list && \
apt update -y && \
apt -y install postgresql-16 && \
apt -y install openjdk-11-jre-headless && \
apt clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && \
curl https://raw.githubusercontent.com/birdayz/kaf/master/godownloader.sh | BINDIR=/bin bash && \
# wiremock service
mv /app/mocks/http-service/mocks/ /home/wiremock/ && \
mkdir -p /etc/service/http-service && \
mv /app/mocks/http-service/scripts/run-wiremock.sh /etc/service/http-service/run && \
# postgres service
mv /app/mocks/db/mocks /home/postgres && \
mkdir /etc/service/postgres && \
mv /app/mocks/db/scripts/run-postgres.sh /etc/service/postgres/run

HEALTHCHECK --interval=10s --timeout=1s --retries=12 CMD /bin/bash -c 'test -f "/app/healthy"'
EXPOSE 8080
EXPOSE 5432

ENTRYPOINT [ "/entrypoint.sh" ]
HEALTHCHECK --interval=10s --timeout=1s --retries=12 --start-period=30s \
CMD (/bin/bash -c 'test -f "/app/healthy"' && pg_isready -d mocks -U mocks && curl -f http://localhost:8080/__admin/) || exit 1
4 changes: 3 additions & 1 deletion mocks/Readme.md → quickstart-setup/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@
## Resources:
https://github.com/wiremock/wiremock-faker-extension/blob/main/docs/reference.md
https://docs.wiremock.io/response-templating/basics/
https://docs.wiremock.io/response-templating/dates-and-times/
https://docs.wiremock.io/response-templating/dates-and-times/

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What with this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a reminder for me. I'm going to write a readme in the next PR (in which I will introduce the docker image with quickstart-setup).

# todo: delete
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ function runRequestSending() {
echo "Starting to send to '$OPENAPI_SERVICE_SLUG' OpenAPI service, requests generated by '$REQUEST_GENERATOR_SCRIPT' generator script"

mkdir -p /var/log/continuously-send-http-requests
nohup ../utils/http/continuously-send-http-requests.sh "$OPENAPI_SERVICE_SLUG" "$REQUEST_GENERATOR_SCRIPT" > /var/log/continuously-send-http-requests/output.log 2>&1 &
nohup ../../utils/http/continuously-send-http-requests.sh "$OPENAPI_SERVICE_SLUG" "$REQUEST_GENERATOR_SCRIPT" > /var/log/continuously-send-http-requests/output.log 2>&1 &
}

echo "Starting to send generated requests to Nu OpenAPI services ..."
Expand All @@ -27,12 +27,12 @@ while IFS= read -r OPENAPI_SERVICE_SLUG; do
continue
fi

REQUEST_GENERATOR_SCRIPT=$(find ../../data/http/generate-requests -iname "$OPENAPI_SERVICE_SLUG.sh" | head)
REQUEST_GENERATOR_SCRIPT=$(find generate-requests -iname "$OPENAPI_SERVICE_SLUG.sh" | head)

if [[ -f "$REQUEST_GENERATOR_SCRIPT" ]]; then
runRequestSending "$OPENAPI_SERVICE_SLUG" "$(realpath $REQUEST_GENERATOR_SCRIPT)"
fi

done < "../../data/http/slugs.txt"
done < "slugs.txt"

echo -e "DONE!\n\n"
2 changes: 1 addition & 1 deletion quickstart-setup/data/http/generate-requests/loan.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

cd "$(dirname "$0")"

source ../../../scripts/utils/lib.sh
source ../../../utils/lib.sh

ID="$(random_4digit_number)"
AMOUNT="$(random_4digit_number)"
Expand Down
Loading
Loading