Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO-1557 / Support Network Gateway #29

Merged
78 commits merged into from
Aug 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
78 commits
Select commit Hold shift + click to select a range
e063388
reenable gateway config for docker
Jun 26, 2023
8ee460f
update Gateway environment variable names
Jun 26, 2023
ee1c0d2
remove unused function
Jun 26, 2023
a20c83e
add database-migrations
Jun 28, 2023
312f2b1
update tests to reflect change in prompts
Jun 28, 2023
ead24d5
fix dependency bug
Jun 28, 2023
6a2a865
add postgres installation
Jun 29, 2023
1e484c0
add overrides for gateway images
Jun 29, 2023
945c73e
setup/docker
Jun 30, 2023
7de3704
add option to use binary genesis file
Jun 30, 2023
4c014ff
make data_aggregator depends on database_migration
Jun 30, 2023
94eefb0
autoapprove default false
Jun 30, 2023
4520491
fix mount of genesis binary file
Jun 30, 2023
dfb1d92
fixture corrections
Jun 30, 2023
da185af
ensure postgres installation via ansible works
Jun 30, 2023
c5092c6
update test prompts to include network gateway
Jul 10, 2023
129a873
Merge branch 'main' into DO-1557
Jul 10, 2023
68f4be0
Merge branch 'main' into DO-1557
Jul 27, 2023
75043db
stop logging in to gcr for ci tests
Jul 27, 2023
03b9009
add option to install postgres when running systemd
Jul 28, 2023
481af75
refactor clean up systemd config command
Jul 28, 2023
3b3d230
refactor docker commands to load settings object instead of dict
Jul 28, 2023
7ed0c81
update the release yaml
Jul 31, 2023
556f5a8
rename binary to babylonnode
Jul 31, 2023
5c52b73
refactor dockercommand to seperate arg parsing from questionary
Jul 31, 2023
72cdf96
fix the update command to use babylonnode
Jul 31, 2023
eef7ec2
finish refactoring dockercommand
Jul 31, 2023
bb40134
refactor systemd config command
Jul 31, 2023
8d533c4
refactoring
Aug 1, 2023
0d119a0
extract command argument objects
Aug 1, 2023
dcee033
refactor gateway
Aug 1, 2023
7bc7010
extract DockerCompose setup actions
Aug 1, 2023
a7d3c22
rework the way to_dict works
Aug 3, 2023
43eeb38
remove from_dict and replace by an improved constructor
Aug 3, 2023
adff829
test docker config round trip
Aug 3, 2023
a32e2da
replace the name settings with config matching the folder
Aug 3, 2023
7c7018b
Merge branch 'main' into DO-1557
Aug 3, 2023
e649c85
Merge branch 'main' into DO-1557
Aug 7, 2023
e9a4ecc
minior renaming
Aug 5, 2023
f21c0dd
ensure gateway is optional on systemd
Aug 5, 2023
01655d7
extrac common systemd config
Aug 5, 2023
162db17
recommend host ip based on ipify return value
Aug 5, 2023
931174d
add generate docker compose and keyfile commands
Aug 5, 2023
0672120
add dummy command to see sync status
Aug 5, 2023
4f456a9
fix gateway api compose errors
Aug 7, 2023
c9c9bfc
move tests fix snyk vulnerabilit
Aug 7, 2023
b8f220e
fix keygen test
Aug 7, 2023
0bf1f74
...
Aug 7, 2023
f5cf747
fix constructor creates dict bug
Aug 7, 2023
36ae587
fix bug where versions did not get asked or added
Aug 7, 2023
12545c2
break and fix everything
Aug 7, 2023
e854fd1
migrate all datastructures to the new constructor format
Aug 7, 2023
9d12eed
chown on systemd install
Aug 8, 2023
5b5cc92
chown docker install
Aug 8, 2023
8c08194
syntax fix
Aug 8, 2023
21c7813
update compose version
Aug 8, 2023
6f3fe98
update compose fixure version
Aug 8, 2023
17f596b
fix keydetails not showing
Aug 8, 2023
2fdabf2
fix docker install command
Aug 8, 2023
3a9ec79
i dont know anymore
Aug 8, 2023
efc63ca
add feedback from santiago
Aug 8, 2023
10d6997
commit after successfully testing the monitoring setup
Aug 8, 2023
62511b9
fix keygen bug
Aug 8, 2023
0431419
chown the keyfile directly after creation
Aug 8, 2023
3363844
upload babylonnode instead of radixnode
Aug 8, 2023
c297c65
rename everything to babylonnode
Aug 8, 2023
055822d
update e2e tests
Aug 9, 2023
8c96292
change behaviour of docker install not finding existing compose file
Aug 9, 2023
6fc3e9a
return empty dict on compose does not exist
Aug 9, 2023
04ba357
add core path to coreapi url
Aug 9, 2023
8ab78b5
remove echo command
Aug 9, 2023
5494cd9
change import order of getpass
Aug 9, 2023
d005eb7
reenable GATEWAY for systemd
Aug 9, 2023
4f91a9a
update default values of migration config
Aug 9, 2023
b7f9d86
fix conditionally install postgresql command
Aug 9, 2023
002904c
fix bug when keydetails where not read
Aug 9, 2023
8e11ffc
add human readable diff function
Aug 9, 2023
794b2be
remove tests about unnused functions
Aug 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 65 additions & 87 deletions .github/workflows/ci.yml

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions node-runner-cli/Dockerfile.ubuntufocal
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt

COPY . /app
RUN pyinstaller --onefile --windowed radixnode.spec
RUN pyinstaller --onefile --windowed babylonnode.spec

RUN DISABLE_VERSION_CHECK=true /app/dist/radixnode version
RUN DISABLE_VERSION_CHECK=true /app/dist/babylonnode version

FROM scratch AS export-stage
COPY --from=BUILD /app/dist /
4 changes: 2 additions & 2 deletions node-runner-cli/Dockerfile.ubuntujammy
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ COPY requirements.txt /app/requirements.txt
RUN pip install -r requirements.txt

COPY . /app
RUN pyinstaller --onefile --windowed radixnode.spec
RUN pyinstaller --onefile --windowed babylonnode.spec

RUN DISABLE_VERSION_CHECK=true /app/dist/radixnode version
RUN DISABLE_VERSION_CHECK=true /app/dist/babylonnode version

FROM scratch AS export-stage
COPY --from=BUILD /app/dist /
35 changes: 32 additions & 3 deletions node-runner-cli/ansible/project/provision.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,19 @@
when: setup_swap | bool

- block:
- name: Ensure gpg is installed
apt:
name: gpg
- name: Add repository signing key
apt_key:
url: "https://www.postgresql.org/media/keys/ACCC4CF8.asc"
state: present
- name: Add postgresql repository
apt_repository:
repo: "deb https://apt.postgresql.org/pub/repos/apt/ {{ ansible_distribution_release }}-pgdg main"
state: present
filename: pgdg

- name: install PostgreSQL
ansible.builtin.package:
name:
Expand All @@ -84,11 +97,11 @@
become_user: "{{ postgresql_user }}"
community.postgresql.postgresql_user:
name: "{{ postgresql_user }}"
password: "{{postgress_password}}"
password: "{{ postgress_password }}"

- name: Grant
community.postgresql.postgresql_pg_hba:
dest: /etc/postgresql/12/main/pg_hba.conf
dest: /etc/postgresql/{{ postgresql_version }}/main/pg_hba.conf
contype: host
users: all
source: 0.0.0.0/0
Expand All @@ -110,5 +123,21 @@
- name: restart postgresql
service: name=postgresql state=restarted enabled=yes

- name: Create the database specified in vars
become: true
become_user: "{{ postgresql_user }}"
postgresql_db: name={{ postgresql_db_name }}
template='template0'
state=present

- name: Ensure user has access to the new database
become: true
become_user: "{{ postgresql_user }}"
postgresql_user: db={{ postgresql_db_name }}
name={{ postgresql_user }}
password={{ postgress_password }}
priv=ALL
state=present

become: true
when: postgres_local | bool
when: postgres_local | bool
2 changes: 1 addition & 1 deletion node-runner-cli/api/Api.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#TODO this needs updating when a new python client is created
import os

from env_vars import PRINT_RESPONSE, NGINX, NODE_END_POINT
from config.EnvVars import PRINT_RESPONSE, NGINX, NODE_END_POINT
from utils.utils import Helpers


Expand Down
2 changes: 1 addition & 1 deletion node-runner-cli/api/CustomAPIClient.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ def __init__(self, host=None, username=None, password=None, verify_ssl=True):

self.verify_ssl = verify_ssl

self.user_agent = 'Babylon radixnode cli'
self.user_agent = 'Babylon babylonnode cli'
self.default_headers = {}
self.set_default_header('User-Agent', self.user_agent)

Expand Down
10 changes: 5 additions & 5 deletions node-runner-cli/radixnode.py → node-runner-cli/babylonnode.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from commands.othercommands import other_command_cli
from commands.systemapi import handle_systemapi
from commands.systemdcommand import systemdcli
from env_vars import DISABLE_VERSION_CHECK
from config.EnvVars import DISABLE_VERSION_CHECK
from github.github import latest_release
from utils.utils import Helpers

Expand All @@ -40,13 +40,13 @@ def check_latest_cli():
if Helpers.cli_version() != cli_latest_version:
os_name = "ubuntu-22.04"
print(
f"Radixnode CLI latest version is {cli_latest_version} and current version of the binary is {Helpers.cli_version()}.\n.")
f"babylonnode CLI latest version is {cli_latest_version} and current version of the binary is {Helpers.cli_version()}.\n.")
print(f"""
---------------------------------------------------------------
Update the CLI by running these commands
wget -O radixnode https://github.com/radixdlt/babylon-nodecli/releases/download/{cli_latest_version}/radixnode-{os_name}
chmod +x radixnode
sudo mv radixnode /usr/local/bin
wget -O babylonnode https://github.com/radixdlt/babylon-nodecli/releases/download/{cli_latest_version}/radixnode-{os_name}
chmod +x babylonnode
sudo mv babylonnode /usr/local/bin
""")


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
block_cipher = None


a = Analysis(['radixnode.py'],
a = Analysis(['babylonnode.py'],
pathex=['.'],
binaries=[],
datas=[('./templates/*.j2', 'templates'),('./testnet-genesis/*', 'testnet-genesis')],
Expand All @@ -24,7 +24,7 @@ exe = EXE(pyz,
a.zipfiles,
a.datas,
[],
name='radixnode',
name='babylonnode',
debug=False,
bootloader_ignore_signals=False,
strip=False,
Expand Down
14 changes: 7 additions & 7 deletions node-runner-cli/commands/authcommand.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
from argparse import ArgumentParser

from commands.subcommand import get_decorator, argument
from setup.Docker import Docker
from setup.SystemD import SystemD
from setup.DockerSetup import DockerSetup
from setup.SystemDSetup import SystemDSetup

authcli = ArgumentParser(
description='Subcommand to aid creation of nginx basic auth users',
usage="radixnode auth "
usage="babylonnode auth "
)

auth_parser = authcli.add_subparsers(dest="authcommand")
Expand Down Expand Up @@ -88,10 +88,10 @@ def set_gateway_password(args):

def set_auth(args, usertype, password=None):
if args.setupmode == "DOCKER":
Docker.setup_nginx_Password(usertype, args.username, password)
DockerSetup.setup_nginx_Password(usertype, args.username, password)
elif args.setupmode == "SYSTEMD":
SystemD.checkUser()
SystemD.install_nginx()
SystemD.setup_nginx_password("/etc/nginx/secrets", usertype, args.username, password)
SystemDSetup.checkUser()
SystemDSetup.install_nginx()
SystemDSetup.setup_nginx_password("/etc/nginx/secrets", usertype, args.username, password)
else:
print("Invalid setupmode specified. It should be either DOCKER or SYSTEMD.")
2 changes: 1 addition & 1 deletion node-runner-cli/commands/coreapi.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
#
# corecli = ArgumentParser(
# description='Subcommand to aid interaction with core api',
# usage="radixnode api core "
# usage="babylonnode api core "
# )
# core_parser = corecli.add_subparsers(dest="corecommand")
#
Expand Down
Loading
Loading