Skip to content

Commit

Permalink
Merge pull request #822 from wireapp/release_2019_08_08
Browse files Browse the repository at this point in the history
  • Loading branch information
fisx authored Aug 9, 2019
2 parents b8818b6 + 3851485 commit cee28d2
Show file tree
Hide file tree
Showing 187 changed files with 5,709 additions and 774 deletions.
40 changes: 40 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,43 @@
# 2019-08-08 #822

## Features

- legalhold (#802), but block feature activation (#823)
- a few shell scripts for self-hosters (#805, #801)
- release nginz_disco (#759)

## Client-facing internal changes

- feature flags (starting with legalhold, sso) (#813, #818)
- SSO is disabled by default now; but enabled for all teams that already have an IdP.
- new public end-points:
- get "/teams/:tid/features/legalhold"
- get "/teams/:tid/features/sso"
- new internal end-points:
- get "/i/teams/:tid/features/legalhold"
- get "/i/teams/:tid/features/sso"
- put "/i/teams/:tid/features/legalhold"
- put "/i/teams/:tid/features/sso"
- new backoffice end-points:
- get "/teams/:tid/features/legalhold"
- get "/teams/:tid/features/sso"
- put "/teams/:tid/features/legalhold"
- put "/teams/:tid/features/sso"
- Always throw json errors, never plaintext (#722, #814)
- Register IdP: allow json bodies with xml strings (#722)

## Backend-internal changes

- [stern aka backoffice] allow galeb returning a 404 (#820)
- Cleanup logging (#816, #819)
- Canonicalize http request path capture names (#808, #809)
- Galley depends on libsodium too now (#807)
- Add generics instances to common, brig, galley types. (#804)
- Upgrade CQL protocol version to V4 (#763)
- Log last prekey used only at debug level (#785)
- Cleanup (#799)


# 2019-07-08 #798

## Internal Changes
Expand Down
6 changes: 6 additions & 0 deletions deploy/services-demo/conf/nginz/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ http {
include common_response_with_zauth.conf;
proxy_pass http://brig;
}

# Cargohold Endpoints

rewrite ^/api-docs/assets /assets/api-docs?base_url=http://127.0.0.1:8080/ break;
Expand Down Expand Up @@ -266,6 +267,11 @@ http {
proxy_pass http://galley;
}

location ~* ^/teams/([^/]*)/features/([^/]*) {
include common_response_with_zauth.conf;
proxy_pass http://galley;
}

# Gundeck Endpoints

rewrite ^/api-docs/push /push/api-docs?base_url=http://127.0.0.1:8080/ break;
Expand Down
82 changes: 82 additions & 0 deletions deploy/services-demo/create_team_members.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/usr/bin/env bash

set -e

ADMIN_UUID="a09e9521-e14e-4285-ad71-47caa97f4a16"
TEAM_UUID="9e57a378-0dca-468f-9661-7872f5f1c910"
BRIG_HOST="http://localhost:8082"
CSV_FILE="myfile.csv"

USAGE="
This bash script can be used to invite members to a given team. Input
is a csv file with email addresses and suggested user names.
Note that this uses internal brig endpoints. It is not exposed over
nginz and can only be used if you have direct access to brig.
USAGE: $0
-a <admin uuid>: User ID of the inviting admin. default: ${ADMIN_UUID}
-t <team uuid>: ID of the inviting team. default: ${TEAM_UUID}
-h <host>: Base URI of brig. default: ${BRIG_HOST}
-c <input file>: file containing info on the invitees in format 'Email,UserName'. default: ${CSV_FILE}
"

# Option parsing:
# https://sookocheff.com/post/bash/parsing-bash-script-arguments-with-shopts/
while getopts ":a:t:h:c:" opt; do
case ${opt} in
a ) ADMIN_UUID="$OPTARG"
;;
t ) TEAM_UUID="$OPTARG"
;;
h ) BRIG_HOST="$OPTARG"
;;
c ) CSV_FILE="$OPTARG"
;;
: ) echo "-$OPTARG" requires an argument 1>&2
exit 1
;;
\? ) echo "$USAGE" 1>&2
exit 1
;;
esac
done
shift $((OPTIND -1))

if [ "$#" -ne 0 ]; then
echo "$USAGE" 1>&2
exit 1
fi

if [ ! -e "$CSV_FILE" ]; then
echo -e "\n\n*** I need the name of an existing csv file.\n\n"
echo "$USAGE" 1>&2
exit 1
fi

# Generate users
while IFS=, read -r EMAIL USER_NAME
do
echo "inviting $USER_NAME <$EMAIL>..." 1>&2

# Generate the invitation
CURL_OUT_INVITATION=$(curl -i -s --show-error \
-XPOST "$BRIG_HOST/teams/$TEAM_UUID/invitations" \
-H'Content-type: application/json' \
-H'Z-User: '"$ADMIN_UUID"'' \
-d'{"email":"'"$EMAIL"'","name":"'"$USER_NAME"'","inviter_name":"Team admin"}')

INVITATION_ID=$(echo "$CURL_OUT_INVITATION" | tail -1 | sed 's/.*\"id\":\"\([a-z0-9-]*\)\".*/\1/')

echo "Created the invitation, sleeping 1 second..." 1>&2
sleep 1

if ( ( echo "$INVITATION_ID" | grep -q '"code"' ) &&
( echo "$INVITATION_ID" | grep -q '"label"' ) ) ; then
echo "Got an error, aborting: $INVITATION_ID"
exit 1
fi

echo "Sleeping 1 second..." 1>&2
sleep 1
done < "$CSV_FILE"
66 changes: 66 additions & 0 deletions deploy/services-demo/create_test_team_admins.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env bash

set -e

COUNT="1"
BRIG_HOST="http://localhost:8082"
CSV="false"

USAGE="
This bash script can be used to create active team admin users and
their teams.
Note that this uses an internal brig endpoint. It is not exposed over
nginz and can only be used if you have direct access to brig.
USAGE: $0
-n <N>: Create <N> users. default: ${COUNT}
-h <host>: Base URI of brig. default: ${BRIG_HOST}
-c: Output as headerless CSV in format 'User-Id,Email,Password'. default: ${CSV}
"

# Option parsing:
# https://sookocheff.com/post/bash/parsing-bash-script-arguments-with-shopts/
while getopts ":n:h:c" opt; do
case ${opt} in
n ) COUNT="$OPTARG"
;;
h ) BRIG_HOST="$OPTARG"
;;
c ) CSV="true"
;;
: ) echo "-$OPTARG" requires an argument 1>&2
exit 1
;;
\? ) echo "$USAGE" 1>&2
exit 1
;;
esac
done
shift $((OPTIND -1))

if [ "$#" -ne 0 ]; then
echo "$USAGE" 1>&2
exit 1
fi

# Generate users

for i in $(seq 1 "$COUNT")
do
EMAIL=$(cat /dev/urandom | env LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 8)"@example.com"
PASSWORD=$(cat /dev/urandom | env LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 8)

CURL_OUT=$(curl -i -s --show-error \
-XPOST "$BRIG_HOST/i/users" \
-H'Content-type: application/json' \
-d'{"email":"'"$EMAIL"'","password":"'"$PASSWORD"'","name":"demo","team":{"name":"Wire team","icon":"default"}}')

UUID=$(echo "$CURL_OUT" | tail -1 | sed 's/.*\"id\":\"\([a-z0-9-]*\)\".*/\1/')
TEAM=$(echo "$CURL_OUT" | tail -1 | sed 's/.*\"team\":\"\([a-z0-9-]*\)\".*/\1/')

if [ "$CSV" == "false" ]
then echo -e "Succesfully created a team admin user: $UUID on team: $TEAM with email: $EMAIL and password: $PASSWORD"
else echo -e "$UUID,$EMAIL,$PASSWORD"
fi
done
137 changes: 137 additions & 0 deletions deploy/services-demo/create_test_team_members.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
#!/usr/bin/env bash

set -e

ADMIN_UUID="a09e9521-e14e-4285-ad71-47caa97f4a16"
TEAM_UUID="9e57a378-0dca-468f-9661-7872f5f1c910"
BRIG_HOST="http://localhost:8082"
START="1"
COUNT="1"
CSV="false"
TARGET_EMAIL_DOMAIN=""

USAGE="This bash script can be used to create active members in a
given team. Every member will have an email address of the form
'w<number>@${TARGET_EMAIL_DOMAIN}', and will have to change that
(after logging in with the password provided to the user from the
output of this script).
Note that this uses internal brig endpoints. It is not exposed over
nginz and can only be used if you have direct access to brig.
USAGE: $0 -d <email domain> [OPTIONS...]
-d <email domain>: Domain part of the emails that the bogus
invitations are sent to. No default, you need
to provide that. Consider 'example.com', or an
internal domain you control.
WARNING: This may boost your reputation as a
spammer. Use with care!
-a <admin uuid>: User ID of the inviting admin. default: ${ADMIN_UUID}
-t <team uuid>: ID of the inviting team. default: ${TEAM_UUID}
-s <S>: Start at offset. default: ${START}
-n <N>: Create <N> users. default: ${COUNT}
-h <host>: Base URI of brig. default: ${BRIG_HOST}
-c: Output as headerless CSV in format 'User-Id,Email,Password'. default: ${CSV}
"

# Option parsing:
# https://sookocheff.com/post/bash/parsing-bash-script-arguments-with-shopts/
while getopts ":a:t:s:n:h:d:c" opt; do
case ${opt} in
a ) ADMIN_UUID="$OPTARG"
;;
t ) TEAM_UUID="$OPTARG"
;;
s ) START="$OPTARG"
;;
n ) COUNT="$OPTARG"
;;
h ) BRIG_HOST="$OPTARG"
;;
d ) TARGET_EMAIL_DOMAIN="$OPTARG"
;;
c ) CSV="true"
;;
: ) echo "-$OPTARG" requires an argument 1>&2
exit 1
;;
\? ) echo "$USAGE" 1>&2
exit 1
;;
esac
done
shift $((OPTIND -1))

if [ "$#" -ne 0 ]; then
echo "$USAGE" 1>&2
exit 1
fi

# Warn about sending emails

if [ "$TARGET_EMAIL_DOMAIN" == "" ]; then
echo -e "\n\n*** Please provide an email domain if you want to run this script.\n\n"
echo "$USAGE" 1>&2
exit 1
fi

# Generate users
END=$((COUNT + START - 1))
for i in $(seq "$START" "$END")
do
EMAIL='w'$(printf "%03d" "$i")"@$TARGET_EMAIL_DOMAIN"
PASSWORD=$(cat /dev/urandom | env LC_CTYPE=C tr -dc a-zA-Z0-9 | head -c 8)

# Generate the invitation

CURL_OUT_INVITATION=$(curl -i -s --show-error \
-XPOST "$BRIG_HOST/teams/$TEAM_UUID/invitations" \
-H'Content-type: application/json' \
-H'Z-User: '"$ADMIN_UUID"'' \
-d'{"email":"'"$EMAIL"'","name":"Replace with name","inviter_name":"Team admin"}')

INVITATION_ID=$(echo "$CURL_OUT_INVITATION" | tail -1 | sed 's/.*\"id\":\"\([a-z0-9-]*\)\".*/\1/')

echo "Created the invitation, sleeping 1 second..." 1>&2
sleep 1

if ( ( echo "$INVITATION_ID" | grep -q '"code"' ) &&
( echo "$INVITATION_ID" | grep -q '"label"' ) ) ; then
echo "Got an error while creating $EMAIL, aborting: $INVITATION_ID"
exit 1
fi

# Get the code
CURL_OUT_INVITATION_CODE=$(curl -i -s --show-error \
-XGET "$BRIG_HOST/i/teams/invitation-code?team=$TEAM_UUID&invitation_id=$INVITATION_ID")

INVITATION_CODE=$(echo "$CURL_OUT_INVITATION_CODE" | tail -1 | sed -n -e '/"code":/ s/^.*"\(.*\)".*/\1/p')

echo "Got the code, sleeping 1 second..." 1>&2
sleep 1

# Create the user using that code
CURL_OUT=$(curl -i -s --show-error \
-XPOST "$BRIG_HOST/i/users" \
-H'Content-type: application/json' \
-d'{"email":"'"$EMAIL"'","password":"'"$PASSWORD"'","name":"demo","team_code":"'"$INVITATION_CODE"'"}')

TEAM_MEMBER_UUID=$(echo "$CURL_OUT" | tail -1 | sed 's/.*\"id\":\"\([a-z0-9-]*\)\".*/\1/')
TEAM=$(echo "$CURL_OUT" | tail -1 | sed 's/.*\"team\":\"\([a-z0-9-]*\)\".*/\1/')

if [ "$TEAM" != "$TEAM_UUID" ]; then
echo "unexpected error: user got assigned to no / the wrong team?!"
echo ${CURL_OUT}
exit 1
fi

if [ "$CSV" == "false" ]
then echo -e "Succesfully created a team member: $TEAM_MEMBER_UUID on team: $TEAM_UUID with email: $EMAIL and password: $PASSWORD"
else echo -e "$UUID,$EMAIL,$PASSWORD"
fi

echo "Sleeping 1 second..." 1>&2
sleep 1
done
12 changes: 12 additions & 0 deletions docs/developer/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,3 +166,15 @@ docker login --username=<MY_DOCKER_USERNAME>
* [Install docker](https://docker.com)
* [Install docker-compose](https://docs.docker.com/compose/install/)
## Nix
Using Stack's [Nix integration](https://docs.haskellstack.org/en/stable/nix_integration/), Stack will take care of installing any system
dependencies automatically - including `cryptobox-c`. If new system dependencies are needed, add them to the `shell.nix` file in the project root.
Just type `$ nix-shell` and you will automatically have `make`, `docker-compose` and `stack` in `PATH`.
You can then run all the builds, and the native dependencies will be automatically present.

We are currently on a snapshot that uses `ghc863` but Nix only ships `ghc864` as `ghc863` is officially deprecated. The releases should be totally compatible, but we need to convince stack that this is true. Hence, we can add the following to our global stack config in `~/.stack/config.yaml`
```bash
skip-ghc-check: true
```
Loading

0 comments on commit cee28d2

Please sign in to comment.