Change integration testing

[jamshale]

Changes the automated integration testing so that there is 3 different types of tests.

1. BDD Interop tests (AATH)- During a PR the workflow will clone AATH and then locate the forked repo location. Adjust the target for the tests to the forked repo and branch and run the acapy <-> acapy tests. These take about 43 mins. A little bit longer than the existing integration tests. These tests are more thorough and we want to encourage coverage here for the interop profile. The biggest problem here is that tests will need to be updated in AATH if they fail from aca-py, but I think that this shouldn't happen often as the interop protocols should be stable.
2. Local BDD Tests - These are the current integration tests, but they have been scoped down considerably so they don't cover the same things as the AATH tests. They will still be available for those who like the Behave/Gherkin and demo agent style of tests. These only take about 6 minutes now and only include the endorsement tests and the anoncreds upgrade. We could still make it so all the tests run on releases. I want to make sure we aren't losing coverage removing all these tests, but I don't think we are.
3. Scenario testing - These tests will use the acapy-controller (minimal-example) library created by Indicio. The examples have been copied from their repo as base tests. I thought about doing something trickier by cloning the repo but don't think it's necessary. If Indicio wants to copy the tests from acapy or vice versa then we can just copy and paste them. These tests only take about 3 minutes right now.

The scenario testing could probably be extended to have for e2e tests like what happens in the aries-acapy-tools repo. I imagine being able to test things like shut downs and upgrades and changing configurations from this setup.

TODO:

• Adjust the AATH tags and run a smaller subset on PR's
• What to do for nightly/release/manual workflow runs? Currently same as existing.
• Tests are all pointing to http://test.bcovrin.vonx.io/ ledger and tails server. Maybe change to dev so we can wipe the env more often?

[jamshale]

The remaining warnings aren't an issue. Using http inside the test container network.

[esune]

Left a couple minor comments, but overall looks good to me.
Still wondering about executing ALL interop tests every time a PR gets opened/updated/reopened as it is potentially a big overhead and use of executor time. We could maybe create a matrix mapping code paths with test tags and dynamically specify which sets to execute (e.g.: only issue-credential and not connections) if nothing has changed, but this would definitely be for a future improvement 🤔

[jamshale]

Left a couple minor comments, but overall looks good to me. Still wondering about executing ALL interop tests every time a PR gets opened/updated/reopened as it is potentially a big overhead and use of executor time. We could maybe create a matrix mapping code paths with test tags and dynamically specify which sets to execute (e.g.: only issue-credential and not connections) if nothing has changed, but this would definitely be for a future improvement 🤔

I do think that's a good idea and I could do it in an upcoming PR. I think we should only run a subset that only includes most of the happy path flows in AATH and the new scenarios. They will catch a vast majority of mistakes. Then for nightly and release runs we could do a more thorough testing suite on off hours.

The ultimate way would be to only run tests which matched the code changes. I don't think there's really a path forward for this with how complicated the codebase can be.

[jamshale]

@esune I fixed the spelling mistakes and change the acapy-controller dependency in scenarios to target a release.

I'm going to create 3 tickets for additional work.

1. Scoping the PR tests from AATH down
2. Only run integration tests when there's source code or testing changes.
3. Rework the nightly and release testing with scenarios and AATH testing.

[sonarcloud]

Quality Gate failed

Failed conditions
15 Security Hotspots
0.0% Coverage on New Code (required ≥ 80%)
9.8% Duplication on New Code (required ≤ 3%)

See analysis details on SonarCloud