Merge branch 'develop' into revert/push-gha-metrics-action-removal

.github/E2E_TESTS_ON_GITHUB_CI.md

@@ -1,58 +1,90 @@
 # E2E Tests on GitHub CI
 
-E2E tests are executed on GitHub CI using the [E2E Tests Reusable Workflow](#about-the-reusable-workflow) or dedicated workflows.
+- [E2E Tests on GitHub CI](#e2e-tests-on-github-ci)
+ - [Scheduled test workflows](#scheduled-test-workflows)
+ - [PR E2E Tests](#pr-e2e-tests)
+ - [Nightly E2E Tests](#nightly-e2e-tests)
+ - [Release E2E Tests](#release-e2e-tests)
+ - [Integration (smoke) Tests](#integration-smoke-tests)
+ - [Client Compatibility Tests](#client-compatibility-tests)
+ - [On-Demand Workflows](#on-demand-workflows)
+ - [Test workflows setup in CI](#test-workflows-setup-in-ci)
+ - [Configuration Overrides](#configuration-overrides)
+ - [Test Secrets](#test-secrets)
 
-## Automatic workflows
+E2E tests are executed on GitHub CI using the [E2E Tests Reusable Workflow](https://github.com/smartcontractkit/.github/blob/main/.github/workflows/README.md) or dedicated workflows.
 
-These workflows are designed to run automatically at crucial stages of the software development process, such as on every commit in a PR, nightly or before release.
+## Scheduled test workflows
+
+These workflows are designed to run on every commit in a PR, nightly or before release (see `triggers` in [e2e-tests.yaml](./e2e-tests.yml)).
 
 ### PR E2E Tests
 
-Run on every commit in a PR to ensure changes do not introduce regressions.
+Tests triggered on every commit in a PR to ensure changes do not introduce regressions.
 
-[Link](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/integration-tests.yml)
+**Workflow:** [integration-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/integration-tests.yml)
 
 ### Nightly E2E Tests
 
-Conducted nightly to catch issues that may develop over time or with accumulated changes.
+Nightly E2E test runs.
 
-[Link](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-nightly-e2e-tests.yml)
+**Workflow:** [nightly-e2e-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-nightly-e2e-tests.yml)
 
 ### Release E2E Tests
 
-This section contains automatic workflows triggering E2E tests at release.
+E2E tests triggered on a release tag.
+
+#### Integration (smoke) Tests
+
+**Workflow:** [integration-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/integration-tests.yml)
 
 #### Client Compatibility Tests
 
-[Link](https://github.com/smartcontractkit/chainlink/actions/workflows/client-compatibility-tests.yml)
+**Workflow:** [client-compatibility-tests.yml](https://github.com/smartcontractkit/chainlink/actions/workflows/client-compatibility-tests.yml)
 
 ## On-Demand Workflows
 
-Triggered manually by QA for specific testing needs.
+These are dispatched parametrized workflows, that may be triggered manually for specific testing needs. For more details refer [integration-tests README](../integration-tests/README.md) and [per-product test run books](../integration-tests/run-books/).
 
 **Examples:**
 
+- [Selected E2E Tests Workflow](https://github.com/smartcontractkit/chainlink/actions/workflows/run-selected-e2e-tests.yml)
+- [Client Compatibility Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/client-compatibility-tests.yml)
+- [Chaos Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/integration-chaos-tests.yml)
+- [OCR Soak Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/on-demand-ocr-soak-test.yml)
 - [On-Demand Automation Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/automation-ondemand-tests.yml)
 - [CCIP Chaos Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/ccip-chaos-tests.yml)
-- [OCR Soak Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/on-demand-ocr-soak-test.yml)
+- [CCIP Load Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/ccip-load-tests.yml)
 - [VRFv2Plus Smoke Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/on-demand-vrfv2plus-smoke-tests.yml)
+- [VRFv2Plus Performance Tests](https://github.com/smartcontractkit/chainlink/actions/workflows/on-demand-vrfv2plus-performance-test.yml)
+
+### Test workflows setup in CI
 
-## Test Configs
+Most workflows may be triggered with default configs. Some, nevertheless, may be overridden.
 
-E2E tests utilize TOML files to define their parameters. Each test is equipped with a default TOML config, which can be overridden by specifying an alternative TOML config. This allows for running tests with varied parameters, such as on a non-default blockchain network. For tests executed on GitHub CI, both the default configs and any override configs must reside within the git repository. The `test_config_override_path` workflow input is used to provide a path to an override config.
+> [!TIP]
+> Use `gh` CLI commands to run workflows from local machine.
 
-Config overrides should be stored in `testconfig/*/overrides/*.toml`. Placing files here will not trigger a rebuild of the test runner image.
+#### Configuration Overrides
 
-**Important Note:** The use of `base64Config` input is deprecated in favor of `test_config_override_path`. For more details, refer to [the decision log](https://smartcontract-it.atlassian.net/wiki/spaces/TT/pages/927596563/Storing+All+Test+Configs+In+Git).
+> [!CAUTION]
+> Test configurations should not keep any [sensitive data or secrets](#test-secrets).
 
-To learn more about test configs see [CTF Test Config](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md).
+1. Reference sources:
+ 1. [Integration-Tests configurations](../integration-tests/testconfig/README.md);
+ 2. [CTF Test Config](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md).
+2. Defaults and overrides should be stored (committed) in repository under `../integration-tests/testconfig/<product>/overrides/<override-name>.toml` (see example [here](../integration-tests/testconfig/ocr2/overrides/base_sepolia.toml)).
+3. Use `test_config_override_path` to point to an override config. For example: `test_config_override_path="testconfig/ocr2/overrides/base_sepolia.toml"`
 
-## Test Secrets
+#### Test Secrets
 
-For security reasons, test secrets and sensitive information are not stored directly within the test config TOML files. Instead, these secrets are securely injected into tests using environment variables. For a detailed explanation on managing test secrets, refer to our [Test Secrets documentation](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#test-secrets).
+> [!CAUTION]
+> Pay attention to never store/expose/commit your test secrets in repository.
 
-If you need to run a GitHub workflow using custom secrets, please refer to the [guide on running GitHub workflows with your test secrets](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#run-github-workflow-with-your-test-secrets).
+Test secrets allow provisioning and override the sensitive data such as EOA's private key, RPCs, Docker registry links, etc.
 
-## About the E2E Test Reusable Workflow
+Reference sources:
 
-For information on the E2E Test Reusable Workflow, visit the documentation in the [smartcontractkit/.github repository](https://github.com/smartcontractkit/.github/blob/main/.github/workflows/README.md).
+1. [BASE64_CONFIG_OVERRIDE](../integration-tests/testconfig/README.md#base64_config_override).
+2. [CTF Test Secrets documentation](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#test-secrets).
+3. [Guide on running GitHub workflows with your test secrets](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#run-github-workflow-with-your-test-secrets).

.github/actions/golangci-lint/action.yml

@@ -80,6 +80,7 @@ runs:
  with:
  name: golangci-lint-report
  path: ${{ inputs.go-directory }}/golangci-lint-report.xml
+ retention-days: 7
  - name: Collect Metrics
  if: always()
  uses: smartcontractkit/push-gha-metrics-action@d9da21a2747016b3e13de58c7d4115a3d5c97935 # v3.0.1

.github/workflows/ci-core.yml

@@ -224,6 +224,7 @@ jobs:
  ./race.*
  ./coverage.txt
  ./postgres_logs.txt
+ retention-days: 7
  - name: Notify Slack
  if: ${{ failure() && steps.print-races.outputs.post_to_slack == 'true' && matrix.type.cmd == 'go_core_race_tests' && (github.event_name == 'merge_group' || github.ref == 'refs/heads/develop') && needs.filter.outputs.changes == 'true' }}
  uses: slackapi/slack-github-action@6c661ce58804a1a20f6dc5fbee7f0381b469e001 # v1.25.0
@@ -343,6 +344,7 @@ jobs:
  name: flakey_test_runner_logs
  path: |
  ./output.txt
+ retention-days: 7
 
  scan:
  name: SonarQube Scan

GNUmakefile

@@ -113,6 +113,10 @@ testscripts: chainlink-test ## Install and run testscript against testdata/scrip
 testscripts-update: ## Update testdata/scripts/* files via testscript.
  make testscripts TS_FLAGS="-u"
 
+.PHONY: start-testdb
+start-testdb:
+ docker run --name test-db-core -p 5432:5432 -e POSTGRES_PASSWORD=postgres -d postgres
+
 .PHONY: setup-testdb
 setup-testdb: ## Setup the test database.
  ./core/scripts/setup_testdb.sh

core/capabilities/integration_tests/framework/capabilities_registry.go

@@ -0,0 +1,129 @@
+package framework
+
+import (
+ "context"
+
+ "github.com/ethereum/go-ethereum"
+ "github.com/ethereum/go-ethereum/accounts/abi/bind"
+ "github.com/ethereum/go-ethereum/common"
+ "google.golang.org/protobuf/proto"
+
+ "github.com/smartcontractkit/chainlink-common/pkg/capabilities/pb"
+ "github.com/smartcontractkit/chainlink-common/pkg/values"
+ kcr "github.com/smartcontractkit/chainlink/v2/core/gethwrappers/keystone/generated/capabilities_registry"
+
+ "testing"
+
+ "github.com/stretchr/testify/require"
+)
+
+type CapabilitiesRegistry struct {
+ t *testing.T
+ backend *EthBlockchain
+ contract *kcr.CapabilitiesRegistry
+ addr common.Address
+ nodeOperatorID uint32
+ nextDonID int
+}
+
+func NewCapabilitiesRegistry(ctx context.Context, t *testing.T, backend *EthBlockchain) *CapabilitiesRegistry {
+ addr, _, contract, err := kcr.DeployCapabilitiesRegistry(backend.transactionOpts, backend)
+ require.NoError(t, err)
+ backend.Commit()
+
+ _, err = contract.AddNodeOperators(backend.transactionOpts, []kcr.CapabilitiesRegistryNodeOperator{
+ {
+ Admin: backend.transactionOpts.From,
+ Name: "TEST_NODE_OPERATOR",
+ },
+ })
+ require.NoError(t, err)
+ blockHash := backend.Commit()
+
+ logs, err := backend.FilterLogs(ctx, ethereum.FilterQuery{
+ BlockHash: &blockHash,
+ FromBlock: nil,
+ ToBlock: nil,
+ Addresses: nil,
+ Topics: nil,
+ })
+
+ require.NoError(t, err)
+
+ recLog, err := contract.ParseNodeOperatorAdded(logs[0])
+ require.NoError(t, err)
+
+ nopID := recLog.NodeOperatorId
+
+ return &CapabilitiesRegistry{t: t, addr: addr, contract: contract, backend: backend, nodeOperatorID: nopID}
+}
+
+func (r *CapabilitiesRegistry) getAddress() common.Address {
+ return r.addr
+}
+
+type capability struct {
+ donCapabilityConfig *pb.CapabilityConfig
+ registryConfig kcr.CapabilitiesRegistryCapability
+}
+
+// SetupDON sets up a new DON with the given capabilities and returns the DON ID
+func (r *CapabilitiesRegistry) setupDON(donInfo DonConfiguration, capabilities []capability) int {
+ var hashedCapabilityIDs [][32]byte
+
+ for _, c := range capabilities {
+ id, err := r.contract.GetHashedCapabilityId(&bind.CallOpts{}, c.registryConfig.LabelledName, c.registryConfig.Version)
+ require.NoError(r.t, err)
+ hashedCapabilityIDs = append(hashedCapabilityIDs, id)
+ }
+
+ var registryCapabilities []kcr.CapabilitiesRegistryCapability
+ for _, c := range capabilities {
+ registryCapabilities = append(registryCapabilities, c.registryConfig)
+ }
+
+ _, err := r.contract.AddCapabilities(r.backend.transactionOpts, registryCapabilities)
+ require.NoError(r.t, err)
+
+ r.backend.Commit()
+
+ nodes := []kcr.CapabilitiesRegistryNodeParams{}
+ for _, peerID := range donInfo.peerIDs {
+ n, innerErr := peerToNode(r.nodeOperatorID, peerID)
+ require.NoError(r.t, innerErr)
+
+ n.HashedCapabilityIds = hashedCapabilityIDs
+ nodes = append(nodes, n)
+ }
+
+ _, err = r.contract.AddNodes(r.backend.transactionOpts, nodes)
+ require.NoError(r.t, err)
+ r.backend.Commit()
+
+ ps, err := peers(donInfo.peerIDs)
+ require.NoError(r.t, err)
+
+ var capabilityConfigurations []kcr.CapabilitiesRegistryCapabilityConfiguration
+ for i, c := range capabilities {
+ configBinary, err2 := proto.Marshal(c.donCapabilityConfig)
+ require.NoError(r.t, err2)
+
+ capabilityConfigurations = append(capabilityConfigurations, kcr.CapabilitiesRegistryCapabilityConfiguration{
+ CapabilityId: hashedCapabilityIDs[i],
+ Config: configBinary,
+ })
+ }
+
+ _, err = r.contract.AddDON(r.backend.transactionOpts, ps, capabilityConfigurations, true, donInfo.AcceptsWorkflows, donInfo.F)
+ require.NoError(r.t, err)
+ r.backend.Commit()
+
+ r.nextDonID++
+ return r.nextDonID
+}
+
+func newCapabilityConfig() *pb.CapabilityConfig {
+ return &pb.CapabilityConfig{
+ DefaultConfig: values.Proto(values.EmptyMap()).GetMapValue(),
+ }
+}