diff --git a/docusaurus/docs/develop/developer_guide/test_suites.md b/docusaurus/docs/develop/developer_guide/test_suites.md deleted file mode 100644 index bbe6a63df..000000000 --- a/docusaurus/docs/develop/developer_guide/test_suites.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -sidebar_position: 5 -title: Test Suites ---- - -// TODO_DOCUMENT(@bryanchriswhite) \ No newline at end of file diff --git a/docusaurus/docs/develop/developer_guide/testing/app_integration.md b/docusaurus/docs/develop/developer_guide/testing/app_integration.md new file mode 100644 index 000000000..14a0bcb5a --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/app_integration.md @@ -0,0 +1,135 @@ +--- +sidebar_position: 3 +title: App Integration Tests +--- + +// TODO(@bryanchriswhite): Replace github source links with godocs links once available. + +## Table Of Contents + +- [Overview](#overview) +- [Using `integration.App`](#using-integrationapp) + - [Constructors](#constructors) + - [Customizing `integration.App` Configuration](#customizing-integrationapp-configuration) + - [Module Configuration](#module-configuration) + - [Setting Module Genesis State](#setting-module-genesis-state) + - [Message / Transaction / Block Processing](#message--transaction--block-processing) +- [Example Test](#example-test) + +## Overview + +[**App integration level**](testing_levels#app-integration-tests) tests leverage a custom construction of the poktroll appchain (for testing only). + +This construction integrates all the poktroll modules (and their cosmos-sdk dependencies) and exercises the appchain's message routing/handling and transaction processing logic. + +Tests in this level conventionally use the `testutil/integration` package's `App` structure and constructors to set up the appchain, execute messages, and make assertions against the resulting appchain state. + +:::info +See [App Integration Suites](integration_suites) for organizing larger or higher-level app integration tests. +::: + +## Using `integration.App` + +### Constructors + +To create a new instance of the `IntegrationApp` for your tests, use the `NewCompleteIntegrationApp` constructor, which handles the setup of all modules, multistore, base application, etc.: + +```go +// NewCompleteIntegrationApp creates a new instance of the App, abstracting out +// all the internal details and complexities of the application setup. +func NewCompleteIntegrationApp(t *testing.T, opts ...IntegrationAppOptionFn) *App + +// IntegrationAppOptionFn is a function that receives and has the opportunity to +// modify the IntegrationAppConfig. It is intended to be passed during integration +// App construction to modify the behavior of the integration App. +type IntegrationAppOptionFn func(*IntegrationAppConfig) +``` + +If more granular control over the application configuration is required, the more verbose `NewIntegrationApp` constructor exposes additional parameters: + +```go +// NewIntegrationApp creates a new instance of the App with the provided details +// on how the modules should be configured. +func NewIntegrationApp( + t *testing.T, + sdkCtx sdk.Context, + cdc codec.Codec, + txCfg client.TxConfig, + registry codectypes.InterfaceRegistry, + bApp *baseapp.BaseApp, + logger log.Logger, + authority sdk.AccAddress, + modules map[string]appmodule.AppModule, + keys map[string]*storetypes.KVStoreKey, + msgRouter *baseapp.MsgServiceRouter, + queryHelper *baseapp.QueryServiceTestHelper, + opts ...IntegrationAppOptionFn, +) *App { +``` + +#### Customizing `integration.App` Configuration + +If the existing [`IntegrationAppConfig`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/options.go#L13) is insufficient, it may be extended with additional fields, corresponding logic, and `IntegrationAppOptionFn`s to set them. + +### Module Configuration + +Integrated modules can be configured using `IntegrationAppOptionFn` typed option functions. + +Example use cases include: + +- Setting custom genesis states one or more modules (see [`integration.WithModuleGenesisState()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/options.go#L40)). +- Setting up a faucet account (see: [`newFaucetInitChainerFn()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/app.go#L985)). +- Collecting module info (see: [`newInitChainerCollectModuleNames()`](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/base.go#L157)). + +#### Setting Module Genesis State + +```go +supplierGenesisState := &suppliertypes.GenesisState{ + // ... +} +app := NewCompleteIntegrationApp(t, + WithModuleGenesisState[suppliermodule.AppModule](supplierGenesisState), +) +``` + +### Message / Transaction / Block Processing + +The `IntegrationApp` provides several methods to manage the lifecycle of transactions and blocks during tests: + +- `RunMsg`/`RunMsgs`: Processes one or more messages by: + - calling their respective handlers + - packaging them into a transaction + - finalizing the block + - committing the state + - advancing the block height + - returning the message responses +- `NextBlock`/`NextBlocks`: Only advances the blockchain state to subsequent blocks. + +## Example Test + +Here's a simple example of how to create a new integration app instance and run a message using the helper functions: + +```go +func TestAppIntegrationExample(t *testing.T) { + // Initialize a new complete integration app with default options. + app := NewCompleteIntegrationApp(t) + + // Example message to be processed + msg := banktypes.NewMsgSend(fromAddr, toAddr, sdk.NewCoins(sdk.NewInt64Coin("upokt", 100))) + + // Run the message in the integration app + res, err := app.RunMsg(t, msg) + require.NoError(t, err) + + // Check the result + require.NotNil(t, res, "Expected a valid response for the message") + + // Type assert the result to the message response type + sendRes, ok := res.(*banktypes.MsgSendResponse) + + require.True(t, ok) + require.NotNil(t, sendRes) +} +``` + +This example initializes the app, processes a bank message, and validates the result. diff --git a/docusaurus/docs/develop/developer_guide/testing/e2e.md b/docusaurus/docs/develop/developer_guide/testing/e2e.md new file mode 100644 index 000000000..eac93bae4 --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/e2e.md @@ -0,0 +1,6 @@ +--- +sidebar_position: 7 +title: End-to-End Tests +--- + +// TODO_DOCUMENT(@bryanchriswhite) diff --git a/docusaurus/docs/develop/developer_guide/testing/in_memory_integration.md b/docusaurus/docs/develop/developer_guide/testing/in_memory_integration.md new file mode 100644 index 000000000..889ae3ac8 --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/in_memory_integration.md @@ -0,0 +1,6 @@ +--- +sidebar_position: 6 +title: In-Memory Network Integration Tests +--- + +// TODO_DOCUMENT(@bryanchriswhite): Explain cosmos-sdk in-memory network and its appropriate usage. diff --git a/docusaurus/docs/develop/developer_guide/testing/integration_suites.md b/docusaurus/docs/develop/developer_guide/testing/integration_suites.md new file mode 100644 index 000000000..652857de4 --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/integration_suites.md @@ -0,0 +1,114 @@ +--- +sidebar_position: 5 +title: App Integration Suites +--- + +// TODO(@bryanchriswhite): Replace github source links with godocs links once available. + +## Table of Contents + +- [Overview](#overview) +- [When to Use Test Suites](#when-to-use-test-suites) +- [Using an Existing Integration Suite](#using-an-existing-integration-suite) + - [Example (`ParamsSuite`)](#example-paramssuite) +- [Implementing a Test Suite](#implementing-a-test-suite) + - [Test Suite Gotchas](#test-suite-gotchas) + +## Overview + +The [`suites` package](https://github.com/pokt-network/poktroll/tree/main/testutil/integration/suites) provides interfaces and base implementations for creating and managing **app integration test** suites. + +The foundational components are: + +- [**`IntegrationSuite`**](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/interface.go#L14): An interface defining common methods for interacting with an integration app. +- [**`BaseIntegrationSuite`**](https://github.com/pokt-network/poktroll/blob/main/testutil/integration/suites/base.go#L26): A base implementation of the `IntegrationSuite` interface that can be extended by embedding in other test suites. + +## When to Use Test Suites + +- **Complex Integration Tests**: Testing interactions between several modules; suites facilitate encapsulation and decomposition. +- **Complex Scenarios**: Simulating real-world scenarios that involve several transactions, state changes, and/or complex assertion logic. +- **Reusable Components**: To DRY (Don't Repeat Yourself) up common test helpers which can be embedded in other test suites (object oriented). + +## Using an Existing Integration Suite + +The `testutil/integration/suites` package contains multiple **app integration suites** which are intended to be embedded in [**app integration level**](testing_levels#app-integration-tests) test suites. + +### Example (`ParamsSuite`) + +The following example shows a test suite which embeds `suites.ParamsSuite`, in order to set on-chain module params as part of its `SetupTest()` method: + +```go +package suites + +import ( + "testing" + "github.com/stretchr/testify/require" + "github.com/stretchr/testify/suite" + cosmostypes "github.com/cosmos/cosmos-sdk/types" + banktypes "github.com/cosmos/cosmos-sdk/x/bank/types" +) + +type ExampleTestSuite struct { + suites.ParamsSuite +} + +// SetupTest is called before each test method in the suite. +func (s *ExampleTestSuite) SetupTest() { + // Initialize a new app instance for each test. + s.app = NewApp(s.T()) + + // Setup the authz accounts and grants for updating parameters. + s.SetupTestAuthzAccounts() + s.SetupTestAuthzGrants() + + // Set the module params using the ParamsSuite. + s.RunUpdateParam(s.T(), + sharedtypes.ModuleName, + string(sharedtypes.KeyNumBlocksPerSession), + 9001, + ) +} + +func (s *ExampleTestSuite) TestExample() { + // Query module params using the ParamsSuite. + sharedParams, err := s.QueryModuleParams(s.T(), sharedtypes.ModuleName) + require.NoError(s.T(), err) + + // Utilize other BaseIntegrationSuite methods to interact with the app... + + fundAmount := int64(1000) + fundAddr, err := cosmostypes.AccAddressFromBech32("cosmos1exampleaddress...") + require.NoError(s.T(), err) + + // Fund an address using the suite's FundAddress method. + s.FundAddress(s.T(), fundAddr, fundAmount) + + // Use the bank query client to verify the balance. + bankQueryClient := s.GetBankQueryClient() + balRes, err := bankQueryClient.Balance(s.SdkCtx(), &banktypes.QueryBalanceRequest{ + Address: fundAddr.String(), + Denom: "upokt", + }) + + // Validate the balance. + require.NoError(s.T(), err) + require.Equal(s.T(), fundAmount, balRes.GetBalance().Amount.Int64()) +} + +// Run the ExampleIntegrationSuite. +func TestExampleTestSuite(t *testing.T) { + suite.Run(t, new(ExampleTestSuite)) +} +``` + +## Implementing a Test Suite + +// TODO_DOCUMENT(@bryanchriswhite) + +### Test Suite Gotchas + +- **Setup**: You MAY need to call `SetupXXX()`: check embedded suites for any required setup and copy-paste +- **Accessing Test State**: Avoid using `s.T()` in methods of suites which are intended to be embedded in other suites; pass a `*testing.T` argument instead. +- **Inheritance**: Inheriting multiple suites is hard since only one can be embedded anonymously: others will have to accessed via a named field. + +// TODO_DOCUMENT(@bryanchriswhite): Add a `testutil/integration/suites.doc.go` with testable examples. diff --git a/docusaurus/docs/develop/developer_guide/testing/module_integration.md b/docusaurus/docs/develop/developer_guide/testing/module_integration.md new file mode 100644 index 000000000..800a98404 --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/module_integration.md @@ -0,0 +1,6 @@ +--- +sidebar_position: 2 +title: Module Integration Tests +--- + +// TODO_DOCUMENT(@bryanchriswhite): Explain `testkeeper.NewTokenomicsKeepers()` (to be renamed) and its appropriate usage. diff --git a/docusaurus/docs/develop/developer_guide/testing/testing_levels.md b/docusaurus/docs/develop/developer_guide/testing/testing_levels.md new file mode 100644 index 000000000..49ccee9d5 --- /dev/null +++ b/docusaurus/docs/develop/developer_guide/testing/testing_levels.md @@ -0,0 +1,152 @@ +--- +sidebar_position: 1 +title: Testing Levels +--- + +## Table Of Contents + +- [Unit Tests](#unit-tests) +- [Module Integration Tests](#module-integration-tests) + - [Unit Test Example](#unit-test-example) + - [Unit Test - Good Fit](#unit-test---good-fit) + - [Unit Test - Bad Fit](#unit-test---bad-fit) + - [Unit Test - Limitations](#unit-test---limitations) +- [App Integration Tests](#app-integration-tests) + - [Integration Test Example](#integration-test-example) + - [Integration Test - Good Fit](#integration-test-0-good-fit) + - [Integration Test - Bad Fit](#integration-test-0-bad-fit) + - [Integration Test - Limitations](#integration-test---limitations) +- [In-Memory Network Integration Tests](#in-memory-network-integration-tests) + - [In-Memory Network Example](#in-memory-network-example) + - [In-Memory Network - Good Fit](#in-memory-network---good-fit) + - [In-Memory Network - Bad Fit](#in-memory-network---bad-fit) + - [In-Memory Network - Limitations](#in-memory-network---limitations) +- [End-to-End Tests](#end-to-end-tests) + - [E2E Test Example](#e2e-test-example) + - [E2E Test - Good Fit](#e2e-test---good-fit) + - [E2E Test - Bad Fit](#e2e-test---bad-fit) + - [E2E Test - Limitations](#e2e-test---limitations) + +## Unit Tests + +**Unit tests** are the most granular level of testing, focusing on individual functions or methods within a module or module subcomponent. +These tests are used to verify that each unit of code behaves as expected in isolation. + +## [Module Integration Tests](module_integration.md) + +**Module integration tests** focus on testing the interaction between modules in the appchain without mocking individual components. +This level of testing ensures that cross-module interactions can be exercised but without the overhead of the full appchain. + +### Unit Test Example + +```go +// TODO_DOCUMENT(@bryanchriswhite): Add example +``` + +### Unit Test - Good Fit + +- Exercising a `Keeper` method +- Code has dependencies on other module `Keeper`s + +### Unit Test - Bad Fit + +- Test depends on network events +- Test depends on `Tx` assertions + +### Unit Test - Limitations + +- No transactions +- No events +- No message server + +## [App Integration Tests](app_integration) + +**App integration tests** focus on testing the behavior of the fully integrated _appchain_ from a common message server interface. +This level of testing ensures message handling logic is exercise while fully integrated with cosmos-sdk but without the overhead of the cometbft engine and networking. + +### Integration Test Example + +```go +// TODO_DOCUMENT(@bryanchriswhite): Add example +``` + +_NOTE: See [App Integration Suites](integration_suites) for organizing larger or higher-level app integration tests._ + +### Integration Test - Good Fit + +- Exercising a user story involving multiple messages +- Exercising a scenario involving multiple messages +- Exercising cross-module dependencies & interactions +- Asserting against a new integrated state + +### Integration Test - Bad Fit + +- Code under test depends/asserts on networking operations +- Code under test depends/asserts on consensus operations +- Code under test requires setup which would be simpler to do with direct keeper interaction. + +### Integration Test - Limitations + +- No networking +- No consensus +- No keeper API access (intentional) + +## [In-Memory Network Integration Tests](in_memory_integration) + +**In-memory network integration tests** focus on testing the behavior of a multi-validator network from the perspective of the ABCI message interface. +This level of testing ensures that the appchain behaves as expected in a multi-validator environment. + +### In-Memory Network Example + +```go +// TODO_DOCUMENT(@bryanchriswhite): Add example +``` + +### In-Memory Network - Good Fit + +- Exercising CometBFT RPC +- Exercising consensus scenarios +- Exercising multi-validator scenarios +- Integrating with external tools via network + +### In-Memory Network - Bad Fit + +- Most cases; use sparingly +- Prefer other levels unless it's clearly appropriate + +### In-Memory Network - Limitations + +- No parallelization +- No `Keeper` module access +- Depends on cosmos-sdk APIs (less customizable) +- Slow startup time (per network). + +## [End-to-End Tests](e2e) + +**End-to-end tests** focus on testing the behavior of a network containing both on- and off-chain actors; typically exercising "localnet". + +### E2E Test Example + +```go +// TODO_DOCUMENT(@bryanchriswhite): Add example +``` + +### E2E Test - Good Fit + +- Asserts or dependent on off-chain assertions +- Asserts or dependent on off-chain actors +- Asserts or dependent on off-chain behavior + +### E2E Test - Bad Fit + +- Scenarios which require many blocks/sessions to complete +- Scenarios which are not idempotent +- Scenarios which assume specific/complex network states + +### E2E Test - Limitations + +- Depends on LocalNet to be running and healthy +- Depends on other environments (DevNet/TestNet) to be running and healthy +- Shared mutable network state on-chain +- Shared mutable network state off-chain +- Intolerant of non-idempotent operations (CI re-runnability). diff --git a/docusaurus/yarn.lock b/docusaurus/yarn.lock index 93cee387e..5c2dcf935 100644 --- a/docusaurus/yarn.lock +++ b/docusaurus/yarn.lock @@ -1810,15 +1810,10 @@ dependencies: "@types/mdx" "^2.0.0" -"@node-rs/jieba-linux-x64-gnu@1.10.0": +"@node-rs/jieba-darwin-arm64@1.10.0": version "1.10.0" - resolved "https://registry.npmjs.org/@node-rs/jieba-linux-x64-gnu/-/jieba-linux-x64-gnu-1.10.0.tgz" - integrity sha512-rS5Shs8JITxJjFIjoIZ5a9O+GO21TJgKu03g2qwFE3QaN5ZOvXtz+/AqqyfT4GmmMhCujD83AGqfOGXDmItF9w== - -"@node-rs/jieba-linux-x64-musl@1.10.0": - version "1.10.0" - resolved "https://registry.npmjs.org/@node-rs/jieba-linux-x64-musl/-/jieba-linux-x64-musl-1.10.0.tgz" - integrity sha512-BvSiF2rR8Birh2oEVHcYwq0WGC1cegkEdddWsPrrSmpKmukJE2zyjcxaOOggq2apb8fIRsjyeeUh6X3R5AgjvA== + resolved "https://registry.npmjs.org/@node-rs/jieba-darwin-arm64/-/jieba-darwin-arm64-1.10.0.tgz" + integrity sha512-IhR5r+XxFcfhVsF93zQ3uCJy8ndotRntXzoW/JCyKqOahUo/ITQRT6vTKHKMyD9xNmjl222OZonBSo2+mlI2fQ== "@node-rs/jieba@^1.6.0": version "1.10.0" @@ -4619,6 +4614,11 @@ fs.realpath@^1.0.0: resolved "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz" integrity sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw== +fsevents@~2.3.2: + version "2.3.3" + resolved "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz" + integrity sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw== + function-bind@^1.1.2: version "1.1.2" resolved "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz"