Skip to content

Commit

Permalink
test: Isolate integration, external tests (#1156)
Browse files Browse the repository at this point in the history
Signed-off-by: Dave Skender <[email protected]>
  • Loading branch information
DaveSkender authored Nov 3, 2024
1 parent 0605107 commit 070a00e
Show file tree
Hide file tree
Showing 46 changed files with 625 additions and 131 deletions.
36 changes: 6 additions & 30 deletions .github/workflows/test-indicators.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Indicators

on:
push:
branches: ["main"]
branches: ["main","v3"]

pull_request:
types: [opened, synchronize, reopened]
Expand All @@ -27,29 +27,17 @@ jobs:
# identifying primary configuration so only one reports coverage
IS_PRIMARY: ${{ matrix.os == 'ubuntu-latest' && matrix.dotnet-version == '8.x' }}

# .NET SDK versions in the matrix that support `ga` quality spec
# versions before 5.x do not support it
SUPPORT_GA: ${{ contains(fromJson('["6.x", "8.x"]'), matrix.dotnet-version) }}

steps:

- name: Checkout source
uses: actions/checkout@v4

- name: Setup .NET
id: dotnet-new
uses: actions/setup-dotnet@v4
if: env.SUPPORT_GA == 'true'
with:
dotnet-version: ${{ matrix.dotnet-version }}
dotnet-quality: "ga"

- name: Setup .NET (older)
uses: actions/setup-dotnet@v4
if: env.SUPPORT_GA == 'false'
with:
dotnet-version: ${{ matrix.dotnet-version }}

- name: Build library
run: >
dotnet build
Expand All @@ -58,41 +46,29 @@ jobs:
-warnAsError
- name: Test indicators
env:
ALPACA_KEY: ${{ secrets.ALPACA_KEY }}
ALPACA_SECRET: ${{ secrets.ALPACA_SECRET }}
run: >
dotnet test tests/indicators/Tests.Indicators.csproj
dotnet test
--configuration Release
--settings tests/tests.unit.runsettings
--results-directory ./test-results
--no-build
--verbosity normal
--logger trx
--collect:"XPlat Code Coverage"
--results-directory ./test-indicators
# the remaining steps are only needed from one primary instance

- name: Test other items
if: env.IS_PRIMARY == 'true'
run: >
dotnet test tests/other/Tests.Other.csproj
--configuration Release
--no-build
--verbosity normal
--logger trx
--results-directory ./test-other
- name: Post test summary
uses: dorny/test-reporter@v1
if: env.IS_PRIMARY == 'true' && always()
with:
name: Test results
path: ./test-indicators/**/*.trx
path: ./test-results/**/*.trx
reporter: dotnet-trx

- name: Publish coverage to Codacy
uses: codacy/codacy-coverage-reporter-action@v1
if: env.IS_PRIMARY == 'true'
with:
project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
coverage-reports: ./test-indicators/**/coverage.cobertura.xml
coverage-reports: ./test-results/**/coverage.cobertura.xml
58 changes: 58 additions & 0 deletions .github/workflows/test-integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Indicators

on:
push:
branches: ["main","v3"]

pull_request:
types: [opened, synchronize, reopened]

jobs:
test:
name: integration tests
runs-on: ubuntu-latest

permissions:
contents: read
actions: read
checks: write

steps:

- name: Checkout source
uses: actions/checkout@v4

- name: Setup .NET
id: dotnet-new
uses: actions/setup-dotnet@v4
with:
dotnet-version: "8.x"
dotnet-quality: "ga"

- name: Build library
run: >
dotnet build
--configuration Release
--property:ContinuousIntegrationBuild=true
-warnAsError
- name: Test integrations
env:
ALPACA_KEY: ${{ secrets.ALPACA_KEY }}
ALPACA_SECRET: ${{ secrets.ALPACA_SECRET }}
run: >
dotnet test
--configuration Release
--settings tests/tests.integration.runsettings
--results-directory ./test-results
--no-build
--verbosity normal
--logger trx
- name: Post test summary
uses: dorny/test-reporter@v1
if: always()
with:
name: Test results
path: ./test-results/**/*.trx
reporter: dotnet-trx
36 changes: 20 additions & 16 deletions Stock.Indicators.sln
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,24 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Configuration", "Configurat
.gitattributes = .gitattributes
.gitignore = .gitignore
src\gitversion.yml = src\gitversion.yml
tests\tests.integration.runsettings = tests\tests.integration.runsettings
tests\tests.unit.runsettings = tests\tests.unit.runsettings
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Indicators", "src\Indicators.csproj", "{8D0F1781-EDA3-4C51-B05D-D33FF1156E49}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Tests.Indicators", "tests\indicators\Tests.Indicators.csproj", "{11CD6C7E-871F-4903-AEAD-58E034C6521D}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Tests.Other", "tests\other\Tests.Other.csproj", "{97905D26-4854-41FF-A4F7-CE042B2ACD02}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Tests.Performance", "tests\performance\Tests.Performance.csproj", "{3BD4837B-D197-41FD-A286-A3256D0770E1}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test.Application", "tests\application\Test.Application.csproj", "{14DEC3AF-9AF2-4A66-8BEE-C342C6CC4307}"
ProjectSection(ProjectDependencies) = postProject
{11CD6C7E-871F-4903-AEAD-58E034C6521D} = {11CD6C7E-871F-4903-AEAD-58E034C6521D}
{8D0F1781-EDA3-4C51-B05D-D33FF1156E49} = {8D0F1781-EDA3-4C51-B05D-D33FF1156E49}
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test.Simulation", "tests\simulate\Test.Simulation.csproj", "{9C9045D2-9928-41F8-97FC-ECCBDD3B9868}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Test.Application", "tests\external\application\Test.Application.csproj", "{F98B09DA-0E0B-41D1-B7B6-3A40EE6C1DBE}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Tests.Integration", "tests\external\integration\Tests.Integration.csproj", "{88C59340-F6C6-497B-A7F3-08ACCC8873EE}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Tests.PublicApi", "tests\external\public-api\Tests.PublicApi.csproj", "{D6CF664F-8232-4EE9-B044-CA34A0BA522E}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand All @@ -41,22 +41,26 @@ Global
{11CD6C7E-871F-4903-AEAD-58E034C6521D}.Debug|Any CPU.Build.0 = Debug|Any CPU
{11CD6C7E-871F-4903-AEAD-58E034C6521D}.Release|Any CPU.ActiveCfg = Release|Any CPU
{11CD6C7E-871F-4903-AEAD-58E034C6521D}.Release|Any CPU.Build.0 = Release|Any CPU
{97905D26-4854-41FF-A4F7-CE042B2ACD02}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{97905D26-4854-41FF-A4F7-CE042B2ACD02}.Debug|Any CPU.Build.0 = Debug|Any CPU
{97905D26-4854-41FF-A4F7-CE042B2ACD02}.Release|Any CPU.ActiveCfg = Release|Any CPU
{97905D26-4854-41FF-A4F7-CE042B2ACD02}.Release|Any CPU.Build.0 = Release|Any CPU
{3BD4837B-D197-41FD-A286-A3256D0770E1}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{3BD4837B-D197-41FD-A286-A3256D0770E1}.Debug|Any CPU.Build.0 = Debug|Any CPU
{3BD4837B-D197-41FD-A286-A3256D0770E1}.Release|Any CPU.ActiveCfg = Release|Any CPU
{3BD4837B-D197-41FD-A286-A3256D0770E1}.Release|Any CPU.Build.0 = Release|Any CPU
{14DEC3AF-9AF2-4A66-8BEE-C342C6CC4307}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{14DEC3AF-9AF2-4A66-8BEE-C342C6CC4307}.Debug|Any CPU.Build.0 = Debug|Any CPU
{14DEC3AF-9AF2-4A66-8BEE-C342C6CC4307}.Release|Any CPU.ActiveCfg = Release|Any CPU
{14DEC3AF-9AF2-4A66-8BEE-C342C6CC4307}.Release|Any CPU.Build.0 = Release|Any CPU
{9C9045D2-9928-41F8-97FC-ECCBDD3B9868}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{9C9045D2-9928-41F8-97FC-ECCBDD3B9868}.Debug|Any CPU.Build.0 = Debug|Any CPU
{9C9045D2-9928-41F8-97FC-ECCBDD3B9868}.Release|Any CPU.ActiveCfg = Release|Any CPU
{9C9045D2-9928-41F8-97FC-ECCBDD3B9868}.Release|Any CPU.Build.0 = Release|Any CPU
{F98B09DA-0E0B-41D1-B7B6-3A40EE6C1DBE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{F98B09DA-0E0B-41D1-B7B6-3A40EE6C1DBE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{F98B09DA-0E0B-41D1-B7B6-3A40EE6C1DBE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{F98B09DA-0E0B-41D1-B7B6-3A40EE6C1DBE}.Release|Any CPU.Build.0 = Release|Any CPU
{88C59340-F6C6-497B-A7F3-08ACCC8873EE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{88C59340-F6C6-497B-A7F3-08ACCC8873EE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{88C59340-F6C6-497B-A7F3-08ACCC8873EE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{88C59340-F6C6-497B-A7F3-08ACCC8873EE}.Release|Any CPU.Build.0 = Release|Any CPU
{D6CF664F-8232-4EE9-B044-CA34A0BA522E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{D6CF664F-8232-4EE9-B044-CA34A0BA522E}.Debug|Any CPU.Build.0 = Debug|Any CPU
{D6CF664F-8232-4EE9-B044-CA34A0BA522E}.Release|Any CPU.ActiveCfg = Release|Any CPU
{D6CF664F-8232-4EE9-B044-CA34A0BA522E}.Release|Any CPU.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Running the `Tests.Performance` console application in `Release` mode will produ
dotnet run -c Release

# run individual performance benchmark
dotnet run -c Release --filter *.GetAdx
dotnet run -c Release --filter *.ToAdx
```

## Documentation
Expand Down
4 changes: 2 additions & 2 deletions src/_common/Quotes/Quote.Converters.cs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ public static partial class Quotes
{
/* LISTS */

// convert TQuote type list to built-in Quote type list
// convert TQuote type list to built-in Quote type list (public API only)
public static IReadOnlyList<Quote> ToQuoteList<TQuote>(
this IReadOnlyList<TQuote> quotes)
where TQuote : IQuote
Expand All @@ -27,7 +27,7 @@ internal static List<QuoteD> ToQuoteDList<TQuote>(

/* TYPES */

// convert any IQuote type to native Quote type
// convert any IQuote type to native Quote type (public API only)
public static Quote ToQuote<TQuote>(this TQuote quote)
where TQuote : IQuote

Expand Down
13 changes: 8 additions & 5 deletions src/s-z/Sma/Sma.Utilities.cs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
using System.Diagnostics.CodeAnalysis;
using System.Numerics;

namespace Skender.Stock.Indicators;
Expand All @@ -22,19 +23,20 @@ public static partial class Sma
/// if incalculable <see langword="double.NaN"/>
/// values are in range.
/// </returns>
internal static double? Average<T>(
public static double? Average<T>( // public API only
this IReadOnlyList<T> values,
int lookbackPeriods,
int? endIndex = null)
where T : IReusable
{
ArgumentNullException.ThrowIfNull(values);

// TODO: unused SMA utility, either make public or remove

=> Increment(
return Increment(
values,
lookbackPeriods,
endIndex ?? values.Count - 1)
.NaN2Null();
}

/// <summary>
/// Simple moving average calculation
Expand Down Expand Up @@ -69,9 +71,10 @@ internal static double Increment<T>(
// TODO: apply this SMA increment method more widely in other indicators (see EMA example)
}

[ExcludeFromCodeCoverage] // experimental SIMD code
internal static double[] Increment(this double[] prices, int period)
{
// TODO: is this used (probably just an experiment, has rounding errors)
// TODO: remove/consider experiment, has rounding errors

int count = prices.Length - period + 1;
double[] sma = new double[count];
Expand Down
80 changes: 80 additions & 0 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Testing

Tests are split into different projects for isolation of purpose.

```bash
# runs all unit
# and integration tests
dotnet test
```

> When developing locally, we recommend that you normally _unload_ the external test projects shown below, except when testing externalities.
## Unit tests

> `indicators/Tests.Indicators.csproj` unit tests library
Our primary full unit test project covers the entire utility of the library. In most IDE, you can [manually select](https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2022#manually-select-the-run-settings-file) the `tests/tests.unit.runsettings` for isolation for local IDE dev/test efficiency, or use the _unload_ approach described above.

```bash
# CLI equivalent
dotnet test --settings tests/tests.unit.runsettings
```

## Performance tests

> `tests/performance/Tests.Performance.csproj` benchmark tests
Running the `Tests.Performance` console application in `Release` mode will produce [benchmark performance data](https://dotnet.stockindicators.dev/performance/) that we include on our documentation site.

```bash
# run all performance benchmarks (~15-20 minutes)
dotnet run -c Release

# run individual performance benchmark
dotnet run -c Release --filter *.ToAdx

# run cohorts of performance benchmarks
dotnet run -c Release --filter **
```

```bash
# to see all cohorts
dotnet run --list
...
# Available Benchmarks:
#0 Incrementals
#1 SeriesIndicators
#2 StreamExternal
#3 StreamIndicators
#4 Utility
#5 UtilityMaths

# to see all tests
dotnet run --list flat
```

## External tests

All external integration and API tests can be run with one CLI

```bash
# CLI equivalent
dotnet test --settings tests/tests.integration.runsettings
```

Since we assume tests are non-integration tests by default, set the category attribute on any new test classes that contain integration tests. This can be applied uniquely to `[TestMethod]` as well.

```csharp
[TestClass, TestCategory("Integration")]
public class MyIntegrationTests : TestBase
...
```

### Public API tests

> `external/public-api/Tests.PublicApi.csproj` E2E libary external tests

### Integration tests

> `external/integration/Tests.Integration.csproj` connected to Live 3rd-party API
19 changes: 0 additions & 19 deletions tests/application/Test.Application.csproj

This file was deleted.

File renamed without changes.
Loading

0 comments on commit 070a00e

Please sign in to comment.