Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Self-tests for Custom Tests #162

Open
Luc45 opened this issue Apr 29, 2024 · 0 comments
Open

Self-tests for Custom Tests #162

Luc45 opened this issue Apr 29, 2024 · 0 comments
Labels
type: documentation This issue is a request for better documentation.

Comments

@Luc45
Copy link
Member

Luc45 commented Apr 29, 2024

This issue is to keep track of what the self-tests of the Custom Tests covers, and to keep track of what scenarios we still need to cover.

Current Test coverage

Environment

Behavior Covered by Tests
Env up with default settings ✔️
Env up with PHP and WP versions passed as parameters ✔️
Env up with PHP and WP versions in config file ✔️
Env up with PHP and WP versions in config file overridden by parameters ✔️
Env up with PHP and WP versions in config file overridden by config file override
Env up with Object Cache ✔️
Env up with Additional Volumes ✔️
Env up with WordPress stable version ✔️
Env up with WordPress rc version
Env up with WordPress WPOrg tag version ✔️
Env up with WooCommerce stable version ✔️
Env up with WooCommerce rc version ✔️
Env up with WooCommerce git tag version
Env up with additional PHP Extensions ✔️
Env up with additional theme(s) ✔️

Tagging

Behavior Covered by Tests
Upload test tag ✔️
List test tags ✔️
List test tags of a specific extension ✔️
Delete test tag ✔️
Upload, delete and list test tags of extension you don't own

E2E Test

Behavior Covered by Tests
Scaffold a basic E2E test ✔️
Run a scaffolded E2E test locally ✔️
Tag a scaffolded test and run it from the tag ✔️
Run a test and assert it fails if SUT failed to activate ✔️
Run an E2E test with multiple tags ✔️
Run an E2E test with theme as SUT ✔️
Run an E2E test with plugin as SUT and theme with test or bootstrap actions
Run an E2E test with snapshot test for visual regression ✔️

Compatibility Test

Behavior Covered by Tests
Run an E2E test and activate additional plugin sourced from slug ✔️
Run an E2E test with bootstrap additional plugin sourced from slug ✔️
Run an E2E test with testing additional plugin sourced from slug ✔️
Run an E2E test with multiple tags and additional plugin ✔️
Run an E2E test with same additional plugin having multiple tags ✔️
Run a scaffolded E2E test with additional plugin sourced from local directory/zip file/URL (activate/bootstrap/test actions), both from runtime parameters, config files and config file override
Run a test with multiple plugins, some of them with tests, some with local directory and others from tags
@Luc45 Luc45 mentioned this issue Apr 29, 2024
@zhongruige zhongruige added the type: documentation This issue is a request for better documentation. label Aug 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: documentation This issue is a request for better documentation.
Projects
None yet
Development

No branches or pull requests

2 participants