Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added more links and Why Docker? #20

Merged
merged 1 commit into from
Aug 23, 2024
Merged

added more links and Why Docker? #20

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@

*<p style="text-align: center;">DataOps Data Quality TestGen, or "TestGen" for short, can help you find data issues so you can alert your users and notify your suppliers. It does this by delivering simple, fast data quality test generation and execution by data profiling, new dataset screening and hygiene review, algorithmic generation of data quality validation tests, ongoing production testing of new data refreshes, and continuous anomaly monitoring of datasets. TestGen is part of DataKitchen's Open Source Data Observability.</p>*

## Documentation

[DataOps TestGen Overview](https://datakitchen.io/dataops-testgen-product/)

[DataOps TestGen Documentation](https://docs.datakitchen.io/articles/dataops-testgen-help/dataops-testgen-help)

[DataOps TestGen Case Study](https://datakitchen.io/datakitchens-data-quality-testgen-found-18-quality-issues-in-a-few-minutes-including-install-time-on-data-boston-gov-building-permit-data/)

## Features

What does DataKitchen's DataOps Data Quality TestGen do? It helps you understand and <b>find data issues in new data</b>.
Expand All @@ -28,8 +36,8 @@ The [dk-installer](https://github.com/DataKitchen/data-observability-installer/?

| Software | Tested Versions | Command to check version |
|-------------------------|-------------------------|-------------------------------|
| [Python](https://www.python.org/downloads/) <br/>- Most Linux and macOS systems have Python pre-installed. <br/>- On Windows machines, you will need to download and install it. | 3.9, 3.10, 3.11, 3.12 | `python3 --version` |
| [Docker](https://docs.docker.com/get-docker/) <br/>[Docker Compose](https://docs.docker.com/compose/install/) | 25.0.3, 26.1.1, <br/> 2.24.6, 2.27.0, 2.28.1 | `docker -v` <br/> `docker compose version` |
| [Python](https://www.python.org/downloads/) <br/>- Most Linux and macOS systems have Python pre-installed. <br/>- On Windows machines, you will need to download and install it. <br/> Why Python? To run the installer. | 3.9, 3.10, 3.11, 3.12 | `python3 --version` |
| [Docker](https://docs.docker.com/get-docker/) <br/>[Docker Compose](https://docs.docker.com/compose/install/) <br/> Why Docker? Docker lets you try TestGen without affecting your local software environment. All the dependencies TestGen needs are isolated in its own container, so installation is easy and insulated. | 25.0.3, 26.1.1, <br/> 2.24.6, 2.27.0, 2.28.1 | `docker -v` <br/> `docker compose version` |

### Download the installer

Expand Down Expand Up @@ -67,10 +75,6 @@ python3 dk-installer.py tg run-demo

In the TestGen UI, you will see that new data profiling and test results have been generated.

## Product Documentation

[DataOps Data Quality TestGen](https://docs.datakitchen.io/articles/dataops-testgen-help/dataops-testgen-help)

## Useful Commands

The [dk-installer](https://github.com/DataKitchen/data-observability-installer/?tab=readme-ov-file#install-the-testgen-application) and [docker compose CLI](https://docs.docker.com/compose/reference/) can be used to operate the installed TestGen application. All commands must be run in the same folder that contains the `dk-installer.py` and `docker-compose.yml` files used by the installation.
Expand Down