Connected tabular generator with dataset_generator #324
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… test between dataset_generator and get_ordered_column
drahc1R
changed the title
taylorfturner
suggested changes
Aug 1, 2023
auto-merge was automatically disabled
August 7, 2023 13:44
Head branch was pushed to by a user without write access
… refactored uncorrelated_synthesize function, and implemented parameters test
taylorfturner
suggested changes
Aug 8, 2023
synthetic_data/generators.py
Outdated
| # edge cases for extracting data from profiler report. | ||
| if generator_name == "datetime": | ||
| col_["format"] = col_["statistics"].get("format", None) | ||
| print(col_["format"], "SHOULD NOT BE EMPTY") |
There was a problem hiding this comment.
Suggested change
| print(col_["format"], "SHOULD NOT BE EMPTY") |
| @@ -39,6 +39,10 @@ def random_text( | |||
| ) | |||
| text_list = [] | |||
|
|
|||
| # edge case | |||
There was a problem hiding this comment.
this file still has a ton of commented out code... needs clean up
There was a problem hiding this comment.
got it -- still WIP on this file
taylorfturner
suggested changes
Aug 8, 2023
…elated_synthesize private
auto-merge was automatically disabled
August 8, 2023 14:40
Head branch was pushed to by a user without write access
taylorfturner
suggested changes
Aug 8, 2023
…e uncorrelated_synthesize private" This reverts commit caabdbf.
…puts, and params_build
taylorfturner
suggested changes
Aug 9, 2023
tests/test_generators.py
Outdated
Comment on lines
136
to
140
| # @staticmethod | ||
| # def setup_tabular_generator_mock(mock_generator): | ||
| # mock_DataLabeler = mock_generator.return_value | ||
|
|
||
| # TEST PARAM_BUILD |
tests/test_generators.py
Outdated
Comment on lines
397
to
446
| # def test_generate_uncorrelated_with_invalid_generator(self): | ||
| # columns_to_gen = [{"generator": "non existent generator"}] | ||
| # with self.assertRaisesRegex( | ||
| # ValueError, "generator: non existent generator is not a valid generator." | ||
| # ): | ||
| # TabularGenerator.generate_dataset( | ||
| # self.rng, | ||
| # columns_to_generate=columns_to_gen, | ||
| # dataset_length=self.dataset_length, | ||
| # ) | ||
|
|
||
|
|
||
| # @mock.patch("synthetic_data.dataset_generator.logging.warning") | ||
| # def test_generate_dataset_with_invalid_sorting_type(self, mock_warning): | ||
| # columns_to_gen = [ | ||
| # { | ||
| # "generator": "integer", | ||
| # "name": "int", | ||
| # "min_value": 4, | ||
| # "max_value": 88, | ||
| # "order": "random", | ||
| # } | ||
| # ] | ||
| # unsupported_sort_types = ["cheese", "random"] | ||
|
|
||
| # for type in unsupported_sort_types: | ||
| # columns_to_gen[0]["order"] = type | ||
| # dataset_generator.generate_dataset( | ||
| # self.rng, | ||
| # columns_to_generate=columns_to_gen, | ||
| # dataset_length=self.dataset_length, | ||
| # ) | ||
| # mock_warning.assert_called_with( | ||
| # f"""{columns_to_gen[0]["name"]} is passed with sorting type of {columns_to_gen[0]["order"]}. | ||
| # Ascending and descending are the only supported options. | ||
| # No sorting action will be taken.""" | ||
| # ) | ||
| # self.assertEqual(mock_warning.call_count, 2) | ||
|
|
||
| # @mock.patch("synthetic_data.dataset_generator.logging.warning") | ||
| # def test_generate_dataset_with_valid_sorting_type(self, mock_warning): | ||
| # columns_to_gen = [ | ||
| # { | ||
| # "generator": "integer", | ||
| # "name": "int", | ||
| # "min_value": 4, | ||
| # "max_value": 88, | ||
| # "order": "ascending", | ||
| # } | ||
| # ] |
auto-merge was automatically disabled
August 9, 2023 19:02
Head branch was pushed to by a user without write access
taylorfturner
approved these changes
Aug 9, 2023
ksneab7
approved these changes
Aug 9, 2023
taylorfturner
merged commit 0cc79fa
into
capitalone:feature/simple-tabular-generator
4 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Connected tabular generator with dataset_generator