-
-
Notifications
You must be signed in to change notification settings - Fork 358
Home
Mahmoud Hanafy edited this page Mar 3, 2020
·
7 revisions
Welcome to the spark-testing-base wiki!
Project Main Modules:
-
SharedSparkContext: Provides
SparkContextto be used in testing. - RDDComparisons: Compares two RDDs.
- DataFrameSuiteBase: Checks for dataframe equality.
- DatasetSuiteBase: Checks for dataset equality.
- StreamingSuiteBase: Applies given operation on given input stream and compare with expected output.
- StreamingActionBase: Tests actions that don't have output on input streams.
- RDDGenerator: Generates arbitrary RDDs.
- DataFrameGenerator: Generates arbitrary DataFrames.
- DatasetGenerator: Generates arbitrary Datasets.
- HDFSCluster: Simulate working with HDFS in tests.