Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nf floats #1011

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Nf floats #1011

wants to merge 2 commits into from

Conversation

Vamp1899
Copy link

This issue is the same issue as mentioned by Lukas but there's a change in code structure by keeping privacy of variables in consideration with regards to more upcoming development .

Pointers -
Handles NaN values in Json as
Python native type - which generates error
Float Null type (Nan , infinity) - Passed as a string in final JSON
Null type - Which modifies values as null in JSON more like allow_nan = True in json python package

Removed float instance check from encode_it function
Comment on lines +7 to +10
# Comment this function to see Warnings in console
def warn(*args, **kwargs):
pass
warnings.warn = warn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A feature to disable warnings is desirable, but it should not be enabled/disabled by commenting code, also the default should be having the warnings enabled.

Comment on lines +192 to +207
class JsonNonFiniteEncoding(Enum):
# Use the default python behaviour, which violates the official JSON standard, basically allow_nan = False
__default = 0
# Encode non-finite numbers as null values, allow_nan = True
__num_null = 1
# Encode non-finite floats as null values, allow_nan = True
__float_null = 2

def fetch_python(self):
return self.__default

def fetch_null_values(self):
return self.__num_null

def fetch_float_values(self):
return self.__float_null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe Lukas' version is a bit more clear in this part. Can you help me understand why we would want private fields instead of Lukas' version?

class PandasSettings(Settings):
pass

class SparkSettings(Settings):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this would make more sense in the spark-branch

Comment on lines +4 to +8
df = pd.DataFrame([1, 1, np.nan], columns=["a"])

profile = ProfileReport(df, title="Pandas Profiling Report", minimal=True)

print(profile.to_json())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should validate the expected behavior for each encoding

Copy link
Contributor

@alexbarros alexbarros left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, but we need to fix some of the issues with the warnings and the tests, and improve the enum.

@fabclmnt
Copy link
Contributor

@Vamp1899 can you please update your commit message as well with the correct and expected format?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants