Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix attribute array comparison #6181

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

ukmo-ccbunney
Copy link
Contributor

@ukmo-ccbunney ukmo-ccbunney commented Oct 18, 2024

🚀 Pull Request

Description

Fixes: #6027

Logic for comparing arrays in attributes was relying on old numpy behaviour where an array compared to another any object would return a scalar True or False. Current numpy behaviour is to do a element-wise comparison which throws an error if the arrays are of different size.

Fix is to use the numpy.array_equal.


Consult Iris pull request check list

@ukmo-ccbunney ukmo-ccbunney self-assigned this Oct 18, 2024
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.81%. Comparing base (3b8b33c) to head (06c997e).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6181      +/-   ##
==========================================
- Coverage   89.82%   89.81%   -0.01%     
==========================================
  Files          88       88              
  Lines       23185    23182       -3     
  Branches     4314     4314              
==========================================
- Hits        20825    20822       -3     
  Misses       1628     1628              
  Partials      732      732              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ukmo-ccbunney
Copy link
Contributor Author

ukmo-ccbunney commented Oct 18, 2024

Note, this change fixes what I believe is a bug here:

iris/lib/iris/_merge.py

Lines 388 to 391 in d3071ff

diff_attrs = [
repr(key[1])
for key in attrs_1
if np.all(attrs_1[key] != attrs_2[key])

Line 391 should be checking for any differences in the array elements, not all.

It also cleans up a now unnecessary try...except block that I believe was only needed for older versions of numpy.

try:
match = bool(match)
except ValueError:
match = match.all()

I have extended the existing unit test to check for correct handling inequality.
However, I question the validity of this test

values = dict(one=np.arange(1), two=np.arange(1), three=np.arange(1))
left = LimitedAttributeDict(dict(one=0, two=0, three=0))
right = LimitedAttributeDict(**values)
self.assertEqual(left, right)
self.assertEqual(left, values)

which now fails as it is testing a scalar value against a single element array. With the old logic, this would work, but it is flawed as np.all( np.array([0]) == 0 ) and np.all( np.zeros(100) == 0 ) would both return True, even though the latter is not what is intended.

I have commented out this test at the moment and would appreciate the opinion of @pp-mo.

…bunney/iris into bugfix/cube_attr_compare

* 'bugfix/cube_attr_compare' of https://github.com/ukmo-ccbunney/iris:
  [pre-commit.ci] auto fixes from pre-commit.com hooks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Cannot load cubes that contain multiple variables with array-type attributes with different sizes
1 participant