Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-stripe-native: make created optional for Accounts #1978

Merged

Conversation

Alex-Bair
Copy link
Contributor

@Alex-Bair Alex-Bair commented Sep 24, 2024

Description:

Sometimes, account.updated events do not have a created field in the document, causing failures when we validate the document against its model. Making created optional will avoid these failures and reflect the API's actual response. The default value of None is removed from the schema to keep it aligned with how we understand the created field: it's either present as an int or it's not present at all. In other words, it's an optional, non-nullable field.

Note: I've only noted that this occurs for account.updated events for when the user's own Stripe account is created. I couldn't confirm if all account.updated events have created missing in the document, or if Accounts documents really don't have created fields at all. If this becomes an issue later, we will need to re-evaluate how we backfill Accounts since we rely on a created field being present.

Discover snapshot change is expected due to switching created from a required field to an optional field for Accounts.

Workflow steps:

(How does one use this feature, and how has it changed)

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

Tested on a local stack. Confirmed Accounts does not cause shard failures when account.updated events do not have a created field in the document.

Existing tasks will need to be re-discovered since we're changing created from required to optional for Accounts.


This change is Reviewable

@Alex-Bair Alex-Bair added the change:unplanned Unplanned change, useful for things like doc updates label Sep 24, 2024
@Alex-Bair Alex-Bair force-pushed the bair/source-stripe-native-account-missing-created-field branch from 7b59ef4 to 122b031 Compare September 24, 2024 17:07
@@ -151,6 +151,8 @@ class Accounts(BaseStripeObjectWithEvents):
"account.updated": "u",
}

# Accounts docs returned in account.updated events may not have a created field.
created: int | None = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to get this to schematize as "not required to be present, but if it is present, it must be int"? I think that's what we want: The field is either present as an int, or not present at all. It shouldn't ever be present as an actual null value.

The schema coming from this is saying it can be a null, and also giving it a default null value (we don't want that default, for sure). Ideally the schema would be this:

created: {
  "type": "integer",
  "title": "created"
}

...and it would not be present in the list of required fields.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've looked through a handful of Pydantic discussions, and I haven't found any straightforward way to have an optional, non-nullable field. I've looked in Pydantic's docs, and it looks like optional fields always have to have a default value.

To avoid saying created can be null in the schema, we could have a default value like -1, then overwrite created with the event's timestamp whenever we see that it's -1. That's not 100% ideal, but I'm not seeing a better way to handle this right now. What do you think about that approach?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have used this hack for not schematizing the default value before - does that help here?

I think we need a way to represent this correctly. If it the field is either present and is some integer, or is not present at all, we need to have the schema say that. A default of null is not ideal, but a default of -1 is even worse since that is not what the value of the field is, and since it's a unix timestamp might be problematic in other ways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That code you linked did help - thank you! I'll get the PR updated.

Sometimes, `account.updated` events do not have `created` in the
document, causing failures when we validate the document against its
model. Allowing a default value of `None` optional will avoid these
failures and reflect the API's actual response.

Note: We remove the `None` default value with a lambda to keep the
schema more inline with what actually happens: the `created` field is
either present as an `int` or it's not present at all.
@Alex-Bair Alex-Bair force-pushed the bair/source-stripe-native-account-missing-created-field branch from 122b031 to 34d27ef Compare September 24, 2024 19:58
Copy link
Member

@williamhbaker williamhbaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Alex-Bair Alex-Bair merged commit 47fc346 into main Sep 24, 2024
65 of 76 checks passed
@Alex-Bair Alex-Bair deleted the bair/source-stripe-native-account-missing-created-field branch September 24, 2024 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
change:unplanned Unplanned change, useful for things like doc updates
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants