-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Accept an interface instead of []byte in producer's Put method #11
Comments
I think the general idea is good and useful. I’ll give it more attention tomorrow morning. |
Has there been any progress on this front? It would be a great feature addition. |
I didn't have any progress on this, but I'm in favor of adding it, since it's actually needed for other uses cases as well, like #17. |
this would be quite useful, will look at submitting a PR. quick question: do we think this should be added using the existing |
Put expects records to be written to Kinesis as []byte. This works fine for simple cases but often a more complex layer wraps kinesis-producer and needs more control/flexibility over how different cases (such as failures) are handled. For example, a program might want to handle failures in a special way but right now the FailureResult struct only contains the partition key and the raw data as bytes. There is no other identifying information that might help identify the records.
Accepting an interface instead of raw bytes would make the library a lot more flexible. For example,
Or a record could marshal itself as bytes that kinesis-producer could use internally by calling the Read() method and then wouldn't have to unmarshal the bytes on every failure in order to be able to inspect the record (for specialized logging, retries, failure handling etc).
kinesis-producer API would add an interface definition
Or partition key could be rolled into the record as well like:
In this case, failure channel would just return failed Records and client side implementation could look like:
The library could also ship with a simple implementation to cover simple cases. Usage would look like:
The text was updated successfully, but these errors were encountered: