-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose unpack_stream on strucutures #28
Comments
Exposing this would be possible but I'm debating if it is a good idea. The current, canonical way to do this is to use
If you have a stream where you know all bytes required will be available (or which is blocking and you expect failures to be handled acceptably well by suitcase) you can use Structure._packer.unpack_stream(stream) This violates encapsulation, so if there is a use case, I think I would be inclined to expose |
I'm fine with 'Structure._packer.unpack_stream(stream)' exposing. The reason is that I have the following similar state machine in a open review/code, and suitcase (with an exposed https://review.openstack.org/#/c/244376/20/taskflow/utils/executor_utils.py (see
|
The other example that might be useful in that |
Btw, if u are going to recommend just using feed, there needs to be a way to know how many more bytes are needed to feed something. That is needed to say read from a socket, and know how many bytes to read for feeding (to make the next object get dispatched)... That's also why in that code above it has:
|
Thanks for the context @harlowja. That always helps. I'll probably move forward with exposing
Unfortunately, this is not possible in the general case for all Structures. There is a The idea with feed is that you would do non-blocking reads (as would be the case with asyncore) of whatever max chunk size makes sense for your application and feed all of those bytes into the parser and let it handle the rest. If you need to do synchronous IO, you can feed one byte at a time. The OS will probably buffer for you anyway so it's not all that bad (though there would definitely be some extra copies within suitcase at the moment [although for small string buffers in CPython, maybe not]). |
`feed one byte at a time' :( |
For unpacking a strucuture from a stream its very much appreciated to expose an equivalent method to do so on structure objects.
Currently all that exists is:
But to unpack from a stream something like the following is needed;
The text was updated successfully, but these errors were encountered: