Issue
I am using the aioisotp library and sometimes messages are received close together. This means that aioisotp can, on occasion, provide 2 messages combined together. For example,
03 7F 36 78 00 00 00 00
02 76 02 00 00 00 00 00
I receive these 2 messages <2ms apart. aioisotp
returns it as one combined message: 7f36787602
What I believe is happening:
StreamReader.feed_data()
is receiving the first message (7F3678) and adds to the buffer.- It then flags for the
read()
to consume the data using_wakeup_waiter()
which sets the result on thefuture()
- Before the
read()
can be scheduled to consume the data, the second message (7602) is received; andStreamReader.feed_data()
is called and the buffer is extended with the new data. So, the buffer now contains 7F36787602. - The
read()
now occurs and both pieces of data are received together.
There is nothing in the received data to tell me this is >1 message so I need to ensure the order of one feed_data()
followed by read()
.
As a test, in StreamReader.feed_data()
I changed self._buffer.extend(data)
to self._buffer=bytearray(data)
Whilst this worked, it doesn't feel like the correct way of handling this. What would be the correct way to ensure read()
only gets one message at a time?
Solution
StreamReader
expects streaming data, such as that coming from a TCP/IP socket or from a pipe. Such data has no concept of "messages", only a stream of bytes. Even if the peer sends bytes in individual bursts, those may be merged or split up by the sender's or the receiver's OS, or by the network equipment between them. No change to feed_data
or other part of asyncio can fix that in the general case.
If you need to send individual messages over a TCP stream, you need to add explicit framing. For example, you can prefix the message with the message length, or you can insert a separator that cannot appear in the message between two messages. See this answer for an example of the former.
Answered By - user4815162342
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.