Merge "Support for binary data in the websocket transport"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
7ccb24eb8a
@ -5,5 +5,5 @@
|
||||
.. toctree::
|
||||
:glob:
|
||||
:maxdepth: 2
|
||||
.. FIXME(mdnadeem): Uncomment after adding first real spec.
|
||||
.. *
|
||||
|
||||
*
|
||||
216
specs/mitaka/websocket-binary-support.rst
Normal file
216
specs/mitaka/websocket-binary-support.rst
Normal file
@ -0,0 +1,216 @@
|
||||
..
|
||||
This template should be in ReSTructured text. The filename in the git
|
||||
repository should match the launchpad URL, for example a URL of
|
||||
https://blueprints.launchpad.net/zaqar/+spec/awesome-thing should be named
|
||||
awesome-thing.rst.
|
||||
|
||||
Please do not delete any of the sections in this
|
||||
template. If you have nothing to say for a whole section, just write: None
|
||||
|
||||
For help with syntax, see http://sphinx-doc.org/rest.html
|
||||
To test out your formatting, see http://www.tele3.cz/jbar/rest/rest.html
|
||||
|
||||
===================================================
|
||||
Support for binary data in the websocket transport
|
||||
===================================================
|
||||
|
||||
https://blueprints.launchpad.net/zaqar/+spec/websocket-binary-support
|
||||
|
||||
Implement support for binary communications over websocket transport which will
|
||||
allow to reduce network traffic between Zaqar and it's clients and also to
|
||||
increase performance.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
Currently websocket transport uses only text messages packed in JSON format to
|
||||
exchange data with clients.
|
||||
|
||||
An attempt to send binary data to Zaqar's websocket transport will lead to
|
||||
receiving 'bad request' (HTTP error 400) response from Zaqar.
|
||||
|
||||
But websocket protocol actually allows communication via binary messages.
|
||||
Data packed(serialized) in binary format can be smaller and also may take less
|
||||
CPU time to pack/unpack it.
|
||||
|
||||
By using binary messages for communications it's possible to reduce network
|
||||
traffic between Zaqar and it's clients and also to increase Zaqar's
|
||||
performance. It can also increase client performance if the client is not
|
||||
written in JavaScript.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
We can serialize/deserialize data via MsgPack technology which was already
|
||||
proposed and approved in persistent-transport_ specification.
|
||||
|
||||
.. _persistent-transport: https://github.com/openstack/zaqar-specs/blob/master/specs/kilo/approved/persistent-transport.rst
|
||||
|
||||
Deserialization of incoming messages to Zaqar
|
||||
---------------------------------------------
|
||||
|
||||
As you can see in `protocol.py#L55`_, Zaqar already can distinguish binary and
|
||||
text messages.
|
||||
|
||||
.. _protocol.py#L55: https://github.com/openstack/zaqar/blob/stable/liberty/zaqar/transport/websocket/protocol.py#L55
|
||||
|
||||
So we will make Zaqar try to deserialize each received binary message via
|
||||
'msgpack' library into a python object, just like Zaqar currently deserializes
|
||||
JSON messages via 'json' library. After successful deserialization the
|
||||
message can be processed as usual.
|
||||
|
||||
Serialization of response and subscription messages from Zaqar
|
||||
--------------------------------------------------------------
|
||||
|
||||
The response from Zaqar will be serialized in the same format as the incoming
|
||||
request.
|
||||
|
||||
The subscription messages from Zaqar will be serialized in binary or text
|
||||
format based on the format of subscription create request.
|
||||
|
||||
Let's take a look at the case where the client suddenly changes it's messaging
|
||||
format while connection is open and subscription is still alive. There might
|
||||
be a problem when the client will not be able to deserialize the messages
|
||||
coming from the old subscription, because it has switched the format by
|
||||
changing some boolean variable (switch). But this situation is nearly
|
||||
impossible and, if not, the problem is solvable:
|
||||
|
||||
#. It's hard to imagine a situation when it might be useful for the client to
|
||||
change it's serialization format. Such client will probably never exist.
|
||||
#. Even if such client will exist in the future, the client developer will
|
||||
take care of it by adding the code checking the format of each incoming
|
||||
message and deserializing the message accordingly inside "on message"
|
||||
method. Zaqar does exactly this, for example. All websocket implementations
|
||||
provide a convenient way to determine the format of each particular message.
|
||||
#. Even if the client developer will not take care of this potential problem
|
||||
(he is inexperienced, for example) and will make the client just expect
|
||||
all incoming messages in the new format after switching while the old
|
||||
subscription is still alive, the client will crash on the **first incoming
|
||||
message from this subscription** and the problem will not go unnoticed.
|
||||
|
||||
|
||||
Websocket html client example upgrade
|
||||
-------------------------------------
|
||||
|
||||
The websocket html `client example`_ will be changed to make it also being able
|
||||
to communicate with Zaqar via binary messages.
|
||||
|
||||
.. _client example: https://github.com/openstack/zaqar/blob/stable/liberty/examples/websocket.html
|
||||
|
||||
Drawbacks
|
||||
---------
|
||||
|
||||
The performance of web clients on JavaScript
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Seems like MsgPack serialization/deserialization works slower than JSON in
|
||||
JavaScript. Well, no surprise since JSON abbreviation expands to "JavaScript
|
||||
Object Notation".
|
||||
It's a drawback for Zaqar web clients as it decreases their performance.
|
||||
|
||||
How the change addresses this drawback:
|
||||
|
||||
#. The server is one and clients are many. The server processes much more
|
||||
messages than each particular client. So there are many cases when it's
|
||||
reasonable to use binary serialization to increase server performance by
|
||||
sacrificing performance of JavaScript clients.
|
||||
|
||||
#. The additional delays on JavaScript clients are slightly compensated by
|
||||
faster responses from the server.
|
||||
|
||||
#. JavaScript clients can always use good old text format for communications
|
||||
with the server, while other clients can use binary format with the same
|
||||
server simultaneously.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
These alternatives completely exclude the nearly impossible case where the
|
||||
client may suddenly switch it's messaging format, while the connection is open
|
||||
and the old subscription is still alive, and while proper deserialization of
|
||||
incoming messages to the client is managed by some client's boolean variable
|
||||
(switch).
|
||||
|
||||
Alternatives of serialization of response and subscription messages from Zaqar
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Optional parameter approach
|
||||
"""""""""""""""""""""""""""
|
||||
|
||||
The response from Zaqar will be serialized in the same format as the incoming
|
||||
request.
|
||||
|
||||
The subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on the new optional parameter passed in the message
|
||||
body of subscription create request. We can add this parameter to websocket
|
||||
API, name it ``accept``, allow it to have only two possible values: ``json``
|
||||
and ``msgpack`` and make ``json`` value as default if ``accept`` parameter was
|
||||
not passed.
|
||||
By making ``accept`` parameter optional, we are achieving backward
|
||||
compatibility.
|
||||
|
||||
Subscription request message example before implementation::
|
||||
|
||||
{
|
||||
'action': 'subscription_create',
|
||||
'headers': {'Client-ID': 31209ff3-ba03-4cec-b4ca-655f4899f8aa, 'X-Project-ID': superproject},
|
||||
'body': {'queue_name': 'superqueue', 'ttl': 3600}
|
||||
}
|
||||
|
||||
Subscription request message after implementation can be the same as before or
|
||||
it can also pass ``accept`` parameter explicitly::
|
||||
|
||||
{
|
||||
'action': 'subscription_create',
|
||||
'headers': {'Client-ID': 31209ff3-ba03-4cec-b4ca-655f4899f8aa, 'X-Project-ID': superproject},
|
||||
'body': {'queue_name': 'superqueue', 'ttl': 3600, 'accept': 'json'}
|
||||
}
|
||||
|
||||
First message approach
|
||||
""""""""""""""""""""""
|
||||
|
||||
The response and subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on in which format the particular client has sent it's
|
||||
first successfully parsed message to Zaqar since connecting.
|
||||
|
||||
Once the format of communications between Zaqar and the client was
|
||||
established by that way, the client will not be able to change it, unless the
|
||||
client will reconnect to Zaqar and send the first message in other format.
|
||||
|
||||
Zaqar configuration approach
|
||||
""""""""""""""""""""""""""""
|
||||
|
||||
The response and subscription messages from Zaqar will be serialized in binary
|
||||
or text format based on the boolean variable in Zaqar server configuration.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
ubershy
|
||||
|
||||
Milestones
|
||||
----------
|
||||
|
||||
Target Milestone for completion:
|
||||
Mitaka-2
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
None
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
.. note::
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
Loading…
Reference in New Issue
Block a user