https://www.npmjs.org/package/utf8 ? not to reinvent the wheel ?

Yes, if we decide binary messages are a good idea, I will get the encoding from a library rather than ship a copy from StackOverflow.

I am still not clear on the usage case for this. Won't the existence of a binary channel encourage people to start creating widgets and such that only work on a particular kernel. Forcing JSON in this case makes sure that all kernels can easily implement the kernel side of a comm/widget and reuse the frontend.

There's nothing kernel specific about binary messages. Our wire (zmq) format already supports arbitrary binary data attached to every message; there's nothing Python-specific about it.

But the binary stuff exists mainly to support the parallel computing stuff right. Do we use it outside that context? While the binary support itself is not language specific, I think it will tend to be used in language specific ways.

I would still like to understand the usage cases where having binary comm messages is important. I suspect the idea is that it would be used for things like a spreadsheet widget where you want to send lots of data back and forth. Even in that context I want to push back a bit. To have such a widget work for all kernels, we have to make a decision about a a binary array format that all languages agree upon and that is also implemented in JavaScript. I am not at all convinced that is where we want to go if that is the main usage case.

But if the main usage case is sending binary - non base64 encoded images - through comms, I could imagine that might be useful in certain cases.

This came out of conversations with @mdboom in supporting a live comm-based matplotlib object. Michael noticed a nontrivial CPU drain when base64-encoding the images to send over comm messages.

@minrk: how will this affect people implementing a frontend in another language besides javascript?

But the binary stuff exists mainly to support the parallel computing stuff right. Do we use it outside that context? While the binary support itself is not language specific, I think it will tend to be used in language specific ways.

Only data_pub and apply use binary data at the moment. The main use case I can think of is user-defined widgets with comm messages, sending data like images without having to b64-encode them. There are plenty of binary data formats that one could handle in javascript, images being the simplest and most useful (e.g. matplotlib nbagg). It's inherently not language-specific, because it must be understood at the very least in both javascript and the kernel.

@minrk: how will this affect people implementing a frontend in another language besides javascript?

I think the answer to my question is that the wire format of the websocket messages changes here from JSON text to this new wire format. So people would have to implement the unserialize function like you did in javascript. Is that correct?

So people would have to implement the unserialize function like you did in javascript. Is that correct?

Yes, this changes the websocket wire format (when binary messages are present) to be more consistent with the zmq wire format.

Yes, this changes the websocket wire format (when binary messages are present) to be more consistent with the zmq wire format.

Ah, looking at the code, I see what you mean. There is no change needed if no messages are sent with binary buffers. It's only when receiving messages with binary buffers that you need to implement the new wire format.

Thanks for a bit more background on this. I guess I am fine with us moving forward with this.

Overall the code looks good. Can you add some tests though?

I have a few things I need to do on this, but various other things I have to get to first. I probably won't have time to come back to this for at least a few weeks.

To add my +1 to this idea, the use case of sending array data to the clients without multiple encoding/decoding steps is a real one we're already encountering. This thread on the list (esp. subsequent messages) illustrate well cases where too many encode/decode steps will introduce significant latencies with very real impact on user experience, making certain kinds of analysis/visualization widgets impossible to implement. They would simply be too slow to be usable.

Right now, projects that implement these kinds of widgets with high framerate image exploration tend to simply ship the entire dataset over to JS in one giant blob and hold it all in memory in JS. But that precludes the kind of kernel-side richer computation we can offer, since they are then forced to implement any analytics entirely client-side in JS. Being able to efficiently send binary array data to the clients will open lots of possibilities here.

Need rebase.

I've written js tests, but can't run them because of phantomjs being stuck on a super old websocket implementation. That being the case, I can commit my test notebook to examples/tests, but that's about it.

I just included the test and restricted it to slimer while we wait for phantom 2.0.

Need rebase.

rebased, and should be ready to go

Ok, let's not block thin on wether to use $ or _.

Merging soon if no objections.

Thanks, last quick read and merging.

Thanks for this!
Can this feature be readily used in a widget's send() method?

The widget's underlying comm takes a buffers arg, so just call that and you should be set.