The python nailgun client doesn't seem to like system processes

[jvican]

I have tried to run the Echo nail example via a system process, where my driver (which is a java process) runs the python script, sends input and waits for output. To do this, I use NuProcess.

I am, in fact, using system in and system out to communicate with a process with a certain protocol. The message I am sending from the java driver has the following shape: "$HEADER\r\n$BODY".

Here's the thing: when I run my process without nailgun, my application logic is run; when I run it with nailgun, it gets stucked and the java process doesn't get anything back via stdout. I have dug into the code and the culprit seems to be the nailgun python script.

This is what I have found:

• There is indeed a race condition between the stdin thread and the logic that either sends the stdin to the server or receives it back. Sometimes it may read one part of what I send to the nailgun process (I found by adding some printlns + flush), but nothing is outputted after.
• The stdin reading logic of the python script only reads line per line, read. This is problematic for me because (following the shape of my messages) the script will block and never read $BODY if there's no subsequent message including \n. This makes nailgun unusable for communication protocols.

Is there a way these issues can be addressed? I'm happy to have a look at them and implement a better approach, but would like to get some feedback on this first.

[jvican]

Note that this is related with the latency issue explained in #88, but the showstopper here is that it reads line by line, and the whole communication gets blocked.

[sbalabanov-zz]

Race conditions should be fixed with Nailgun 0.9.3, however readline() will still remain in python client. One can change readline() to just buffered fetch with read(size), but the main problem remains - there is no good portable way to read as much as possible from stdin without blocking. There are some less portable ways like fcntl() and select() but we would like to avoid custom code.
stdin is usually used to transfer text-formatted data thus readline() works fairly well in most cases.

[jvican]

Good. I'm fine living without nailgun for protocol communication and I'm instead using IPC sockets. Let's leave this ticket open in case other folks wonder if nailgun is fit for something else that is not human-friendly (with always a newline at the end) communication.