Java NIO inherent flaw in reading from one socket and writing to another?

Ben :

I working on a basic minimal server concept flowing data from one port to another destination using NIO and socket channels.

Things work great at full speed, fast links, etc. Things fail terribly and consume a ton of CPU when one side is faster than the other. Things work, just work very inefficiently.

Example:

Iterator keys = selector.selectedKeys().iterator();
while (keys.hasNext()) {
    SelectionKey key = (SelectionKey) keys.next();
    keys.remove();
    try {
        if (!key.isValid()) continue;
        if (key.isReadable()) read(key);
    }
    catch (Exception e) {
        e.printStackTrace();
        key.cancel();
    }
}

The read call happens anytime the socket has data that can be read. But what if the writeable portion of this socket can't be written to because the client side has high latency or just isn't reading the data fast? We end up looping at an insane rate doing nothing until a little bit more writeable buffer frees up:

public void read(SelectionKey key) throws IOException {
    ByteBuffer b = (ByteBuffer) buffers.get(readable); //prior buffer that may or may not have been fully used from the prior write attempt
    int bytes_read = readable.read(b);
    b.flip();
    if (bytes_read > 0 || b.position() > 0) writeable.write(b);
    b.compact();
}

So say we can read from the socket at a gigabit, but the receiver is only reading from our writeable socket at 100kilobit....we might loop a million times between each little piece of data we can write over to the client since they just aren't consuming the data in the socket buffers as fast as we want.

If I did this with a Thread, there would be no issue as we would be blocking on the write call. But with NIO, what are you supposed to do to let it skip the read notifications?

Ben :

I figured this out. I still didn't find docs pointing me at this, but after considering things more I realized this is what the OP_WRITEABLE scenario is for.

So when the writeable returns that it accepted 0 bytes, I deregister the OP_READ on the readable channel, and register for OP_WRITEABLE on the writeable channel. Once I am notified that its writeable, I swap back the read/write channels, including what they are registered for dropping the OP_WRITEABLE now.

So when a write can't be done, de-register the read that is triggering you to try and write to it, and register for a write notification on it instead. Once you get that, swap back registrations.

Things work fine now.