Here is a mental transcript of a bunch of things I tried to do this morning when trying to figure out how to listen in on a socket and get information from a client. I have never done this in a language with manual memory allocation, and that small detail expanded into many issues.
- On the server-side, we have a
TcpListner, giving us a
accept(). Remember the canonical node.js "echo server" example?
- The stream should allow me to both read and write data. Since this is Fancy Systems Programming(TM) I should definitely consider the nuts and bolts of getting the bytes into my program.
TcpStreamdocs don't have any
writemethod documentation, but links over to the
read(buffer)takes in a buffer (and the buffer knows its size, thankfully). So I should probably just make a buffer of adequate size. 4 kilobytes maybe?
Wait, does this mean that
socket.read(buffer)blocks until I receive 4 kilobytes of data? What about performance?
read's behaviour anyways?
This function does not provide any guarantees about whether it blocks waiting for data, but if an object needs to block for a read and cannot, it will typically signal this via an Err return value.
- This is probably one of those "depends on the implementing structure" things.
TcpStreammight have an answer. It has
set_nodelay... but that disables the Nagle algorithm. This is about sending, not reading!
- Maybe I set a
read_timeout? But if I just send a short command of less than 4kb then my server is waiting for timeouts each time... something feels off.
- Wait, is it even OK for me to do
buffer = [0; 4096];? Is stuffing this data in the stack good? Is heap allocation better? I think that's slower? The Alpine Linux people were complaining about people using the stack..
- Oh there's a
readbut also a
- I have read enough documentation, I should actually try writing some code. Let's pull out
netcatand at least experimentally figure out what's going on with a basic
readcall into my 4 kb buffer.
netcatgives me... line-buffering? Is this a default quality of TCP? I feel like "TCP line buffering" should show up somewhere on the internet?
man netcathas an input buffer size, maybe if I make it 1 byte it will remove line buffering.
- that doesn't affect line buffering.
- Alright this seems to help me out, and reminded me of terminal line buffering being a thing.
- OK this doesn't work, maybe this is an emacs/vterm thing.
- Alright this doesn't work in
- OK I was doing the input and outbut buffers in the wrong order
- Gonna use Google instead of Duck Duck Go to look a bit more. I find this
- I am no longer using my Rust program for tests, instead trying with netcat on both ends (as I should have a while back probably). That should remove all sort of details here, I hope...
- Finally, I found an incantation that works!
stty -icanon && nc 127.0.0.1 4501 # sender side stdbuf -i0 -o0 nc 127.0.0.1 4501 -lk -I 1 # receiver side
Typing keys goes over instantly! No buffering!
- Time to work backwards a bit. Turns out that "normal" netcat as a listener works (no
nc 127.0.0.1 4501 -lk), and the sending just requiring
- Pointing the canonized netcat to my Rust program seems to indicate reading being instant (and, unsurprisingly, I am incapable of typing faster than the read/write loop working, meaning my buffer is always just one character wide).
- Someone has now pointed me to
BufReader, which could manage the buffer stuff for me anyways, resolving my "should I through stuff on the stack or not" question.
Main Lesson: when trying to figure something out, try to use the simplest tool to remove as many variables as possible. Even that won't be enough, but it will help. I also have a newfound appreciation for "just use HTTP".