Here is a mental transcript of a bunch of things I tried to do this morning when trying to figure out how to listen in on a socket and get information from a client. I have never done this in a language with manual memory allocation, and that small detail expanded into many issues.

  • On the server-side, we have a TcpListner, giving us a TcpStream when calling accept(). Remember the canonical node.js "echo server" example?
  • The stream should allow me to both read and write data. Since this is Fancy Systems Programming(TM) I should definitely consider the nuts and bolts of getting the bytes into my program.
  • TcpStream docs don't have any read/write method documentation, but links over to the std::io::Read
  • read(buffer) takes in a buffer (and the buffer knows its size, thankfully). So I should probably just make a buffer of adequate size. 4 kilobytes maybe?

  • Wait, does this mean that socket.read(buffer) blocks until I receive 4 kilobytes of data? What about performance?

  • What is read's behaviour anyways?

    This function does not provide any guarantees about whether it blocks waiting for data, but if an object needs to block for a read and cannot, it will typically signal this via an Err return value.

  • This is probably one of those "depends on the implementing structure" things. TcpStream might have an answer. It has set_nodelay... but that disables the Nagle algorithm. This is about sending, not reading!
  • Maybe I set a read_timeout? But if I just send a short command of less than 4kb then my server is waiting for timeouts each time... something feels off.
  • Wait, is it even OK for me to do buffer = [0; 4096]; ? Is stuffing this data in the stack good? Is heap allocation better? I think that's slower? The Alpine Linux people were complaining about people using the stack..
  • Oh there's a read but also a read_exact?
  • I have read enough documentation, I should actually try writing some code. Let's pull out netcat and at least experimentally figure out what's going on with a basic read call into my 4 kb buffer.
  • Using netcat gives me... line-buffering? Is this a default quality of TCP? I feel like "TCP line buffering" should show up somewhere on the internet?
  • man netcat has an input buffer size, maybe if I make it 1 byte it will remove line buffering.
  • that doesn't affect line buffering.
  • Alright this seems to help me out, and reminded me of terminal line buffering being a thing.
  • OK this doesn't work, maybe this is an emacs/vterm thing.
  • Alright this doesn't work in kitty either...
  • OK I was doing the input and outbut buffers in the wrong order
  • Gonna use Google instead of Duck Duck Go to look a bit more. I find this
  • I am no longer using my Rust program for tests, instead trying with netcat on both ends (as I should have a while back probably). That should remove all sort of details here, I hope...
  • Finally, I found an incantation that works!
  stty -icanon && nc 127.0.0.1 4501 # sender side
  stdbuf -i0 -o0 nc 127.0.0.1 4501 -lk -I 1 # receiver side

Typing keys goes over instantly! No buffering!

  • Time to work backwards a bit. Turns out that "normal" netcat as a listener works (no stdbuf, nc 127.0.0.1 4501 -lk), and the sending just requiring -icanon is sufficient.
  • Pointing the canonized netcat to my Rust program seems to indicate reading being instant (and, unsurprisingly, I am incapable of typing faster than the read/write loop working, meaning my buffer is always just one character wide).
  • Someone has now pointed me to BufReader, which could manage the buffer stuff for me anyways, resolving my "should I through stuff on the stack or not" question.

Main Lesson: when trying to figure something out, try to use the simplest tool to remove as many variables as possible. Even that won't be enough, but it will help. I also have a newfound appreciation for "just use HTTP".