-------- Original Message -------- Subject: Network performance on Mac and Linux Date: Mon, 29 Aug 2005 15:15:41 -0400 From: David P. Reed dpreed@reed.com To: David A. Smith davidasmith@bellsouth.net, Andreas Raab Andreas.Raab@SqueakLand.org
I'm sure you'd be as excited as I am to hear that I finally found the last piece of the network performance issue I've been chasing on the Mac and Linux.
Over the weekend, I built a Mac VM with the same Socket improvements that I had done on Windows and Linux. It had the same flaws as the Linux code (not surprisingly, since the SocketPlugin source code is identical).
The pieces of the puzzle that were Mac/Unix specific were:
1) the rate of polling for I/O events was once per 200 milliseconds, hard coded into checkForInterrupts part of the interpreter. (Windows Socket I/O didn't depend on this since it uses high-priority threads to wait for I/O). I changed it to once per 1 millisecond. This reduced the delay from well over 200 milliseconds to about 32 msec., averaged over a number of packets.
2) The unix driver didn't set TCP_NODELAY as a socket option by default, but Windows did. I changed the plugin to do this. This cut the measured end-to-end delay on packets from 32 msec. to < 1 msec. (Gaah - I had just assumed that this was done compatibly, and hadn't checked for it).
So I will soon (tonight) have 3 VMs that have higher performance TCP network code. They should support the VP2PPN layer cross-platform.
I'm wondering whether TCP_NODELAY should be default on or off for Squeak. The default should be the same on ALL platforms, in any case, and documented. For "real-time" stuff like Croquet and VoIP, TCP_NODELAY should be on (as it was for Windows). For batch file transfer, TCP_NODELAY should be off, because it helps by packing packets full of data.
Andreas Raab wrote:
I'm wondering whether TCP_NODELAY should be default on or off for Squeak. The default should be the same on ALL platforms, in any case, and documented. For "real-time" stuff like Croquet and VoIP, TCP_NODELAY should be on (as it was for Windows). For batch file transfer, TCP_NODELAY should be off, because it helps by packing packets full of data.
Since the option can be set/reset for each socket, the default setting should be the one which satisfies most naive socket uses, and IMHO this would be the batch file transfer kind of stuff. TCP_NODELAY only makes sense for realtime applications where interactive response is critical, such as Croquet, and these applications should know what they're doing and should set TCP_NODELAY...
Cheers, Hans-Martin
Don't forget we added the feature to alter socket options via a primitive call. Applications could set TCP_NODELAY as needed.
FYI the unix ones supported are below (from the source code).
static socketOption socketOptions[]= { { "SO_DEBUG", SOL_SOCKET, SO_DEBUG }, { "SO_REUSEADDR", SOL_SOCKET, SO_REUSEADDR }, { "SO_DONTROUTE", SOL_SOCKET, SO_DONTROUTE }, { "SO_BROADCAST", SOL_SOCKET, SO_BROADCAST }, { "SO_SNDBUF", SOL_SOCKET, SO_SNDBUF }, { "SO_RCVBUF", SOL_SOCKET, SO_RCVBUF }, { "SO_KEEPALIVE", SOL_SOCKET, SO_KEEPALIVE }, { "SO_OOBINLINE", SOL_SOCKET, SO_OOBINLINE }, { "SO_LINGER", SOL_SOCKET, SO_LINGER }, { "IP_TTL", SOL_IP, IP_TTL }, { "IP_HDRINCL", SOL_IP, IP_HDRINCL }, { "IP_MULTICAST_IF", SOL_IP, IP_MULTICAST_IF }, { "IP_MULTICAST_TTL", SOL_IP, IP_MULTICAST_TTL }, { "IP_MULTICAST_LOOP", SOL_IP, IP_MULTICAST_LOOP }, { "TCP_MAXSEG", SOL_TCP, TCP_MAXSEG }, { "TCP_NODELAY", SOL_TCP, TCP_NODELAY },
On 29-Aug-05, at 12:24 PM, Andreas Raab wrote:
-------- Original Message -------- Subject: Network performance on Mac and Linux Date: Mon, 29 Aug 2005 15:15:41 -0400 From: David P. Reed dpreed@reed.com To: David A. Smith davidasmith@bellsouth.net, Andreas Raab Andreas.Raab@SqueakLand.org
I'm sure you'd be as excited as I am to hear that I finally found the last piece of the network performance issue I've been chasing on the Mac and Linux.
Over the weekend, I built a Mac VM with the same Socket improvements that I had done on Windows and Linux. It had the same flaws as the Linux code (not surprisingly, since the SocketPlugin source code is identical).
The pieces of the puzzle that were Mac/Unix specific were:
- the rate of polling for I/O events was once per 200 milliseconds,
hard coded into checkForInterrupts part of the interpreter. (Windows Socket I/O didn't depend on this since it uses high-priority threads to wait for I/O). I changed it to once per 1 millisecond. This reduced the delay from well over 200 milliseconds to about 32 msec., averaged over a number of packets.
- The unix driver didn't set TCP_NODELAY as a socket option by
default, but Windows did. I changed the plugin to do this. This cut the measured end-to-end delay on packets from 32 msec. to < 1 msec. (Gaah - I had just assumed that this was done compatibly, and hadn't checked for it).
So I will soon (tonight) have 3 VMs that have higher performance TCP network code. They should support the VP2PPN layer cross-platform.
I'm wondering whether TCP_NODELAY should be default on or off for Squeak. The default should be the same on ALL platforms, in any case, and documented. For "real-time" stuff like Croquet and VoIP, TCP_NODELAY should be on (as it was for Windows). For batch file transfer, TCP_NODELAY should be off, because it helps by packing packets full of data.
-- ======================================================================== === John M. McIntosh johnmci@smalltalkconsulting.com 1-800-477-2659 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ======================================================================== ===
vm-dev@lists.squeakfoundation.org
-
Andreas Raab -
Hans-Martin Mosner -
John M McIntosh