From rwatson@FreeBSD.org Sat Aug 28 13:04:46 2004 Date: Sat, 28 Aug 2004 11:20:40 -0400 (EDT) From: Robert Watson To: current@FreeBSD.org Subject: HEADS UP: Giant-free networking now the default in CVS HEAD (6.x) HEADS UP! The default setting for debug.mpsafenet has changed from 0 to 1, which means that by default, a number of kernel elements will now run without Giant by default, and often in parallel or preemptively. This includes: - Network interface ithreads for supported (MPSAFE) network device drivers. - The netisr thread when processing MPSAFE network protocols. - Many network-related system calls, including the send and receive paths. - Many network related timeout and callout routines happening with clock ticks. As mentioned in the commit message, there are now a number of seatbelts and ways to change the setting to restore more conservative operation, as this change may expose existing bugs or introduce new bugs. In fact, that is highly likely to be the case. If you experience new network related problems, as a first debugging step, please try putting the Giant lock back over the network stack. This can be done by setting the loader tunable "debug.mpsafenet" to 0, or by compiling "options NET_WITH_GIANT" into your kernel configuration. If the problem goes away, that doesn't necessarily mean it's a bug in MPSAFE operation, as we could just be encouraging an existing bug to expose itself, but it's also fairly likely it is. If you want to run in an environment with even more parallelism, you can set net.isr.enable to 1 to run network processing to completion in interrupt threads, which has the effect of deferring less work to the netisr. If you'd like to read more about the on-going work, including seeing earlier posts about the work, precautions to take, etc, you can read about it at: http://www.watson.org/~robert/freebsd/netperf/ I do anticipate a slightly bumpy ride with this setting now as the default, but we have easy ways to back off the change at run-time and compile-time should it get too bumpy. There will likely be some serious nits we pick up and fix in the next three to four days, in particular, so some caution might need to be exercised. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research ---------- Forwarded message ---------- Date: Sat, 28 Aug 2004 15:11:13 +0000 (UTC) From: Robert Watson To: src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org Subject: cvs commit: src/sys/conf options src/sys/sys kernel.h src/sys/net netisr.c rwatson 2004-08-28 15:11:13 UTC FreeBSD src repository Modified files: sys/conf options sys/sys kernel.h sys/net netisr.c Log: Change the default disposition of debug.mpsafenet from 0 to 1, which will cause the network stack to operate without the Giant lock by default. This change has the potential to improve performance by increasing parallelism and decreasing latency in network processing. Due to the potential exposure of existing or new bugs, the following compatibility functionality is maintained: - It is still possible to disable Giant-free operation by setting debug.mpsafenet to 0 in loader.conf. - Add "options NET_WITH_GIANT", which will restore the default value of debug.mpsafenet to 0, and is intended for use on systems compiled with known unsafe components, or where a more conservative configuration is desired. - Add a new declaration, NET_NEEDS_GIANT("componentname"), which permits kernel components to declare dependence on Giant over the network stack. If the declaration is made by a preloaded module or a compiled in component, the disposition of debug.mpsafenet will be set to 0 and a warning concerning performance degraded operation printed to the console. If it is declared by a loadable kernel module after boot, a warning is displayed but the disposition cannot be changed. This is implemented by defining a new SYSINIT() value, SI_SUB_SETTINGS, which is intended for the processing of configuration choices after tunables are read in and the console is available to generate errors, but before much else gets going. This compatibility behavior will go away when we've finished the last of the locking work and are confident that operation is correct. Revision Changes Path 1.478 +1 -0 src/sys/conf/options 1.12 +84 -3 src/sys/net/netisr.c 1.118 +6 -0 src/sys/sys/kernel.h _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"