FreeBSD Monthly Status Report, July 2001 Robert Watson Introduction Last month's status report was apparently a great success: I received countless e-mails with comments, questions, and suggestions. I've tried to incorporate any suggestions and address any problems from these e-mails in this month's report, which captures a far more extensive snapshot of FreeBSD activity in the last month. Unlike last month's report, it does a better job of reflecting non-development activity, such as on-going conference planning, documentation, and so on. This is a trend I hope to see improve in future months as well. On the topic of conferences, in the future I'd like to report more on publication activities relating to FreeBSD, including online journals with articles relating to FreeBSD, paper journals, conference papers, and so on. Likewise, I would be interested in including references to Call for Papers relating to FreeBSD. I'll take this opportunity to plug both registration and paper submission for BSDCon Europe in November, which has status included in this report, and for the general BSD Conference being hosted by USENIX in February. Your attendance and submissions make these conferences "happen", and promote FreeBSD as a platform for new research, feature development, and application products. Work of extremely high calibre is performed on FreeBSD, and we need to get the word out. Submission for Future Editions Next month, we're maintain much the same submission requirements: reports should be one or two paragraphs long, sent by e-mail, and approximate the layout of the entries this month (Project, Contact, URL, and text). I'll send out reminders again over the week before the deadline. An area where I'd like to explore improvement lies in the coordination of related status reports for larger projects, such as new architectural work or platform ports. This might even have the effect of encouraging communication on these projects :-). Projects The following projects submitted summaries for the July 2001 report: ACPI ARM Port BIND 9 binup BSDCon Europe CAM "Close a PR drive" Documentation Project Fibre Channel Support Hardware Watchpoints in the Kernel Debugger ifconfig support for IEEE 802.11 wireless devices jailNG FreeBSD Java Project jpman project Kernel Summit - Usenix 2001 KSE threading the kernel FreeBSD Monthly Development Status Reports NetBSD rc.d port Netgraph ATM network device cloning Next Generation POSIX threads (NGPT) OLDCARD upgrade to support PCI cards Open Runtime Platform (ORP) OpenPackages PAM PowerPC Port PPP IPv6 Support Porting ppp to hurd & linux pppoed PRFW - Hooks within the FreeBSD kernel SCSI Tape Support SMPng SMPng mbuf allocator sparc64 port FreeBSD/sparc64 kernel loader SYN cache implemetation for FreeBSD TrustedBSD Project Status Reports Project: ACPI Contact: Mike Smith ACPI (Advanced Configuration and Power Interface) is an industry standard which obsoletes APM, Intel MPS, PnPBIOS, and other Intel PC firmware interface standards. It is also used on the IA64 platform. More information on ACPI is available at http://developer.intel.com/technology/iapc/acpi The FreeBSD ACPI subsystem project is based heavily on the Intel ACPI Component Architecture. This status report outlines the current state of the project; future updates will focus on changes as they occur. The Intel ACPI interpreter is fully integrated, although bugs are still coming out of the woodwork occasionally. - PCI bus detection and interrupt routing are functional, but power management interaction will require work on the core PCI subsystem. - Non-PCI motherboard peripheral probing is implemented, but believed to have problems on some systems. - A power policy manager has been implemented. The initial policy manager has two modes, "performance" and "economy". - CPU speed throttling is integrated with the platform power policy. - System thermal monitoring is implemented, but fan control is believed to have problems. - Pushbutton suspend and power-off is implemented. - System time-keeping using the ACPI timer is supported. - Battery status monitoring is implemented. Work is ongoing in the following areas: - System suspend and resume. - Timekeeper accuracy/reliability. - Power profiles. - User-level management interfaces. - PCI power management. - Bug-hunting. Project: ARM Port Contact: Stephane E. Potvin The ARM port is currently going pretty well. The kernel is compiling and is able to boot to the point where it panics trying to initialize the network subsystem. The current reference platform is the Netwinder but this may change as many people expressed interest in a more broadly available platform. Things that need to be done before it can get further includes adding footbridge, timer and interrupt supports. The pmap module is not completed yet either. Project: BIND 9 Contact: Doug Barton , Jeroen Ruigrok Now that BIND 8.2.4 is finally imported the time has come to look at getting BIND 9 imported into CURRENT. The current idea is to have it imported alongside BIND 8 so that people can play with either one until all import problems have been taken care of and people have tested it a bit. Project: binup Contact: Eric Melville Although gaining a new name, the project has been at a standstill due to both resource availability during the move between BSDi and Wind River, and other commitments of the developers. The project should obtain an official mailing list, as well as return to an active state after the dust settles. Project: BSDCon Europe URL: http://www.bsdconeurope.org Contact: Paul Richards , Josef Karthauser The conference will take place at the Thistle Hotel, Brighton, UK from 9-11 November 2001. The aim of the conference is to provide a focal point for European users and developers of all the BSD derived operating systems. The format will be similar to other conferences, with 2 days of technical sessions over the Saturday and Sunday. We'll be finalizing the schedule towards the end of the month and anybody who is interested in doing a talk should contact us ASAP. There are no restrictions on the use of talks, if it's been done before we may still be interested in having it presented to an European audience, and we make no claims to the talks so speakers are free to present the talks again at other conferences. We're also still looking for sponsors. We had 80 pre-registrations in the first week so we're expecting a good turnout. Project: CAM Contact: mjacob@freebsd.org, gibbs@freebsd.org, ken@freebsd.org The new CAM transport code is starting to get supported in more HBAs and to get refined so that it does the intended per-protocol support. No progress on doing any SMPNG work for CAM has been made yet. This is a fairly high priority. Project: "Close a PR drive" URL: http://phk.freebsd.dk/Gnats/ Contact: phk@FreeBSD.org Thanks to various outstanding individual efforts, we are now down to just below 2300 open bug-reports. This means that we have fought our way back to the level we had around march 2000. Project: Documentation Project URL: http://www.FreeBSD.org/docs.html URL: http://www.FreeBSD.org/docproj/index.html Contact: Documentation Project Work continues (in large part sponsored by WRS) on updating the Handbook ready for the second print edition. There has been a flurry of activity in this area recently, and the ToDo list can be seen at http://www.freebsd.org/docproj/handbook.html Dima and others are doing a stellar job of keeping up with the steady flow of incoming PRs relating to the documentation project. The Developers' Handbook, http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/index.html is a year old; it contains a wealth of useful content for developers developing on, or for, FreeBSD. As ever, more contributions are always required, not only for the developers' handbook, but for all of the FreeBSD documentation set. Project: Fibre Channel Support Contact: mjacob@feral.com The basic design hasn't changed and this project mainly is in the phase of continued hardening and test case development. The next major feature will be to fully integrate into the new CAM TRAN code and to fully support on the fly device addition and removal. The only HBA supported is QLogic at this time. Future support for the QLogic line is planned to have 2300 (2Gb) and IP support before October. Project: Hardware Watchpoints in the Kernel Debugger Contact: Brian Dean Hardware watchpoints are now available for kernel debugging on the IA32 (i386) architecture. One can now set hardware watchpoints using the new ddb command 'hwatch', which is analogous to the existing 'watch' command. Alternatively, if greater flexibility is required, direct access to the debug registers is available using the ddb 'set' command which allows complete control over the processor hardware debug facilities. Hardware watchpoints are very useful in tracking down those elusive memory overwrite bugs in the kernel. Hardware watchpoints can even be used to set a code breakpoint in ROM, which is commonly found in embedded systems. Project: ifconfig support for IEEE 802.11 wireless devices Contact: Brooks Davis Support for configuring IEEE 802.11 wireless devices via ifconfig has been committed to -current and -stable. It contains most of the functionality needed to configure an wireless device. Some missing features are being worked on including integrated support for DHCP so a single entry in /etc/rc.conf can be used to fully configure a wireless device on a DHCP lan and setting the CTS/RTS threshold. Currently the an(4) and wi(4) drivers are supported in -current and -stable with the awi(4) device supported in -current. Further work is needed to support Frequency Hopping devices such as ray(4). Project: jailNG Contact: Robert Watson jailNG is a from-scratch rewrite of the popular jail(8) service, focusing on improved management functions, as well as more fine-grained configurability. An initial prototype has been written, based on explicitly named and configured jails, and work is proceeding on userland integration. Currently, it's not clear if the timeline for this will be 5.0-RELEASE, or 5.1-RELEASE. Project: FreeBSD Java Project URL: http://www.freebsd.org/java/ Contact: glewis@eyesbeyond.com The main development in the FreeBSD Java Project over the last month was the release of an initial "Developers Only" patch set for the JDK 1.3.1. Since that release progress had been made towards a much more usable alpha quality patch set which is likely to be turned into a port, as per the current JDK 1.2.2 patch set. This new patch set will feature a number of bugfixes, which essentially get the JDK to a working state for early adopters, and an initial implementation of "native threads" based on FreeBSD's userland pthreads. Unfortunately this implementation isn't fully functional, but is included in the hope of more getting more eyeballs on the code (particularly experience pthread programmers). We'd also like to welcome Fuyuhiko Maruyama-san as a new committer, the usual punishment for too many good patches. Project: jpman project URL: http://www.jp.FreeBSD.org/man-jp/ (in Japanese) Contact: man-jp@jp.FreeBSD.org We have been working to provide Japanese version of FreeBSD online manuals, since 1996. Currently, RELENG_4 manuals are based. Translated versions are placed on doc/ja_JP.eucJP/man and provided to users using ports/japanese/man-doc. Also, we discuss about related commands (e.g. ports/japanese/man and ports/japanese/groff). Project: Kernel Summit - Usenix 2001 URL: http://www.FreeBSD.org/~jhb/summit/usenix01/ Contact: John Baldwin The first FreeBSD kernel summit meeting was held June 29-30, 2001 in Boston, MA at the Usenix 2001 Annual Technical Conference. Links to a variety of files are posted on the web site. Note: I (jhb) am still working on writing up a general summary of the meeting. When that is completed it will be posted here and mailed to the -hackers mailing list. Project: KSE threading the kernel URL: http://people.freebsd.org/~jasone/kse/ Contact: julian@elischer.org I'm working on multithreading the kernel. So far I have over 400KB of diffs relative to todays -current (I'm keeping my tree updated with changes as they occur rather than get hit with a big update at the end). I have split the proc structure and am changing most of the kernel to pass around a thread identifier instead of a proc structure. The following interfaces have been changed so far: device devsw entries vfs calls mutexes events system calls scheduler + a lot of code in between. I have still a lot of work to go with a lot of "dumb editing" (s/struct proc \*p/struct thread \*td/) usually I change a few items and then fix everything that breaks when I try compile it. I'd like to check it in on a branch so others can help the editing but haven't worked out the best way to do it yet. I have implemented changes to the scheduler so that kse's are scheduled instead of processes, and threads sleep, letting the kse pick up a new thread. but it's not anywhere ready yet (heck it doesn't compile yet :-) Note that I have not yet updated the document listed above.. everywhere it mentions "ksec" or "KSE-context", the code uses the word "thread". I will update it soon as Jason has sent me the source. Project: FreeBSD Monthly Development Status Reports URL: http://www.FreeBSD.org/news/status/ Contact: Robert Watson , Chris Costello The FreeBSD Monthly Development Status Report aims to keep users and developers up-to-date on the latest goings-on in the FreeBSD project by providing summaries of each project and its status. At the time of this writing, the July 2001 status report is being prepared and is very near release. The FreeBSD Web site now has a Status Reports section, which, when the July 2001 report is released, will be updated to include a link to an HTML-ified version. Project: NetBSD rc.d port URL: http://groups.yahoo.com/group/FreeBSD-rc Contact: dougb@FreeBSD.org, sheldonh@FreeBSD.org The NetBSD rc.d port aims to improve the FreeBSD startup process by porting Luke Mewburn's rc.d work from NetBSD to FreeBSD. This will score FreeBSD startup and shutdown dependencies without losing the traditional and much loved monolithic configuration file system. Luke Mewburn's USENIX paper and slides on the system as implemented in NetBSD are available here: http://groups.yahoo.com/group/FreeBSD-rc/message/3 Interested parties are urged to study this material before joining the discussion list. The intention at this stage is to decide on an approach that will ensure that the differences between the NetBSD rc.d system and the system as ported to FreeBSD will be kept to a minimum. This will probably involve discussions with Luke around those areas of the system that are identified as areas for potential improvement. Project: Netgraph ATM Contact: Hartmut Brandt The goal of this project is the implementation of ATM signaling and other ATM protocols by means of the netgraph(4) framework. This should provide an easily extendible architecture for using ATM on FreeBSD. Currently the full UNI4.0 stack (except for the LIJ capability) has been implemented, including ILMI and a first version of the ATM Forum API for UNI. An implementation of Classical IP over ATM is also available. Drivers have been implemented for the Fore PCA200E and Fore HE-155 cards. Project: network device cloning Contact: Brooks Davis Network device cloning support has been imported from NetBSD. This allows virtual devices to be allocated on demand rather then being statically allocated at compile time. Our implementation differs slightly from that of NetBSD's in that we allow both the creation of specific devices (i.e. gif0) and arbitrary devices instead of just allowing specific devices. Currently, the only device in the tree which has been converted is the gif(4) device which has been converted in both -current and -stable. Work is ongoing to convert all other virtual network devices with work in progress on faith, stf, and vlan interfaces. In general this conversion is accompanied by appropriate modifications to make these devices fully modular. Project: Next Generation POSIX threads (NGPT) URL: http://oss.software.ibm.com/developerworks/opensource/pthreads/ Contact: arun@sharmas.dhs.org Porting NGPT (next generation pthreads) to FreeBSD NGPT is an effort led by IBM engineers to implement MxN threads (also known as many user threads to one kernel thread mapping) on Linux. I have ported it to FreeBSD to use rfork(2). The port is right here: http://www.freebsd.org/cgi/query-pr.cgi?pr=29239 Project: OLDCARD upgrade to support PCI cards URL: http://people.freebsd.org/~imp/oldcard-status.html Contact: imp@village.org Funded by: Monzoon Networking, LLC This month has been a month of concentration and consolidation. Much of the changes from current have been migrating into stable. I've improved power support, suspend/resume interactions, interrupt handling, and ability to work after windows/NEWCARD has run. Interrupt routing continues to be a locking issue for a complete MFC. Current patches are available at the above website. I'm racing to get this done before 4.4 is released. Project: Open Runtime Platform (ORP) URL: http://www.intel.com/research/mrl/orp/ Contact: arun@sharmas.dhs.org, orp@egroups.com Information on Intel ORP - a BSD licensed Java VM is right here: http://www.intel.com/research/mrl/orp/ A FreeBSD patch has been tested to work with NGPT and submitted to the ORP project. The patch is available here: http://www.sharma-home.net/~adsharma/projects/orp/orp-freebsd-1.0.5.patch.txt.gz There are some issues to be ironed out to make it work with FreeBSD's default (user level) pthread implementation. Project: OpenPackages URL: http://openpackages.org/ OpenPackages intends to create a software packaging system that will allow third-party programs to be installed, without operating system dependent changes, on as many platforms as are feasible. OpenPackages was originally based on code from the BSD ports systems, and has been improved and extended by developers of many heritages. The OpenPackages Project is pleased to release the Milestone 2 codebase. This release contains a working package building system and a single test package. OP currently is known to build on certain instances of the following operating systems: FreeBSD, HP/UX, IRIX, Linux (Debian, Red Hat, Suse, Mandrake, TurboLinux, Caldera, etc.), NetBSD, OpenBSD, Solaris Project: PAM Contact: Mark R V Murray (First report) Large cleanup and extension of FreeBSD PAM modules. All modules are to be documented, consistent in style (style(9) used) and as complete as possible WRT functionality. Mostly done. Project: PowerPC Port Contact: benno@FreeBSD.org We now have the rudiments of device support. We have a nexus driver for OpenFirmware machines, along with support for the Apple UniNorth PCI/AGP host bridge. I'm currently trying to get the USB hardware working so that I can get closer to having a console driver independent of OpenFirmware, then I'll be trying to get the system to get to single-user mode using NFS. Project: PPP IPv6 Support Contact: brian@freebsd-services.com Work has begun, but nothing has yet been committed. The NCP addresses used by ppp have been abstracted and initial support has been added to the filter set for ipv6 addresses. NCP negotiation hasn't yet been started. Project: Porting ppp to hurd & linux Contact: brian@Awfulhak.org Patches have been submitted to get ppp working under HURD, and mostly under Linux. There are GPL copyright problems that need to be addressed. Project: pppoed Contact: brian@freebsd-services.com Making pppoed function in a production environment. Most of the work is complete and committed. Additional work includes adding a -l option where ``-l label'' is shorthand for ``-e exec ppp -direct label'' and discovering why rogue child processes are being left around. Project: PRFW - Hooks within the FreeBSD kernel Contact: Evan Sarmiento PRFW is a set of hooks which I have integrated into the FreeBSD kernel. This allows modules to easily intercept system calls with less overhead. It also supports per-pid restrictions, which means, one process may not be able to use X function in Y manner, but another process may. Progress: I was working on this in 4.3-RELEASE, but now I'm merging it into current. I will be submitting a patch to the mailing lists in about a week. Project: SCSI Tape Support Contact: mjacob@feral.com This driver is currently not working well under -current and is undergoing some work at this time. No major design or feature changes are planned. There was some notion of adding TapeAlert support, but HP supports that as a binary product via a user library and it was felt that it'd be more politically prudent to leave it alone. Project: SMPng Contact: Peter Wemm , John Baldwin Development: In the 'smpng' p4 branch there is code to make the ast() function loop to close the race when an AST is triggered while we are handling previously triggered AST's. In the 'jhb_preemption' p4 branch work is being done to make the kernel fully preemptive. It is reportedly stable on UP x86, but SMP x86 locks up, UP alpha has problems during shutdown and can recurse indefinitely until it exhausts its stack. Management: We are using a perforce repository for live development work, which can track multiple separate long-lived works-in-progress and collaborate between multiple developers at the same time on the same change set. FreeBSD-current is being imported into p4 hourly, for easy tracking of the moving -current tree. I haven't written up a good primer yet, but we're able to open this up to the general developer community. NEWCARD work looks like it will be done here too. Perforce is ideal for tracking this sort of long-lived project without having to resort to passing patches around. KSE work is now being checked into a kse p4 branch - thanks Julian! KSE work is focusing on getting the main API changes into the base tree well before 5.0. Project: SMPng mbuf allocator URL: http://people.freebsd.org/~bmilekic/code/mb_slab/ Contact: Bosko Milekic mb_alloc is a specialized allocator for mbufs and mbuf clusters. It offers various important advantages over the old mbuf allocator, particularly for MP machines. Additionally, it is designed with the possibility of important future enhancements in mind. The mb_alloc code has been committed to -CURRENT a month ago and appears to be holding up well. Prior to committing it, preliminary performance measurements were done merely to ensure that it is not significantly worse than the old allocator, even with Giant still in place. Results were promising [http://people.freebsd.org/~bmilekic/code/mb_alloc/results.html] - also see jlemon's results (link at the bottom of accompanying text). Since the commit, Matt Jacob has provided useful feedback and bugfixes. Work is now being done to re-enable mbtypes statistics and make appropriate changes to netstat(1) and systat(1). Project: sparc64 port Contact: Jake Burkholder The sparc64 port has been committed to the FreeBSD repository. As such further development will occur in cvs, rather than as a separately maintained patch set. Significant progress has been made since the last status report, including; support for kernel debugging with ddb, much more complete pmap support, support for context switching and process creation, and filling out of important machine dependent data structures. Thomas Moestl has shown a strong interest in working on the port and is in the process of implementing support for saving and restoring a process's floating point context. I look forward to working with him and any other developers that happen to fall out of the wood works. Project: FreeBSD/sparc64 kernel loader Contact: Robert Drehmel The sparc64 loader is functional enough to boot an ELF binary from an UFS filesystem using the existent openfirmware library, which has been revised to work flawlessly on 32-bit and 64-bit architectures. Support for netbooting and modules will be implemented next, followed by a better openfirmware mapping strategy. Project: SYN cache implemetation for FreeBSD Contact: Jonathan Lemon This project brings a SYN cache implementation to FreeBSD, in order to make it more robust to DoS attacks. A SYN cookie approach was considered, but ultimately rejected because it does not conform to the TCP protocol. The SYN cache will work with T/TCP, IPV6 and IPSEC, and the size of each cache element is currently is less than 1/5th the size of a normal TCP control block. Project: TrustedBSD Project URL: http://www.TrustedBSD.org/ Contact: Robert Watson It's been a busy month, with a number of relevant news items. Not least important is that NAI Labs was awarded a $1.2M contract from the US Defense Advanced Research Projects Agency (DARPA) to work on a variety of components relevant to the TrustedBSD Project, including support for pluggable security models, and supporting features such as improving the extended attributes implementation, simple crypto support for swap and file systems, documentation, and much more. On the features side, progress continues on Mandatory Access Control, object labeling, and improving the consistency of kernel access control mechanisms--in particular, with regard to inter-process authorization and credential management. Work has begun on porting LOMAC, NAI Labs' Low-Watermark Mandatory Access Control scheme, from Linux to FreeBSD, and it has been re-licensed under a BSD license. We hope to have an initial port complete in time for 5.0-RELEASE later this year.