NFSv4/Linux Hacking

Bugs, questions to nfsv4@linux-nfs.org. Fixes for this document to bfields@citi.umich.edu.

This is an attempt to provide pointers to the basic information necessary to start hacking the Linux NFSv4 implementation. I assume that you know C and know the basics of administering a Linux box (so I assume, for example, that you know how to build and install a new kernel). I don't assume a knowledge of kernel internals.

setting up NFSv4

There are instructions for setting it up from CITI's NFSv4 project pages. Get the very latest and play around with it. Note that you don't need two machines to test both the client and the server--just install both on one machine, and mount locally.

understanding NFSv4

The authoritative source is RFC 3530. Don't read it! It's too long. But keep a copy of it and of the RFC's it references on hand to refer to when you need to understand something specific.

The best way to watch NFSv4 at work is to run NFSv4 while watching your network with a packet sniffer. Use Wireshark: it's widely available and has up-to-date support for NFSv4. Once again, your traffic doesn't have to be going over a "real" network for this to work; if your client and server are on the same machine, just sniff the loopback interface ("lo").

Wireshark also has a companion program, tshark, with a text-only interface.

I usually adjust the Wireshark preferences to give the "Packet Details" panel the full height of the window. You may also need to set:

	Protocols->TCP->"Allow subdissector to desegment TCP streams"
	Protocols->IP ->"Reassemble fragmented IP datagrams"
	Protocols->RPC->"Desegment all RPC-over-TCP messages"
	Protocols->RPC->"Defragment all RPC-over-TCP messages"

In addition to providing filters in the capture dialog or (with the -f option) on the commandline, Wireshark also gives you some help constructing filters after-the-fact: right-click on an element in the middle pane and play around with the "prepare" and "match" menus. One additional hint: right-clicking on an element and then choosing "expand tree" recursively expands everything in that element.

reading kernel code

The best way to understand how some part of the kernel works is usually just to read the code.

The best way to get the code is to install git, then run:

	# get linus's mainline tree:
	git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
	# if you also want a few nfs-related trees:
	git remote add -f trond git://linux-nfs.org/pub/linux/nfs-2.6.git
	git remote add -f bfields git://linux-nfs.org/~bfields/linux.git
	git remote add -f bhalevy git://linux-nfs.org/~bhalevy/linux-pnfs.git
Then you can check out different versions with:
	git checkout v2.6.25
	git checkout v2.6.26-rc1
	git remote show trond    # what branches does Trond have?
	git checkout trond/devel # Checkout the tip of Trond's "devel" branch
and download new updates with:
	git fetch origin
	git fetch trond
	git fetch bfields
	git fetch bhalevy

(But note this does not affect your working directory; if you want to see what's new on some branch, you'll need to run "git checkout" again.)

"git grep" is useful for finding your way around, but you may also want to set up a good text editor integrated with a database of code cross-references. I use cscope and vim. The cscope home page has instructions on using cscope with vim and emacs, and instructions on using cscope on a large project like the kernel without waiting forever for the indices to build. This allows you to follow the flow of control easily by popping quickly from the use of a function to its definition and back.

Take notes. As an example, I keep some rough notes on the kernel. In many cases they're too rough to be of use to someone else, but they help me organize my thoughts while I'm learning something new.

It's easy to get lead astray if one attempts to understand large subsystems all at once. Instead, try to keep in mind one small goal (e.g., to fix a bug, to learn how to use a certain interface).

Robert Love's "Linux Kernel Development" gives a good overview if read side-by-side with the kernel code. "Linux Device Drivers" is also good, as is "Understanding the Linux Kernel". See also lwn.net's kernel coverage.

NFS Debugging

You can use the rpcdebug command (included in nfs-utils) to get additional debugging information dumped in your logs.

(To see the code that produces this, see include/linux/sunrpc/debug.h, include/linux/nfs_fs.h, include/linux/nfsd/debug.h, the NFSDDBG_FACILITY defines at the top of each .c file, and the dprintk()'s sprinkled throughout.

generating patches

Patches are the basic unit of communication with other kernel hackers. They should be readable by humans, not just by the patch command. To this end:

See also Andrew Morton's The Perfect Patch, and see Documentation/SubmittingPatches in your friendly local kernel tree.

Generating the patches can be done with diff -urNp if you have two clean trees to compare; things get more complicated if you mistakenly do a "make" in one of the two trees, or if you start having to deal with long series of patches.

There are a several ways people deal with this; in rough order by my preference:

Mailing Lists

It's helpful to at least skim the following mailing lists:

Miscellaneous

Use the -s option to make; this eliminates most of its output so that you can see (potentially important) compiler warnings more easily.