CS 502 & CS 3013,
Operating Systems WPI,
Fall 2007
Assigned: Monday, September 10, 2007 Due:
Monday, September 17, 2007
Introduction
This project is intended to get you started with your virtual machine and to introduce you to working with the Linux kernel. In your virtual machine, you will
· Set up your environment for building and installing the Linux kernel;
· Build and install the kernel; and
· Submit the output of the build via the web-based turnin system.
You will not be modifying the kernel during this project, only building it.
Preparation
To the start this project, you must set up and become familiar with your virtual machine and OpenSUSE Linux version 10.2. There are several options:–
1. Starting with the virtual machines distributed via DVD, unzip the files to a directory on your own PC (or Macintosh capable of running Windows programs) and use VMware Player or some other VMware product to run it. Instructions can be found here: – .doc, html.
2. Unzip a copy of the virtual machine found in the /xtra_space/CS-502 directory of the CS Department server csopt4.wpi.edu to your own directory on that server and use VMware client to access and control it, as described here:– .doc, html.
3. Install OpenSUSE Linux version 10.2 from the distribution DVD into some other virtual environment, such as Macintosh Parallels, or onto any hardware system that you can control. See the instructor if you want to do this.
Setting up your Kernel Development Directory
Kernel sources are already be installed in the directory /home/src on your virtual machine. The official place for Linux sources is /usr/src, but we have linked this to /home/src. If you list this directory, you will find at least two versions of Linux kernel: –
linux-2.6.18.2-34
linux-2.6.18.8-0.5
You will note that the link /home/src/linux points to the latest version. Be sure to note which version this is. You will also notice that when you are logged in as an ordinary user (i.e., not root), you do not have write-access to the sources. This is a good thing, because it is far too easy for even the most experienced kernel hackers to mess up the tree of kernel sources by editing it directly.
Instead, you will clone the original source directory by making a linked copy of the kernel tree in your own directory. This is a subdirectory in which every file is hard-linked back to the same file in the original kernel tree, so no extra disk space is consumed. Only the directories themselves are replicated (think about why that is). To create a clone of the kernel tree, execute the following commands in a command shell
cp –al /usr/src/linux-2.6.18.8-0.5 kernelSrc
To make changes to any files in your kernelSrc tree, it is necessary to unlink individual files and replace them. If you use EMACS or the patch program to make the changes, this is done automatically. That is, these programs make their changes to a (hidden) copy of the file, and then they move the original aside and substitute the copy in its place. This has the effect of breaking the Linux hard link and making your kernel tree point to the changed file.
Most other editors — including vi, eclipse, and the visual editors in KDE — simply try to write the changes back to the same file. Not only is this fragile (you could get a corrupted file if the system crashes in the middle of the write), but it is also a violation of file permissions when you are not running as root. Instead, whenever you want to edit a file — for example, syscall.h — first move it aside to a “backup” file, say, syscall.h~. Next, edit this backup file and save it as the original. This save has the effect of replacing the original hard link with a new (unlinked) pointer to your updated file. Finally, “delete” the backup file using the rm command. This removes the hard link in your directory, but because there is still another pointer to the file in the original directory, the file itself does not go away; the original source tree is unchanged.
This may seem like a pain in the neck now, but you will be glad in the long run for the time saved, the safety factor, and the ability to keep track of your small changes amidst the vast number of files in the kernel.
Configuring the Kernel
The method for compiling the kernel has changed from version 2.4 (as documented in pages 74-78 of the Silbershatz text) to version 2.6 (the version you have now). Among other things, it has become simpler and cleaner, and it has a graphic user interface for configuring the kernel. There are basically three steps: –
· update the configuration,
· make the kernel and the kernel modules, and
· install the kernel modules and then the kernel.
You should do the first two steps under your ordinary user identity (i.e., without root privileges). You will do the third with narrowly constrained root privileges via the sudo command.
First, you need to update the configuration. In a command shell, change to your kernel directory and execute the commands
mkdir ~/kernelDst
make O=~/kernelDst gconfig
The directory ~/kernelDst is the directory into which you will build. For this course, you must build into a separate directory, not into your kernel source tree! The reason is that it keeps out of your kernel tree all of the stuff built by the configuration step, so that when you create a patch file, the only patches that appear are the files that you actually changed yourself. Otherwise, you will end up with megabyte-sized patch files, and the instructor will be unwilling to grade your project.
The make O=~/kernelDst gconfig command will bring up a graphic window to walk you through the configuration process. It reads information from your kernel directory and then populates a set of configuration options in the graphic interface. The window looks something like this:–
(Yes, the tiny font is hard to read, both in this document and also on the screen.)
At this point, you do not have to change the configuration very much. However, one small change is important. In the left panel, click the arrow at the left of the second line labeled General setup. This expands the line to look something like the following: –
Click on the word “default” at the right of the line beginning with the words “Local version”. Edit this word by replacing it with something to identify the kernel as yours — e.g., your name and the project name. When you install the compiled kernel and reboot, the string will appear in the boot menu at start-up time to identify your kernel.
You don’t need to make any other changes at this time. Simply invoke Save from the File menu and exit. This writes a new .config file at the root of your ~/kernelDst tree.
In the future, if you wish to rebuild using the same configuration, you should execute
make O=~/kernelDst oldconfig
This captures the previous configuration file. You should get in the practice of doing make gconfig or make oldconfig everytime that you rebuild the kernel. Among other things, the configuration process resolves all of the dependencies among kernel files and selects those that are needed for a particular build.[1] Never, ever try to edit the .config file yourself!
Making and Installing the Kernel
You are now ready to build your kernel. To do this, simply type the command
make O=~/kernelDst > make-out.txt
to a command shell. It helps to pipe the output to a file so that you can examine it later. Also, for this project, the output is your sole project submission.
On a virtual machine running on csopt4, the make operation takes about forty minutes, depending upon what configuration options have been chosen. On a 3-GHz Windows XP system running VMware Workstation, it takes almost two hours and causes a lot of disk activity. Much of time is spend building the kernel modules — i.e., the parts of the kernel that are dynamically loaded at boot time or run time in response to particular features and devices installed in the machine. The result is a boot image file called bzImage located somewhere in your kernel tree, plus a bunch of kernel modules. (You actually don’t need to know where they are.)
Next, you have to install the newly built kernel and modules. This installation is reasonably well automated on SUSE Linux[2]. Therefore, you do not have to copy the boot image anywhere, as indicated by the textbooks. Instead, you simply type the command
sudo make O=~/kernelDst modules_install install
to your command shell. Installing the kernel and the kernel modules is the one part of the process that requires root privileges. The sudo command provides these privileges for one command only. It will ask you for the root password before executing the command.
SUSE Linux uses grub, the Grand Unified Bootloader, to manage the booting process. In principle, you do not need to edit the configuration files for grub. Last year, we needed to do some manual configuration in YaST after installing the kernel, but those problems seem to have gone away in OpenSUSE Linux 10.2.
Booting your Kernel and Controlling the Boot Configuration
After you install your new kernel, you should reboot. You will get a boot screen resembling the now-familiar OpenSUSE blue boot screen, but with one new line added to the boot choices. This line is your newly-built kernel, named as you configured it above. Click in the boot window, and then use the up- and down-arrow keys to control which kernel or system to boot. If you try booting one of them and it fails, you can recover by booting from another.
When you have successfully booted and logged in, you can find out which kernel you are running by typing the following command in a shell:–
uname -a
This will print out the actual identification of the kernel, as built. It contains the string from the “Local version” field of the configuration process that you set above.
The booting configuration information is stored in the text file /boot/grub/menu.lst. There are many ways and a number of tools for managing this configuration file; for this course, we will use YaST. After you invoke YaST, select “System” in the left panel and select “Boot Loader Configuration” in the right panel. You can then manipulate and/or delete boot files, set the linux link to the one you want to be default, etc.
A Tiny Glitch
One small problem occurred last summer and again this fall. This is that the build fails after a minute or so, because it can not find the file
~/kernelDst/include2/asm/asm-offsets.h
This file is supposed to get installed by the configuration step, but it seems not to. You can find a copy in the machine specific include file, i.e.,
~/kernelDst/include/asm-i386, or
~/kernelDst/include/asm-x86_64.
I prefer that you link the file into ~/kernelDst/include2/asm rather than copy it. The reason is that the modification date is not changed with a link, but it is changed with a copy. If you copy, it takes much longer for me or the grader to build your kernel. Link using the following shell commands.
pushd ~/kernelDst/include/asm-i386 /* or asm-x86_64 */
ln asm-offsets.h ~/kernelDst/include2/asm
popd
After you have linked the file, you must restart the make command, but without re-running the configuration step.
Using more than one processor
The make system for the Linux kernel is designed to support builds by more than one processor. This can be invoked by using the -j switch to specify the number of parallel build jobs – for example,
make –j4 O=~/kernelDst
The SUSE Linux documentation suggests that two jobs per processor is about right. Your virtual machine has two processors by default, but if you are running on a Pentium with only one processor, then –j2 is probably the right choice. The departmental server has two dual-core processors (four in all), and they seem to support a two virtual processors in your virtual machine.
Submitting Part 1
Be sure to put your
name at the top of every file you submit and/or at the top of every file that
you edit!
Submit your assignment for grading as directed in class. We will use the web-based Turnin tool developed by Professor Fisler’s group. A brief introduction can be found at
http://web.cs.wpi.edu/~kfisler/turnin.html
and access to the turnin system itself can be found at
http://turnin.cs.wpi.edu:8088/servlets/turnin.ss
For purposes of Turnin, this assignment is Project-0.
Since one of the goals of this project is to debug the submission process, you need only submit the output of your kernel build (or a portion of it).
Do not put your submitted files into a folder or attempt to zip them to save space. The Turnin system does this automatically, behind your back.
Individual Assignment
This is an individual project, not a team project. Each student should submit his/her own work, not copies of jointly developed code.
Nevertheless, if you are puzzled or unsure of some aspect of this assignment, you should consult your friends, classmates, or the instructor to help clarify your understand or derive an approach to the problem. Using the class e-mail list for discussion is considered a good idea and contributes strongly to the “class participation” component of your grade.