CS 3013, Operating Systems WPI,
A-term 2008
Assigned: Friday, August 29, 2008 Due:
6:00 PM, Tuesday, September 2, 2008
Introduction
This project is intended to get you started with your virtual machine and to introduce you to working with the Linux kernel. In this project, you will
· Set up your virtual machine in the Fossil Lab;
· Set up your environment for building and installing the Linux kernel;
· Build and install the kernel; and
· Submit the output of the build via the web-based turnin system.
You will not modify the kernel during this project, only build and run it.
Because of the number of students registered for this course, the professor will permit students to work in two-person teams on kernel projects.
Preparation
Set up your virtual machine in the Fossil Lab as described
here:– .doc, html. Alternatively, adventurous
students may clone the virtual machine of this course and run it on their own
PCs using the freeware VMware Player.
There is plenty of information in textbooks and on-line regarding the building of the Linux kernel, much of which is contradictory! You should consult Chapter 2 of Linux Kernel Development, 2nd edition, by Robert Love. However, some of the steps in this project are simpler than what he prescribes. Eventually, you may want to consult the files README.SUSE and README in the Linux source tree. These describe more complex and more arcane procedures than needed in this course, but they are useful for professional developers of Linux.
Setting up your Kernel Development Directory
Kernel sources are already be installed in the directory /home/src on your virtual machine. The official place for Linux sources is /usr/src, but we have linked this to /home/src. If you list this directory, you will find one or more versions of Linux kernel. This course uses the latest version supported by OpenSUSE Linux 10.3 as of the start of the term, namely
linux-2.6.22.18-0.2
You will note that the link /home/src/linux points to this latest version. You will also notice that when you are logged in as an ordinary user (i.e., not root), you do not have write-access to the sources. This is a good thing, because it is far too easy for even the most experienced kernel hackers to mess up the tree of kernel sources by editing it directly.
Instead, you will clone the original source directory by making a linked copy of the kernel tree in a directory of your own choosing. This is a directory tree in which every file is hard-linked back to the same file in the original kernel tree, so no extra disk space is consumed. Only the directories themselves are replicated (think about why that is). To create a clone of the kernel source tree, execute the following command in a command shell
cp –al /usr/src/linux-2.6.22.18-0.2 kernelSrc
Here, kernelSrc is the name you have chosen for your working directory. To make changes to any files in this directory, it is necessary to unlink individual files and replace them. If you use EMACS or the patch program to make their changes, unlinking is done automatically. That is, these programs make their changes to a hidden copy of the file; when you save and exit, they move the original file aside and substitute the copy in its place. This has the effect of breaking the Linux hard link and making your kernel tree point to the changed file.
Most other editors — including vi, eclipse, kwrite, and other editors in KDE — simply try to write the changes back to the same file. Not only is this fragile (you could get a corrupted file if the system crashes in the middle of the write), but it is also a violation of file permissions when you are not running as root. Instead, whenever you want to edit a file — for example, syscall.h — first change its name to a “backup” name, say, syscall.h~ using the mv command. Next, edit this “backup” file and save it with the original name. This save has the effect of replacing the original hard link with a new reference to your updated file. Finally, “delete” the backup file using the rm command. This removes the hard link in kernelSrc, but because there is still another reference to the file in the original source code directory, the file itself does not go away; and the original source tree is unchanged.
This may seem like a pain in the neck now, but you will be glad in the long run for the time saved, the safety factor, and the ability to keep track of your small changes amidst the vast number of files in the kernel.
Configuring the Kernel
The method for compiling the kernel has changed from version 2.4 (as documented in of textbooks) to version 2.6 (the version you have now) and even from that described by Robert Love in Linux Kernel Development, 2nd ed. Among other things, it has become simpler and cleaner, and it has a graphic user interface for configuring the kernel. There are basically three steps: –
· make the configuration file (or re-make the previous one),
· make the kernel and the kernel modules, and
· install the kernel modules and then the kernel.
You will do the first two steps under your ordinary user identity — i.e., without root privileges. You will do the third step with narrowly constrained root privileges via the sudo command.
First, you need to make the configuration file. In a command shell, execute the following: –
mkdir ~/kernelDst /*
or some temporary directory */
cd kernelSrc
make O=~/kernelDst gconfig
The directory ~/kernelDst is the directory into which you will build. For this course, you must build into a separate directory, not into your kernel source tree! The reason is that it keeps out of your kernel tree all of the stuff built by the configuration step, so that when you create a patch file, the only patches that appear are the files that you actually changed yourself. Otherwise, you will end up with megabyte-sized patch files, and the graders will be unwilling to grade your project.
The make O=~/kernelDst gconfig command brings up a window to walk you through the configuration process. It reads information from your kernel directory and then populates a set of configuration options in the graphic interface. The window looks something like this:–
(Yes, the tiny font is hard to read, both in this document and also on the screen.)
At this point, you do not have to change the configuration very much. However, one small change is important. Click the arrow at the left of the second line labeled General setup. This expands the line to look something like the following: –
Double-click on the word “default” at the right of the line beginning with the words “Local version”. Edit this word by replacing it with something to identify the kernel as yours — e.g., your name and/or the project name. When you install the compiled kernel and reboot, the string will appear in the boot menu at start-up time to identify your kernel.
You don’t need to make any other changes at this time.[1] Simply invoke Save from the File menu and exit. This writes a new .config file at the root of your ~/kernelDst tree.
In the future, if you wish to rebuild using the same configuration, you should execute
make O=~/kernelDst oldconfig
This captures the previous configuration file (i.e. .config). You should get in the practice of doing make gconfig or make oldconfig every time that you rebuild the kernel. Among other things, the configuration process resolves all of the dependencies among kernel files and selects those that are needed for a particular build.[2] Never, ever try to edit the .config file yourself!
Making and Installing the Kernel
You are now ready to build your kernel. To do this, simply execute the commands
cd kernelSrc
make –j4 O=~/kernelDst > make-out.txt
in a command shell. Piping the output to a file is optional, but it enables you to examine it later. Also, for this project, the output is your sole project submission.
On a Fossil Lab machine, this takes about 30 minutes. On a 3-GHz Windows XP system running VMware Workstation, it takes more than an hour and causes a lot of disk activity. Much of time is spend building the kernel modules — i.e., the parts of the kernel that are dynamically loaded at boot time or run time in response to particular features and devices installed in the machine. The result of the build is a boot image file called bzImage located somewhere in your kernel tree, plus a bunch of kernel modules. (You actually don’t need to know where they are.)
Next, you have to install the newly built kernel and modules. This installation is reasonably well automated on SUSE Linux. Therefore, you do not have to copy the boot image anywhere, as indicated by the textbooks. Instead, you simply type the command
sudo make O=~/kernelDst modules_install install
to your command shell. Installing the kernel and the kernel modules is the one part of the process that requires root privileges. The sudo command provides these privileges for this command only. It will ask you for the root password before executing the command.
SUSE Linux uses grub, the Grand Unified Bootloader, to manage the booting process. In general, you do not need to edit the configuration files for grub.
Booting your Kernel and Controlling the Boot Configuration
After you install your new kernel, you should reboot. You will get a boot screen resembling the now-familiar OpenSUSE green boot screen, but with one new line added to the boot choices. This line identifies your newly-built kernel, named as you configured it above. Click in the boot window, and then use the up- and down-arrow keys to control which kernel or system to boot. If you try booting and it fails, you can recover by booting another option.
When you have successfully booted and logged in, you can find out which kernel you are running by typing the following command in a shell:–
uname -a
This will print out the actual identification of the kernel, as built. It contains the string from the “Local version” field of the configuration process that you set above.
The booting configuration information is stored in the text file /boot/grub/menu.lst. There are many ways and a number of tools for managing this configuration file; for this course, we will use YaST. [3] After you invoke YaST, select “System” in the left panel and then select “Boot Loader” in the right panel. This brings up the GUI interface for manipulating and/or deleting boot files, for setting the linux link to the one you want to be default, etc.
Using more than one processor
The make system for the Linux kernel is designed to support builds by more than one processor. This can be invoked by using the -j switch to specify the number of parallel build jobs – for example,
make –j4 O=~/kernelDst
The SUSE Linux documentation suggests that the right setting is approximately two jobs per processor core. Your virtual machine has two processors by default, and all of the host Fossil Lab workstations have dual-core processors, so –j4 is the right setting. However, if you are running on a Pentium at home with only one processor, then –j2 might be better.
Also, previous experience has taught us that you should limit to memory size of your virtual machine to no more than about one-half of the physical memory on your PC. Since Fossil Lab machines have 2 gigabytes of memory, your virtual machine is configured to be one gigabyte. If you are running on some other machine, you should adjust accordingly.
An annoying nuisance
You may find that when you boot with your newly built kernel, the network connection does not work. This is an artifact of VMware Tools, which are pre-installed in your virtual machine to make virtualization go a little smoother. This will make it difficult to submit the project, as described below.
For now, simply reboot to the original kernel and submit your project from there.
Submitting Part 1
Be sure to put your
name at the top of every file you submit and/or at the top of every file that
you edit!
Submit your assignment for grading as directed in class. We will use the web-based Turnin tool developed by Professor Fisler’s group. This can be found at
http://web.cs.wpi.edu/~kfisler/turnin.html
For purposes of Turnin, this assignment is Project0.
Since one of the goals of this project is to debug the submission process, you need only submit a portion of the output of your kernel build, along with the output of the command
uname -a
Do not put your submitted files into a folder or attempt to zip them to save space. The Turnin system does this automatically, behind your back.
Individual or Team project
Each team should submit one copy on behalf of both members.
If you are puzzled or unsure of some aspect of this assignment, you should consult your friends, classmates, TAs, or the instructor to help clarify your understand or derive an approach to the problem. Using the class e-mail list for discussion is considered a good idea and contributes strongly to the “class participation” component of your grade.