Announcements
- Textbooks:
- The Design and Implementation of the FreeBSD Operating System, by McKusick and Neville-Neil
(Amazon)
- Linux Kernel Development (3rd Edition), by Love
(Amazon)
- Windows Internals, by Russinovich, Solomon and Ionescu
(Amazon)
- C: A reference manual, by Harbison and Steele
(Amazon)
- Computers:
- We will be working in various modes. I recommend strongly that
you have your own computer to work on. We will install Virtual Box and work on virtual
machines hosted by Virtual Box.
- We will also explore cloud computing using Amazon Web Services.
- If you do not have a computer of sufficient power to work comfortably with
Virtual Box, please we will discuss options.
- We will make extensive use of subversion, a source code control system.
You will turn in your assignments using subversion, and if you use subversion,
I and the TA can help you by examining and correcting your code, at any
time and from any location, when we are on-line.
- Work:
- In this course we will write essays, enter into classroom
discussion and do a lot of
Hacking.
- For a syallabus, or near equivalent, see last
year's offering of the course, or the many
previous year's
offerings.
- Lateness: One week grace automatic; homeworks are generally scored
on a 0 to 5 scale; over one week late, one point off for each week late.
- Last day of class is Friday, December 2. I expect all work to be done by
that date.
- Contact:
- The "lab instructor" is Nagin Arhami, n.arhami _at_ cs _dot_ miami _dot_ edu.
Her lab times are posted on her blog.
She is available to help with any topic in the course, but most likely
you will seek her aid for the practical problems that arise with Linux,
and with your C language problems.
- Office hours: I am available Friday, 1:30-3:30, and by appointment.
- New media:
- There is a CSC521 topic on my blog.
It includes both my posts and as a sub-category
the grader's posts.
- You follow twitter.com/cs521. I
will keep it super-low traffic for the courtesy of those who want to enable SMS.
- Writing credit:
- Writing credit is offered. The student submits
three essays of length 1500 words each (not counting references section,
if any) on a subject related to computer
operating systems, technical, historical, or even market-business.
-
New: Recommended:
- Grants/Sponsorship:
-
Thanks to Amazon: for a grant under their AWS
for Education program, to explore cloud computing, and integrate cloud computing
concepts into the course.
Class notes
- Introduction
- What is an operating
system? (blog entry)
- Unix:
- Microsoft
- DOS, Windows for Workgroups
- Little windows
- Windows NT/VMS
- OS X
- Xerox
- Mach, Darwin and XNU
- Kernels
- Processes
- General remarks: (blog entry)
- Scheduling (blog entry)
- Other dispatched entities:
- Interrupts and exceptions
- APC/DPC, (Windows)
- Tasklets, Work queues (Linux)
- System class (Solaris)
- Soft interrupts (FreeBSD)
- Work loops (OS X)
- Priority inversion, priority boosting
- Synchronization
- Class notes: Introduction (blog entry)
- Class notes: Algorithms (blog entry)
- Class notes: Semaphores and Monitors (blog entry)
- Concepts:
- test and set, spin locks
- semaphores, mutex, monitors.
- deadlock
- References:
- Signals, time and timers.
- Interprocess Communication
- Code:
- Memory
- Virtual memory system
- Memory management
- Buddy System
- Slab
- heap storage, e.g. malloc
- Linking and loading
- 64 bits and PAE
- References
- File systems
- Properties and varieties
- Nameing, folders, case sensitive, embedded semantics in names.
- Sharing and locking (op-locks in CIFS)
- streams, resource fork (internal structures)
- Types (resource, extensions, magic numbers)
- ownership and protections
- Stream, block, special file systems such as /dev, /proc and /tmp
- remote file services
- mounting, non-mounting
- Features: ACL's, Journals, roll-backs and snapshots
- Fragments, extents, disk performance, I/O block schedulers
- Disk technology
- Examples:
- Journaling file systems
- Log-structured file systems
- Security and Protection
- Graphical Systems
- Cloud computing and virtual machines
Pop quizes!
- Pop quiz 1:
svn update to get class/quiz/quiz1.txt; Copy it to
[your repo name]/quiz/quiz1.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Sept 14.
- Pop quiz 2:
svn update to get class/quiz/quiz2.txt; Copy it to
[your repo name]/quiz/quiz2.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Sept 21.
- Pop quiz 3:
svn update to get class/quiz/quiz3.txt; Copy it to
[your repo name]/quiz/quiz3.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Sept 28.
- Pop quiz 4:
svn update to get class/quiz/quiz4.txt; Copy it to
[your repo name]/quiz/quiz4.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Oct 5.
- Pop quiz 5:
This week your pop quiz is a C program.
svn update to get class/quiz/quiz5.c.
Copy it to
[your repo name]/quiz/quiz5.c and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Oct 12.
- Pop quiz 6:
Pop quiz 6 is at class/quiz/quiz6.txt. It contains an
extra credit part.
Copy it to
[your repo name]/quiz/quiz6.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Friday, Oct 21.
- Pop quiz 7:
Pop quiz 7 is at class/quiz/quiz7.txt. It contains an
extra credit part.
Copy it to
[your repo name]/quiz/quiz7.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Oct 26.
- Pop quiz 8:
This week your pop quiz is a C program.
svn update to get class/quiz/quiz8.c.
Copy it to
[your repo name]/quiz/quiz8.c and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Nov 2.
- Pop quiz 9:
Pop quiz 9 is at class/quiz/quiz9.txt.
Copy it to
[your repo name]/quiz/quiz9.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Nov 9.
- Pop quiz 10:
Pop quiz 10 is at class/quiz/quiz10.txt.
Copy it to
[your repo name]/quiz/quiz10.txt and edit it to add
your answers; commit it by class time of the due date.
Due: Wednesday, Nov 23.
Assignments
- Assignment 1:
Due:
Monday, Sept 5. Wednesday, Sept 7.
- Assignment 2:
- Install Virtual Box. Install Ubuntu Linux 10.04 in a Virtual Box image.
Upate the software.
- Write the program helloworld.c, and the Makefile which compiles the
program to the executable helloworld.
Submit by syncing your work directory to your subversion respository,
under directory proj2.
- The helloworld.c program must have your name
and date in the comments.
- The Makefile must have three targets. The default
target, named all, compiles (cc) helloworld.c to helloword (use the
-o option of cc). The target "clean" removes all buildable files (i.e.
the executable). The target "go" runs the program.
- The
grader should be able to sync to your repository, type "make clean",
"make"
and "make go" and the result should be "Hello World!".
- See the "class" repository for further guidance and
the subversion notes.
Due: Monday, Sept 12.
- Assignment 3a:
- Read chapters 1 and 2 from Love.
- Read chapters 1 and 2 from Windows Internals.
- Rebuild the kernel of the operating system you installed in Assignment 1.
- Here is a how to build a kernel
page.
Due: Monday, Sept 19.
- Assignment 3b:
- Read chapter 5 from Love.
- Read chapters 1, 2 and 3 from McKusick and Neville-Neil.
- Insert the line printk(KERN_DEBUG "setpriority: entered"); at
a reasonable place in kernel/sys.c and rebuild the kernel.
- Reboot and verify operation of the patch.
- Submit the following in a directory proj3b (see
class/proj3b/Makefile):
- a diff file of the changed kernel file,
- your user file that tests the patch, with Makefile
- an edited piece of the log files (
/var/log/debug possibly)
showing the operation of the patch.
Due: Monday, Sept 26. Note: same due date as 4a.
- Assignment 4a:
- Read chapters 3 and 4 from Love.
- In project 4b you will create a new syscall called "notnice".
It is the same as "nice" (see man nice) but it allows you to
raise your running priority as well as lower it. In this
project you will just get set up for that.
- Alter the syscall_table, at entry 35 add a call to
sys_notnice.
- Alter the sched.c file in ./kernel, adding the function
sys_notnice. Follow the pattern of sys_nice.
- For the moment, the function body of sys_notnice
only logs its entry using the call
printk(KERN_DEBUG "notnice: entered");
- Write a user program that invokes your new system call.
- Submit the following in a directory proj4a:
- diff files for the kernel patch,
- user file that tests the call, with Makefile
- and an edited piece of a script file showing the operation.
Due: Monday, Sept 26. Note: same due date as 3b.
- Assignment 4b:
- Read chapters 1 through 8 from Love.
- Read chapters 1 through 5 from Windows NT 5-th edition.
- Read chapters 1 throgh 4 from FreeBSD 5.2 edition.
- Implement a new system call "notnice", same as nice but it lets
you raise your priority, as well of lower it. (Hint: steal code.)
Due: Wednesday, Oct 12.
- Assignment 5:
- Check that you have read all the assigned chapters in
Love; in Russinovich; and in McKusick.
- Read the first 3 chapters from the Little Book of
Semaphores.
- Implement a monitor in the Linux kernel.
- Use your monitors to implement a solution to Producers-Consumers problem,
and the Dining Philosophers problem.
- A further discussion of the project is found
here....
Due: Wednesday, October 26.
- Assignment 6:
- Create mymalloc and myfree.
- Compile this as a library (.a); test it as a library.
- Compile this as a shared library (.so); test it as a shared library.
- See
class/proj6 for starters, and also
yolinux tutorials
- How to prove it works: Use nm to check on symbols; use ldd to check
on library dependences; to check that the coalesce works, you might need
to write a special call that describes the free list.
Due: Wednesday, November 9.
- Writing credit
- Submit the first of your papers by Friday, Nov 11, by commiting
to [reponame]/papers/.
This is a firm deadline.
- Assignment 7:
- Read Love, chapters on Virtual File System and Block I/O.
- Read the FAT specifications: FAT
Functional Specs
- Read about FUSE:
- Fuse on sourceforge.
- IBM
Article concerning FUSE. Warning: all documentation about FUSE is wrong.
- FUSE
documentation from another source. Possibly no worse than any other source of misinformation.
- Harvey Mudd
- See class/proj7 for a head start on this, and the hello.c test file.
- Implement FAT-32 on FUSE.
- New hints
- Due: Wednesday Nov 30.
References
- Optional reading: Understanding the Linux Kernel, Bovet and Cesati.
- Inside the Mac OS X Chaos club
- Linux Boot
- Operating
Systems (wikipedia)
- Improvements
from Linux 2.4 to 2.6, at IBM Developerworks.
- LinuxChix:
hacking for grrls (sic).
- VBoxManage manual
- GDB manual
- Make manual
- Historical Perspective
- Operating system design philosopies and software engineering
- Schedulers
- Linking and loading
- Virtual Memory
- File systems
- EXT3, Journaling Filesystem by Dr. S. Tweedie
- ext3 Journaling File Sytems (ppt) Chadd Williams
- Linux NTFS
- FAT 16, visually, clark@hushmail
- Journaling the Linux ext2fs Filesystem, Stephen Tweedie
- How NTFS Works Microsoft Technet.
- Implementing CIFS by
Christopher Hertel
- CIFS by
Paul Leach and Dan Perry at MSDN.
- draft-leach-cifs-v1-spec-01.txt by Paul Leach. The standard before the Samba people reversed engineered it.
- Op Locks in SMB from MSDN.
- Common Internet File Systems Technical Reference from Storage
Networking Industry Association.
- Understanding NFS Michael Lucas at O'Reilly books.
- 8.3 Filename, and LFN to SFN algorithm, and directory layout (as of 15 Oct 08) Wikipedia.
- Design and Implementation of the Sun Network Filesystem, Sandberg et at.
- Use of NFS Considered Harmful, Shane Kerr.
- FreeBSD fs technologies:
softupdates, snapshots, and comparision to logging.
- Protection
- Graphical systems
- Cloud computing and virtual machines