A computer operating system is a bunch of programs that help a user and a developer exploit the capabilities of a computer. The features of an operating system are determined by the historical development, and what has proven to be useful for users and developers.

Among the programs normally found useful for users and developers, there can be a dispute which should be included in the operating system, and which are simply applications or utilities. Operating systems are organized in levels, and the boundary of the operating system can be described by one of these boundaries.

The most crucial software comprising an operating system is collected into a chuck of software called the kernel. It is bounded below by hardware, and above by a hardware-enforced change of privilege, called the system trap, system call, or syscall.

Kernels have their own theories and architectures. A machine with just a kernel is very uninteresting. Of all the programs that get delivered with, say, the Microsoft operating system, or Apple’s or with Linux, the kernel is just one program (not counting extensions, usually found collected in an extensions directory, such as the kext directory).

According to Richard Stallman:

An operating system, as we use the term, means a collection of programs that are sufficient to use the computer to do a wide variety of jobs. A general purpose operating system, to be complete, ought to handle all the jobs that many users may want to do.

The kernel is one of the programs in an operating system – the program that allocates the machine’s resources to the other programs that are running. The kernel also takes care of starting and stopping other programs.

This definition seems to be reasonable. However, compare that to some other definitions:

• An operating system is software that manages the computer hardware … [it provides] an environment in which a user can execute programs in a convenient and efficient manner. —Silbershatz, Galvin, Gagne, Operating System Concepts.
• An operating system must fulfill two main objectives:
  1. Interact with the hardware components servicing all low-level programmable elements included in the hardware platform.
  2. Provide an execution environment to all applications that run on the computer system (the so-called user programs).

  —Bovet, Cesati, Understanding the Linux Kernel

• What is an operating system?
  1. a piece of software that turns the hardware into something useful
  2. a software layer between the hardware and applications
  3. provides for the three main functions: to fault isolate applications, to abstract hardware, and to manage hardware

  —Frans Kaashoek

One of these mentions the user, the other definitions do not. One enumerates specific functions, of which “fault isolation” is really only a property of a sophisticated operating system. Older systems like DOS, and operating systems running embedded in, say, a automotive braking system, might not offer this feature.

I have pictured the totality of software running on a computer in a sequence of layers. At the bottom is the hardware, the firmware, and the kernel. These work closely together, and the hardware is often not accessible except through the firmware layer. This layer will provide for fault isolation, hardware abstraction, and resource (hardware) management. To enforce these properties, these elements run in kernel mode, below a hardware enforced protection boundary.

The “greater operating system”, corresponding to Stallman’s view, includes the collection of programs running in user space above the protection boundary. These form the customary complement of utilities hackers use when working at the command line. Even these don’t touch the kernel directly, but through a library (or series of libraries) which the diagram labels libc, the C runtime library. The reference to C in the name of the library overemphasizes the context — in any operating system there should be a software entity that serves the same function as does libc.

Among these utilities, the most used is called (Unix terminology) the shell. The shell is the command line interface that interprets for the kernel the intentions of the user. As we shall see, it works by choosing and launching other utilities, reserving only a small bit of work for itself.

The work that users most want their computer to do is accomplished by applications which are very distinct from the operating system, and which try to maintain a distance from the details of the operating system. In the diagram, the lower boundary of applications is libc as well as the use of operating system utilities.

I show three classes of applications. Besides applications that value their independence from a specific operating system, so that they can run on many operating systems, there are applications which value their close connection to the operating system. There are admin tools and developer tools. The admin tools are specifically adapted to the operating system by choice, they are aids in managing the operating system. Developer tools have no choice but to be specifically adapted to the operating system. They were created to create other programs, and by necessity have detailed knowledge of the operating systems structure and requirements.

The connection between developer tools and the operating system is so tight that I have drawn them as crossing over the line between apps and operating system. For instance, a compiler, and other make tools, might not be shipped with the base operating system; but they tend to come as an additional package, released in coordination with the operating system.