Linux Syscall

by: burt rosenberg
at: university of miami
Revised:

9 september 2020, new formatting and copyright
11 September 2014
adapted from csc521 semester 101

System services

The syscall and the syscall table are the fundamental mechanism for getting system services. The kernel of the operating system exposes an API. These are the operations that can be called on the kernel. Because a program is not linked to the kernel, the call to a system service is done through pre-assigned addresses and operation codes. Also, a call to a kernel function changes the mode of the CPU, and the thread moves from user mode to kernel mode at the instant of the call.

System calls on the Intel architecture involve a special trap instruction. A trap instruction works similar to a call instruction, in that it pushes a return address on the stack and branches to a new instruction address. However, a syscall trap also:

• Changes the state of the processor from user mode to kernel, enabling wide privileges over memory and hardware;
• Branches to an address pre-stored in a protected syscall vector table. For the security of the kernel, only the kernel decides its entry points;
• Switches from the user stack which resides in user memory to a kernel stack that resides in protected kernel memory;
• Will require a special return instruction that will restore the user privilege level and the user stack when the return is executed.

Starting in linux kernel 3.3, the syscall operation code is vectored through a table at

• linux/arch/x86/syscall_32.tbl

• linux/kernel/sys.c.

In the Intel x86 architecture, the code immediately run in response to the trap is determined by a complicated thing called the GDT, the Global Descriptor Table. A pointer to this table is installed in the CPU during linux boot, and it has entries for various sorts of traps. Each entry is called a call gate which directs the change of privilege and gives the address of the function that will respond to he trap. All of the CPU context can change through a call gate. In particular, the stack is changed from the user stack to the kernel stack, and new data segments are installed that allow for full permission over the entire 4G virtual memory space.

More on traps

The syscall depends on a trap. A trap is one of several call-like instructions that perform special operations as part of the call. What they all have in common is that an event causes a call-like instruction branch, with the return address saved on the stack, with additional changes in processor mode.

The names and taxonomy of this special sorts of calls will vary from processor to processor. Intel breaks them down in interrupts and exceptions:

• Interrupts are calls that are forced by outside events, such as a full network card buffer. Interrupts can be maskable or non-maskabled, meaning they can be ignored or cannot be ignored. There is usually a facility for levels of interrupts, such that a higher level interrupt can interrupt a lower level interrupt, but interrupts of same or lower level must wait.
• Exceptions are calls that are forced by some internal event in the processor. They can be programmed, meaning the event is an actual call to the exception (these are the syscall traps) or non-programmed.

Exceptions are classified as faults, traps or aborts.

• A fault requires that the faulting instruction be retried to continue program flow.
• A trap requires that the instruction following the trap be the location to continue program flow. In general, these are the programmed exceptions, and the trap is an instruction. These are used for syscalls, but also for breakpoints in program debugging.
• An abort means that the instruction flow cannot be continued. The aborting instruction completed partially and cannot be restarted or continued.

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.