Adding a new system call to the linux kernel
anomit | April 6, 2009I tried this thing last semester too but I wasn’t too serious about it. I had decided to go for gentoo for obvious benefits that’d support the frequent rebuilding of the kernel. Somewhere down the line gentoo got caught in a cyclic dependency error and I forgot about the whole thing. But I am digressing.
Anyway, I built gentoo from scratch and got things working. This step by step guide is quite good to get started. Note that this is about adding system calls to the kernel, not implementing them.
The guide is a bit old though, and just one thing needs to be changed. Step #16 mentions the use of the _syscallN macro. Don’t use it. From the man page of _syscall
NAME
_syscall – invoking a system call without library support (OBSOLETE)
NOTES
Starting around kernel 2.6.18, the _syscall macros were removed from header files supplied to user space. Use syscall(2) instead.
The _syscall() macros do not produce a prototype. You may have to create one, especially for C++ users.
Instead use a function wrapper like this:
long mycall(int i, int * result)
{
return syscall(__NR_mycall, i, result);
}
i and result are the arguments I used for my syscall and quite obviously it would vary according to whatever you decide to write.
There are quite a few other guides too on this topic but they are generally old and not updated. So in any case you do need to poke around quite a bit to get things working.
Some really good reading material:
- Kernel command using Linux system calls ( uses the _syscallN macro in examples )
- Playing with the cr0 register. This is a bit advanced for my current knowledge level and I’m in the process of fully understanding how the register works. Try at your own risk.








