> You could certainly imagine some kernel extensions that take all of this useful functionality and make it available in ways other than signals, leaving just signals for things you have to deal with immediately like SIGSEGV (so you can print a nice error message before quitting), but they don't exist yet.
In the SIGCHLD case, there's a proposed CLONE_FD flag to clone which would return a file descriptor instead of a PID. This fd could be read poll'd on and read from, which is much nicer than dealing with SIGCHLD. See http://lwn.net/Articles/638613/
clonefd is a very limited solution. What we really need is the ability to open a file descriptor handle to any process. That ability solves all sorts of race conditions. Conveniently, we already have an interface to open file descriptors for processes: /proc. We just need to extend its semantics slightly.
If you get a file descriptor that refers to a child process upon its creation, then that file descriptor should behave like other file descriptors.
That means you ought to be able to transfer it to other processes via file descriptor passing (the SCM_RIGHTS ancillary message; see man unix).
The identity of a process would thus be local to its parent or to a process with which the parent has agreed to share that identity. Not only does this avoid race conditions, it also enables a completely unrelated process to reap a child which can be terrifically useful.
This is exactly the approach the Capsicum sandboxing framework (mentioned elsewhere) is taking. The goal there, though, is to eliminate globally shared identifiers as much as possible -- which makes sense for sandboxing!
Maybe I'm misunderstanding, but wouldn't opening a file descriptor to a process via /proc have the same race condition issues with process id wraparound? After all, processes in /proc are opened by process ID (the only exception I can think of is /proc/self... maybe I missed some other exceptions?)
Overall, it seems easier to avoid process ID wraparound attacks via using the full 32-bit number space for PIDs. There may be a few programs that need to be changed because they did something silly like cast pid_t to short, but I think overall most programs would work just fine. As far as I can remember, the reason for using low numbers was because people didn't want to type longer ones at the shell. Internally the kernel and libraries store everything as 32-bit, at least on Linux.
> Maybe I'm misunderstanding, but wouldn't opening a file descriptor to a process via /proc have the same race condition issues with process id wraparound?
Absolutely. But once you've opened the file descriptor, the kernel would guarantee that its corresponding process ID would remain unused until you closed the file descriptor. (For example, it could keep the process a zombie if it exits.)
This way, it's possible to write a reliable killall: walk /proc, call openpid() on each entry, and with the PID FD open, examine the process's user, command line, or whatever else, kill the process if necessary, and close the process file descriptor.
But once you've opened the file descriptor, the kernel would guarantee that its corresponding process ID would remain unused until you closed the file descriptor. (For example, it could keep the process a zombie if it exits.)
That seems like it would open you up to a trivial denial-of-service attack where some attacker just spawns a bunch of processes and never closes the /proc handles. Then you can't start any more processes because there are no more process IDs available. The only workaround is to have a larger PID space, which poses the question... why not just have a larger PID space in the first place and skip the new, non-portable API?
It works out all right on Windows, which uses exactly the approach I advocate. And you can already DoS the system in myriad ways. If you're still worried: we have ulimits for other resources. We can have a ulimit for this one too.
I agree that there are already many ways to DoS the system-- for example, the age-old fork bomb. But that is not a good reason to add more flaws. People are working on ways to fix the old flaws, such as cgroups.
I don't think a ulimit would be very effective here at preventing denial-of-service. Let's say I set it to 100... I can just have my 100 children each spawn and hold on to 100 children of their own, and so on and so forth. If I just go with a bigger process ID space all these headaches go away, plus existing software works without modification.
32 bits is still too small. I wouldn't be comfortable relying on the size of the PID space to avoid collisions until we made it 128 bits or so. I think you're still seriously overestimating the danger of a DoS here: whatever limits apply to forked processes can apply to process handles. Whatever mitigates fork bombs will also mitigate handle-based attacks.
The advantages of process handles outweigh this small risk.
In the SIGCHLD case, there's a proposed CLONE_FD flag to clone which would return a file descriptor instead of a PID. This fd could be read poll'd on and read from, which is much nicer than dealing with SIGCHLD. See http://lwn.net/Articles/638613/
So those kernel extensions are happening :)