The ongoing discussion of threadlets (or fibrils, or whatever they will be called next week) has considered the addition of a major new API to the kernel. This discussion has, however, studiously ignored an important question: what about the longstanding kevent patch which, at some level, solves the same problems? The motivation for the first fibril patch was to make it easier to provide comprehensive asynchronous I/O in the kernel - and that was one of the reasons for kevents as well. So it has been surprising that kevents have not figured into this conversation.
Kevents have finally become part of the discussion, however, resulting in an interesting exchange between kevent hacker Evgeniy Polyakov, threadlet (and everything else) hacker Ingo Molnar, and several others as well. Benchmarks have been thrown around to illustrate the performance characteristics of both approaches, but the real question is this: what is the best way to allow user-space applications to juggle multiple simultaneous operations in a scalable manner?
Evgeniy"s core claim appears to be that an event-oriented approach is inherently more scalable than using threads. He says: If things decreases performance noticeably, it is a bad things, but it is matter of taste. Anyway, kevents are very small, threads are very big, and both are the way they are exactly on purpose - threads serve for processing of any generic code, kevents are used for event waiting - IO is such an event, it does not require a lot of infrastructure to handle, it only needs some simple bits, so it can be optimized to be extremely fast, with huge infrastructure behind each IO (like in case when it is a separated thread) it can not be done effectively.
In other words, using threads for event management is simply too slow. David Miller has also argued that threads are inherently wrong for network-oriented tasks. One of the big advantages behind the threadlet approach is that it is very fast in the non-blocking case, which is expected to be the situation much of the time. In networking, however, one normally expects to block. As a result, a highly multi-threaded networking application could create massive numbers of threads in short order. Networking is inherently an event-oriented activity.
Ingo challenges the notion that using threads and the scheduler will be slower than maintaining lists of jobs which turn into events:
To me the picture is this: conceptually the scheduler runqueue is a queue of work. You get items queued upon certain events, and they can unqueue themselves. (there is also register context but that is already optimized to death by hardware) So whatever scheduling overhead we have, it"s a pure software thing...
Now look at kevents as the queueing model. It does not queue "tasks", it lets user-space queue requests in essence, in various states. But it"s still the same conceptual thing: a memory buffer with some state associated to it. Yes, it has no legacies, it has no priorities and other queueing concepts attached to it ... yet. If kevents got mainstream, it would get the same kind of pressure to grow "more advanced" event queueing and event scheduling capabilities. Prioritization would be needed, etc.
The point here is that the scheduler has been brutally optimized over the course of many years. The actual overhead of switching contexts is quite small - perhaps less than that of a system call to manage events. The only real difference is that the memory overhead of maintaining threads is quite a bit higher than the overhead of kevents. But, says Ingo, with proper programming that should not be an insurmountable problem.
The real issue, though, tends to be one of ease of programming - on both the kernel and the user sides. In user space, the classic pattern for an event-based application involves a central loop which only blocks when it is waiting for events. Any actual work done within the loop must happen in a non-blocking manner; should the loop block, events will pile up while the application is doing nothing. Blocking in the wrong place can kill performance. But avoiding blocking in all situations is tricky at best, and sometimes impossible. The threadlet model lets the application developer stop worrying about blocking; if an operation blocks, the application simply continues to run in a newly-created thread.
More generally, programs written as state machines - the style necessitated by event-driven models - tend to be hard for people to understand. And there are a number of kernel operations (opening a file, for example) which can block in any of a number of places, and which are just about impossible to code in a state-machine style. Multi-threaded programs present their own challenges for developers who are not prepared to think about concurrency issues, but they still tend to be easier for most to understand. Threadlets, by making any sequence of calls easily implementable in a threaded model, should be relatively easy to program. At least, that"s how the argument goes.
That argument applies to kernel space as well. The struggle to bring event-based asynchronous I/O to Linux has occupied a number of highly-capable kernel developers for years - and the job is still far from complete. It requires the addition of an entirely new infrastructure and the application of state-machine techniques to inherently sequential series of events. The complexity of the retry-based asynchronous buffered file I/O patch set is a case in point: this code has seen work (on and off) for years, and it still hasn"t found its way into the mainline. It still depends on worker threads for some of its operation as well. Threadlets, it is argued, allow for any system call to be invoked asynchronously, with almost no added complexity or overhead at all.
Eventually the discussion reached a point where Linus jumped in to express a bit of frustration. His position is that it"s not a matter of choosing between event-based and thread-based mechanisms, since there is a place for both:
Use select/poll/epoll/kevent/whatever for event mechanisms. STOP CLAIMING that you"d use threadlets/syslets/aio for that.... Event mechanisms are *superior* for events. But they *suck* for things that aren"t events, but are actual code execution with random places that can block.
In this view, it"s not a matter of picking one or the other, but providing both so that the right tool can be used for each job. It seems likely that this opinion is fairly widespread, meaning that some sort of thread-based asynchronous mechanism will probably find its way into the mainline before too long. Event-based interfaces will continue to be supported as well; the big question there is whether the existing interfaces (epoll in particular) are sufficient, or whether the addition of kevents is called for.
Evgeniy"s core claim appears to be that an event-oriented approach is inherently more scalable than using threads. He says: If things decreases performance noticeably, it is a bad things, but it is matter of taste. Anyway, kevents are very small, threads are very big, and both are the way they are exactly on purpose - threads serve for processing of any generic code, kevents are used for event waiting - IO is such an event, it does not require a lot of infrastructure to handle, it only needs some simple bits, so it can be optimized to be extremely fast, with huge infrastructure behind each IO (like in case when it is a separated thread) it can not be done effectively.
In other words, using threads for event management is simply too slow. David Miller has also argued that threads are inherently wrong for network-oriented tasks. One of the big advantages behind the threadlet approach is that it is very fast in the non-blocking case, which is expected to be the situation much of the time. In networking, however, one normally expects to block. As a result, a highly multi-threaded networking application could create massive numbers of threads in short order. Networking is inherently an event-oriented activity.
Ingo challenges the notion that using threads and the scheduler will be slower than maintaining lists of jobs which turn into events:
To me the picture is this: conceptually the scheduler runqueue is a queue of work. You get items queued upon certain events, and they can unqueue themselves. (there is also register context but that is already optimized to death by hardware) So whatever scheduling overhead we have, it"s a pure software thing...
Now look at kevents as the queueing model. It does not queue "tasks", it lets user-space queue requests in essence, in various states. But it"s still the same conceptual thing: a memory buffer with some state associated to it. Yes, it has no legacies, it has no priorities and other queueing concepts attached to it ... yet. If kevents got mainstream, it would get the same kind of pressure to grow "more advanced" event queueing and event scheduling capabilities. Prioritization would be needed, etc.
The point here is that the scheduler has been brutally optimized over the course of many years. The actual overhead of switching contexts is quite small - perhaps less than that of a system call to manage events. The only real difference is that the memory overhead of maintaining threads is quite a bit higher than the overhead of kevents. But, says Ingo, with proper programming that should not be an insurmountable problem.
The real issue, though, tends to be one of ease of programming - on both the kernel and the user sides. In user space, the classic pattern for an event-based application involves a central loop which only blocks when it is waiting for events. Any actual work done within the loop must happen in a non-blocking manner; should the loop block, events will pile up while the application is doing nothing. Blocking in the wrong place can kill performance. But avoiding blocking in all situations is tricky at best, and sometimes impossible. The threadlet model lets the application developer stop worrying about blocking; if an operation blocks, the application simply continues to run in a newly-created thread.
More generally, programs written as state machines - the style necessitated by event-driven models - tend to be hard for people to understand. And there are a number of kernel operations (opening a file, for example) which can block in any of a number of places, and which are just about impossible to code in a state-machine style. Multi-threaded programs present their own challenges for developers who are not prepared to think about concurrency issues, but they still tend to be easier for most to understand. Threadlets, by making any sequence of calls easily implementable in a threaded model, should be relatively easy to program. At least, that"s how the argument goes.
That argument applies to kernel space as well. The struggle to bring event-based asynchronous I/O to Linux has occupied a number of highly-capable kernel developers for years - and the job is still far from complete. It requires the addition of an entirely new infrastructure and the application of state-machine techniques to inherently sequential series of events. The complexity of the retry-based asynchronous buffered file I/O patch set is a case in point: this code has seen work (on and off) for years, and it still hasn"t found its way into the mainline. It still depends on worker threads for some of its operation as well. Threadlets, it is argued, allow for any system call to be invoked asynchronously, with almost no added complexity or overhead at all.
Eventually the discussion reached a point where Linus jumped in to express a bit of frustration. His position is that it"s not a matter of choosing between event-based and thread-based mechanisms, since there is a place for both:
Use select/poll/epoll/kevent/whatever for event mechanisms. STOP CLAIMING that you"d use threadlets/syslets/aio for that.... Event mechanisms are *superior* for events. But they *suck* for things that aren"t events, but are actual code execution with random places that can block.
In this view, it"s not a matter of picking one or the other, but providing both so that the right tool can be used for each job. It seems likely that this opinion is fairly widespread, meaning that some sort of thread-based asynchronous mechanism will probably find its way into the mainline before too long. Event-based interfaces will continue to be supported as well; the big question there is whether the existing interfaces (epoll in particular) are sufficient, or whether the addition of kevents is called for.
