Re: The rationale for IOCP
- From: "David Schwartz" <davids@xxxxxxxxxxxxx>
- Date: Tue, 1 Nov 2005 20:29:31 -0800
"chris noonan" <usenet@xxxxxxxxxxxxxx> wrote in message
news:1130894899.746345.155900@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> David Schwartz wrote:
>> > Obviously that opens the possibility for an application
>> > to conduct a denial of service attack against other
>> > applications on the machine, by throwing off a string
>> > of asynchronous I/O's with large buffers.
>> That's absurd. Without IOCP, a program can do precisely the same
>> thing
>> by opening a large number of TCP connections (to itself or other
>> cooperating
>> machines/processes) and filling the kernel's send/receive buffers and
>> keeping them that way. Every mechanism that can be used in the former
>> case
>> can be used in the latter and vice versa.
> A different mechanism I think, since monopolising kernel
> I/O buffers (if it's possible) won't affect an application
> that doesn't perform I/O.
What? Running the kernel out of kernel memory will affect everything.
And if you argue that the kernel uses a separate pool for I/O memory, I'll
say that it would probably count IOCP memory against that same pool.
> Operating Systems tend to place
> limits an the number of open files each process may have,
> presumably to stop a process hogging RAM. I agree that
> there are many ways for an application to shaft other
> applications, like starting up 1000 threads.
Right. So this is a totally bogus argument. At best, it shows that OSes
that implement IOCP have to limit the amount of memory a process can pin.
Well, that's not really surprising.
>> In any event, you changed the subject. We were not talking about
>> user-space DMA but about saving a copy. Without IOCP, a write to kernel
>> buffers is unlikely to result in the data being copied into the kernel
>> memory it will be DMA'd out of anyway.
> It sounded like you were saying that IOCP performs user-space
> DMA and therefore saves a copy operation on the data:
No. I'm saying that IOCP likely saves a copy.
For read, it's because the kernel knows where the user wants the data at
the time the data is received. Likely it won't be DMA'd to that spot because
the kernel doesn't know it's even for that process at that time. But most
OSes likely won't read it into the TCP stream buffer either. So there will
be a copy after it's DMAd. In the IOCP case, that will likely be right to
where the application wants it. In the non-IOCP case, that will be a kernel
TCP buffer where it will be later copied to the application.
For write, it's because the kernel knows where the user put the data at
the time it's ready to sent the data to the TCP stack. This means it doesn't
have to be copied into a TCP write buffer. (This only applies in the case
where there's already data in the TCP send buffer and the window isn't full.
In this case, both implementations can save the copy.)
>>It [IOCP] has several other benefits. For one thing, it allows you to
>>write to
>>a TCP connection directly out of application buffers. This eliminates a
>>copy
>>for sure and often eliminates alloc/free cycles. It can also reduce
>>kernel/user transitions. The same happens on read.
>> Let's look at the receive case. With IOCP, at the time the data is
>> received on the wire, the application's memory buffer can already be
>> locked
>> in physical memory and we know where the application wants the data.
>> Without
>> IOCP, we have no idea where the application even wants the data when we
>> receive it over the wire.
> Whether the system knows where an application wants to put
> data that is being pushed to it along the wire would seem
> to depend only on whether that application has already made
> its read call. Of course with asynchronous I/O a thread has
> the option of lodging additional read calls, from the same or
> different sources. Is that what you were getting at?
Some asynchronous I/O models make possible the same type of
optimizations that IOCP does. Some don't. Specifically, asynchronous I/O
models that pin application memory (or otherwise arrange for the kernel to
be able to access it) allows these types of optimizations. Asynchronous I/O
that's really blocking I/O under the hood gets you a subset of these
optimizations, but not all of them.
>> Without IOCP, or a similar mechanism, it is impossible to DMA out of
>> user space. The user-space memory may be changed in-between when the
>> 'write'
>> queued the data and when the data was sent over the network connection.
>> With
>> IOCP, DMA out of user space is possible, but even if it's not implement,
>> it
>> is generally possible to save a copy anyway.
> To make my reputation at Microsoft, I'm going to develop
> IOCP2. I'll start with the existing IOCP kernel code. I'll
> make only the following additions:
> 1. Release all threads in the pool and keep them released.
That will result in harmful, spurious pre-emption.
> 2. Upon an I/O call, mark the calling thread as non-runnable
> and invoke the scheduler.
That's a disaster. The whole point of IOCP is to pend a large number of
I/Os very rapidly. I may have one event that requires me to start sending on
100 sockets. The last thing I want to do is wait for each I/O to copy.
> 3. Upon an I/O completion (i.e as part of the interrupt
> handling) mark the thread that initiated the I/O as runnable
> and invoke the scheduler.
Why the thread that initiated the I/O? That makes no sense. How is that
thread somehow better than the others that may already be running?
> 4. Upon a call of GetQueuedCompletionStatus(), hand the calling
> thread back the job identifier it gave on its last I/O call.
Makes no sense. The calling thread may not have ever pended any I/Os.
> Obviously, IOCP2 is simply a rediscovery of TCP; each thread
> handles its own job, all I/O calls block.
> Equally obviously, IOCP2 shows that there is a TCP implementation
> that has exactly the same buffering and copying behaviour as
> IOCP.
What?! IOCP2 doesn't provide any of the functionality or guarantees that
IOCP does. It wouldn't conform to the current interface. At best, you've
shown that it's possible to create a broken, non-conforming implementation
of IOCP that wastes the benefits IOCP could give.
DS
.
- References:
- Re: The rationale for IOCP
- From: David Schwartz
- Re: The rationale for IOCP
- From: chris noonan
- Re: The rationale for IOCP
- Prev by Date: Re: Nanosleep and Pthread
- Next by Date: Re: How to avoid DCLP?
- Previous by thread: Re: The rationale for IOCP
- Next by thread: Re: The rationale for IOCP
- Index(es):
Relevant Pages
|
