Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 18 May 2012 15:08:01 -0500
From: "jfoug" <jfoug@....net>
To: <john-users@...ts.openwall.com>
Subject: RE: Performance Considerations of stdin

>From: magnum [mailto:john.magnum@...hmail.com]
>> Depends upon the OS.  If it is winblows, then the performance
>>is horrible.
>> I get about 30% or so performance for fast hashes, vs a dictionary
>> read into memory
>
>Isn't this Windows problem mitigated a little (or a lot) if using --pipe
>instead of --stdin? If not, I suppose we could make it to.

--pipe and --stdin on winblows both suck. It is the underlying OS level
implementation of pipes under windows. They are very slow. They have tiny
pipe_buffer, and input and output get into lock-step-syncronization between
them. The pipe_buffer is only 1k (or 4k??)   Even if you create a pipe in
code, using _popen(), or _pipe() or even CreatePipe() and set larger buffers
(64k is max), you still end up going through the pipe code in the kernel or
command processor.

I have thought of making some specialty code for the windows build only,
that was like the pipe (bulk reads), but used a couple of large blocks of
shared memory.  I am pretty sure I could get the speeds back up to the
incremental rate, but it would be such unique code, that it may not be super
useful to many, thus I have not taken the time to do it.

I just wish the 'standard' pipe worked efficiently under windows, for moving
large amounts of data between applications, but it does not.

Jim.

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.