[<prev] [next>] [day] [month] [year] [list]
Message-Id: <20150526114540.dc30d64f61e5ec441c34ffd4f788e58e.f86e6148d2.wbe@email15.secureserver.net>
Date: Tue, 26 May 2015 11:45:40 -0700
From: <writeonce@...ipix.org>
To: john-dev@...ts.openwall.com
Cc: "Rich Felker" <dalias@...c.org>
Subject: RE: update: JtR for Windows using musl libc
On 05/26/2015 09:01 AM, writeonce@...ipix.org wrote:> On 05/26/2015
02:14 AM, Solar Designer wrote:> On Mon, May 25, 2015 at
> 10:19:17PM -0700, writeonce@...ipix.org wrote:
>>> I have now looked at x86-64.h and DES_bs.h and have added a few sysv_abi
>>> hints which did not change the outcome. My understanding is that someone
>>> familiar with JtR would have an easier time debugging this.
>>
>> Yes, but we don't have your setup ready for debugging on. And many of
>> us don't have Windows.
>>
>> Can you try building for -avx? The issue should be gone then (in that
>> build), as that disables use of the assembly code.
>>
>>> As for performance: at some point I commented out the code in times(2)
>>> which converts NT measures (units of 100ns) to clock_t (1/100 sec)...
>>> the test was run natively on i3-2100 pc, and with the fixed __sys_times
>>> the numbers now finally seem correct. As an aside, an additional factor
>>> that might slow things down a bit (but nothing like *100000) is that the
>>> entire stack (libc, runtime library, john) is built with -g3 -O0.
>>
>> Yes, the speeds look sane now, and clearly those with no assembly code
>> are impacted by -O0 a lot.
>>
>> One thing that is still unclear to me is why you have a 2x gap between
>> real and virtual times. It looks like the virtual times are doubled,
>> much like they would be when running 2 threads, but I think you are not
>> (the DES code in x86-64.S isn't even thread-safe). So I am puzzled.
>>
>> Alexander
>>
>>
>
>
> Built with -avx, john almost always terminates abnormally in the middle
> of the tripcode test. Since this behavior is inconsistent and goes away
> with debugging I suspect the problem is in the runtime library and am
> currently working to find it.
>
> Regarding the gap between real and virtual times: __sys_times had it
> wrong again, making a distinction between current thread and current
> process, rather than current process and current process and all
> children. While the multithreaded behavior you identified could have
> been related to the runtime's internal threads, the perfect x2 factor is
> too pretty to make that the sole explanation. In case that helps, the
> previous (buggy) implementation of times(2) can be found at
> http://midipix.org/scrap/__sys_times
>
> midipix
>
>
>
As suspected, the cause for the abnormal termination was in the runtime
library (or, better-said, runtime component, since john was statically
linked). Another mystery solved in the process was that of times(2): as
can be seen in its previous implementation, __sys_times erroneously
popualted tms_cutime and tms_cstime with data pertaining to the current
process, which happened to match that of the current thread (hence the
x2 factor). That is actually good news, showing that virtually no cpu
time was spent in the internal threads (daemon, timers).
For the time being, __sys_times simply sets tms_cutime and tms_cstime to
zero, which I believe is irrelevant to ./john --test (for there are no
child processes). Of the 9 tests john performs, LM and tripcode appear
to be the only ones with different real and virtual times.
Last but not least: with -avx, the only functions that require
__attribute__((sysv_abi)) are CPU_detect and CPU_detect_or_fallback
(extern declarations in john.c). Assuming the non-avx flavor is of a
lower priority, next in line should probably be the multi-threaded
version built with -fopenmp.
midipix
View attachment "john.output" of type "text/x-c++" (1105 bytes)
View attachment "john.strace" of type "text/plain" (12189 bytes)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
