[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150315181448.GA22598@openwall.com>
Date: Sun, 15 Mar 2015 21:14:48 +0300
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: [GSoC] building JtR for MIC
On Sun, Mar 15, 2015 at 01:12:48PM +0800, Lei Zhang wrote:
> I fixed the configuration problem and descrypt behaves normally now.
>
> And after deleting some files, we now have over 6GB free memory space on our MIC card. Thanks to that, scrypt can run now.
Great.
> This time john-jumbo ran through a bunch of benchmarks, some of which FAILED although.
There are probably bugs for you to find and fix for those. While this
won't matter much for actually running them on MIC - this is scalar code
currently, and you'd need to switch it to MIC intrinsics later - it may
matter for other uses of the same code, on other platforms. We don't
want bugs sitting in there, waiting to bite someone else.
One way to focus your search for bugs is to make and run a non-OpenMP
build as well. Then see which formats fail self-test in both builds vs.
in just one (and which one). While this won't provide conclusive
results, it may provide clues and let you focus your search.
> It got killed again when running smchapv2-naive. I guess it ran out of memory again?
Perhaps. Can you try running the mschapv2-naive self-test on its own,
not along with the rest? Something like:
./john --test --format=mschapv2-naive
If it runs fine, then the problem was likely caused by memory allocation
by the formats that were tested previously. Some of them intentionally
allocate memory such that it can't be freed. This lowers overhead for
when john is run for cracking, with just one specific format, but it
results in increased total memory usage for a many-formats self-test.
We should only be making such non-freed allocations for things that are
small. For larger allocations, formats should free the memory in their
done() methods. Perhaps some formats allocate "too much" memory (at
least when the number of threads is this large) and don't free it - and
perhaps you should identify them and change them to free the memory.
You can watch memory usage as the self-tests are running to identify
roughly when (at which format) the memory usage (or the remaining free
memory) changes rapidly. You may use e.g.:
[user@...er-mic0 user]$ while :; do free | grep '^-/+ buffers'; sleep 1; done
-/+ buffers: 449172 7433144
-/+ buffers: 449080 7433236
-/+ buffers: 448780 7433536
> The output is a bit lengthy this time. So I put them in a text file attached below. Please see for yourself.
Yes, thanks!
This also shows that django-scrypt is somehow much slower than the main
scrypt format. That's unexpected. But we need to merged them anyway.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
