[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150513182418.GA19396@openwall.com>
Date: Wed, 13 May 2015 21:24:18 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: get_binary_*() and get_hash_*() methods
Solar,
On Wed, May 13, 2015 at 07:53:30PM +0300, Solar Designer wrote:
> Aleksey,
>
> On Wed, May 13, 2015 at 07:37:21PM +0300, Aleksey Cherepanov wrote:
> > On Thu, May 07, 2015 at 05:39:45PM +0300, Solar Designer wrote:
> > > You can see a dirty hack like this here:
> > >
> > > git show 9a6f4f6f69903763e664f03d2adee97486eca9de DES_bs_b.c
> >
> > I tried to perform lookup in crypt_all() to exit earlier not finishing
> > the computations.
>
> The same(?) can be achieved by exiting crypt_all() early all the time
> and completing the computation in cmp_one() when needed. IIRC, myrice's
> raw SHA-512 on GPU formats did that, completing the computation in
> cmp_one() on CPU only when needed.
Not exactly the same: I build the check into the algo so I don't need
global vars to complete computation. Though cmp_one should be called
so rarely that computation of the whole hash from key should be ok.
> > I tried it against raw-sha512, scalar implementation
> > with 2 instructions reversed (endianity altering and addition of the
> > state), I compared h variable (it should be the closest result). Also
> > I replaced salt->index(index) with var & 0xfff (salt->hash_size == 2).
>
> OK, but raw SHA-512 on one core isn't slow enough for this to matter.
"slow enough"? Did you mean "fast enough" as I can get from the text
below?
> These tricks are for faster hashes or/and for use with parallelization,
> to keep the checks parallelized as well and performed in the same
> threads that computed the corresponding hashes (thus, to avoid data
> transfers between different cores).
>
> > salt argument is 0 in self tests, so I tested it with
> > --skip-self-tests:
> > 5g 0:00:01:27 3/3 0.05721g/s 1351Kp/s 1351Kc/s 20274KC/s gok3sb
> > 5g 0:00:01:27 3/3 0.05747g/s 1295Kp/s 1295Kc/s 19426KC/s 159518*
> > #1 is with early exit, #2 without that additional check.
> >
> > The difference (3%) may be a fluctuation.
>
> Yes, may be just a fluctuation, or may be for real, but either way it's
> quite small.
>
> > Is there any solution to not skip self tests?
>
> Yes. crypt_all() tells the caller whether there _might_ be matches.
> You simply return that, yes, there _might_ be matches every time you're
> called with parameters that you have no optimized code for. For
> example, if you only support salt->hash_size == 2, you can return
> *count + 1 whenever you're called with salt->hash_size != 2. Of course,
> you'll do this after computing the hashes sufficiently for your
> get_hash*() and cmp_*() to work properly.
So self tests can't really check the correctness of gpu part of
crypt_all() of fast formats because gpus should check hashes on gpu
and do not return them back for checks in cmp_one().
Or at least benchmark is not accurate if the check is in crypt_all().
> > I had such problem too. Why is there 0 in binary field? Should not
> > such entries be removed? Is it due to multithreading?
>
> I'm not familiar with Sayantan's code, but "binary" may be NULL in
> "single crack" mode. See single.c:
So it is normal behaviour. Thanks!
BTW having high MAX_KEYS_PER_CRYPT, it may be interesting to construct
a small bitmap (or a similar structure) in the end of crypt_all() to
be used in cmp_all()/cmp_one() to speed up the check.
--
Regards,
Aleksey Cherepanov
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
