Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20120710121237.GA8185@openwall.com>
Date: Tue, 10 Jul 2012 16:12:37 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Get hash number in cmp_all()

myrice -

On Tue, Jul 10, 2012 at 06:22:02PM +0800, myrice wrote:
> On Tue, Jul 10, 2012 at 10:40 AM, Solar Designer <solar@...nwall.com> wrote:
> > Are you allocating space for loaded hashes per-salt or for all at once?
> > In the latter case, how does your code know which hashes to compare
> > against (for the current salt only)?
> 
> Refer to my last week report.

I'll take another look at it.  Previously, I found your description of
your plans unclear.  Perhaps I should have asked you to clarify.

> I have two arrays: ld_hash and ld_salt.
> They both have hash number size. If two hashes have same salt, the
> salt will store twice. More specific, in cmp_all(), with each binary,
> ld_hash[index] = binary and ld_salt[index] = current_salt. So I can
> check the salt after crypt_all() to determine which hashes to compare.

Sounds inconvenient and inefficient.  John already groups the hashes by
salt, and you somehow undo that grouping?

> >> 1. Reallocate the memory when hash number exceeds the pre-defined
> >> size. Is it a good idea to add mem_realloc() to memory.c?
> >
> > I am fine with that, but I don't see why you want to be allocating a
> > copy of the hashes on CPU when you need that copy on GPU.  Is this just
> > an intermediate step, before you transfer the hashes to GPU?
> 
> Yes. I don't want to memcpy to GPU with small size of a hash.

Why not, given that you only do it once per hash when you've just
started cracking them?  Well, I understand that it could slow down
startup significantly when you have millions of hashes loaded, but in
that case the increase in memory consumption (even if temporary) may be
problematic (on the other hand, many gigabytes of hashes won't fit in
GPU memory anyway).  So maybe you'd need to transfer hashes in
reasonably sized groups, then - e.g., 1000 at a time.

> >> 2. Remove "static" and declare database as extern variable in my format
> >
> > What would you be removing "static" from, exactly?
> 
> In john.c, the variable database recorded number of hashes/salt. It
> would be helpful if I can use the variable. The "static" modifier
> restrict the scope only in john.c. I don't know if it's good to expose
> it to other file.

It's OK for you to do this for your experiments.  Then we'll decide what
formats interface changes are needed to possibly avoid this.

Thanks,

Alexander

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.