[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAEo4CeMM0qOjjs6St6Vag1-C_N38cip2sRrVxuTuvpORjoqKBw@mail.gmail.com>
Date: Fri, 8 Jun 2012 12:27:34 +0200
From: Albert Veli <albert.veli@...il.com>
To: john-users@...ts.openwall.com
Subject: Re: JtR to process the LinkedIn hash dump
I separated the hashes that starts with 5 zeroes to one file and the
"normal" ones into another. Then I ran JtR in some different ways on
both files (starting with a clean jonh.pot) and got:
cat john.pot | sed 's/^[^:]*://' | ./unique lilist.txt
Total lines read 2066198 Unique lines written 2016885
So that means there where 49313 duplicates (of the ones cracked so
far). The duplicates are in the dump both with and without zeroes.
On Fri, Jun 8, 2012 at 7:22 AM, Frank Dittrich
<frank_dittrich@...mail.com> wrote:
> On 06/08/2012 12:07 AM, jfoug wrote:
>>>> From: Brad Tilley [mailto:brad@...ystems.com]
>>>>> The patches are found here:
>>>>>
>>>>> http://openwall.info/wiki/john/patches
>>>>
>>>> Thanks. That's working well in testing with the rockyou word list.
>>>
>>> You will note that 'most' of them that you crack will have 00000 as the
>>> first bytes of the hash (if you are using the rock-u words, and nothing
>>> more than 'rules' from JtR). This shows that whomever released this,
>>> that they are using 00000 as a 'already cracked' signature.
>>
>> One thing that has been brought up, is there are lots of hashes like this:
>>
>> 0000003ced2802e237e597f6a9d14e963206d6c3
>> 122b603ced2802e237e597f6a9d14e963206d6c3
>>
>> These are broken by the word camp1985hill. Now, since this is a non-salted
>> hash, except we are allowing these to be loaded, AND viewed as the same
>> thing.
>>
>> But there are a few things 'built in' to JtR which will cause a few strange
>> side effects. The biggest, is JtR believes that the above 2 are duplicates.
>> It will list the password was found for both strings. However, only one of
>> them is written to the .pot file. JtR sets a 'dupe' flag, and does not
>> write the other.
>>
>> So, to work around this, run your cracks, then do this:
>>
>> cut -d: -f2- john.pot > pw
>> ./john -w=pw combo_not.txt
>> ./john -w=pw combo_not.txt
>> ./john -w=pw combo_not.txt
>>
>> What this does, is to re-run the already found passwords. JtR will remove
>> the found hashes from memory (exactly matched), when it loads the .pot file.
>> Then what is left, is the hash that was not cracked. I have found that you
>> may have to run JtR 2 times (not sure why, but it could be because I am
>> running multiple instances of JtR at same time), on the already cracked
>> passwords. By the 3rd (possibly 2nd) run, JtR will list that NO passwords
>> were cracked. At that time, all duplicate matching hashes are now in the
>> .pot file, and when JtR loads combo_not.txt, it will properly remove ALL
>> hashes.
>>
>> A simple format fix was NOT enough to change this logic. This duplicate
>> detection logic is deep within the core JtR code, and it is in a couple of
>> places. I will NOT be trying to fix this behavior to work with this (semi
>> busted) format. Simple note the procedure which properly builds the .pot
>> file, removing both hashes.
>
> See this old thread about dupes recognition as well:
>
> http://thread.gmane.org/gmane.comp.security.openwall.john.user/50/focus=60
>
> The logic is still the same as 7 years ago (except that jumbo got the
> --field-separator-char=C option):
>
> In cracker.c, you'll find
>
> log_guess(crk_db->options->flags & DB_LOGIN ? replogin : "?",
> dupe ? NULL : pw->source, repkey, key,
> crk_db->options->field_sep_char);
>
>
> Removing "dupe ? NULL :" will cause john to write duplicate hash
> representations to the pot file.
>
> Removing the dupes recognition logic from john should only be a
> temporary solution for cracking these LinkedIn hashes.
>
> There should be a possible permanent solution for this "special" format:
> During prepare, or during split, always initialize the first 32 bits.
>
> Then, john should see only one hash, and write hashes with '00000' to
> the pot file.
>
> The other solution is similar, but involves you new get_source.
> It should be trivial to restore the correct sha1 hex string, even if you
> loaded the one with '00000'.
> This could make the cracked LinkedIn hashes stored in john.pot valid for
> the normal raw-sha1 format.
> (Assuming that the '00000' don't cause false positives, but that should
> be a valid assumption.)
>
> Frank
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
