[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20100812021434.GA27254@openwall.com>
Date: Thu, 12 Aug 2010 06:14:34 +0400
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: Crack Me If You Can Challenge Writeup
Matt,
Thank you for being on our team, for representing us at DEFCON, and
finally for posting this!
As you know, I am also working on a writeup for the john-users team, and
my writeup ends up being Solar-centric, which I dislike. So it is very
helpful to have this more detailed info on what you did, etc. Rather
than try to merge this into my writeup (the draft of which is arguably
too long as it is), I will likely simply refer to yours, though.
I've included some comments below:
On Wed, Aug 11, 2010 at 01:32:42AM -1000, Charles Weir wrote:
> [...] I'd also like to thank Solar for organizing all of us,
You're welcome, although I don't feel I did a very good job at that.
> and [...] devoting a sleepless weekend to this contest.
Oh, not exactly. I did treat the contest as a priority, but I managed
to do a few other things as well, including getting some sleep.
> [...] I switched to the Crypt-MD5 hashes and ran JtR's password.lst
> using the single mode ruleset, (the reason for this is I desperately
> wanted to get some sleep and figured it was a worthwhile attack to run
> through the night) . I stopped that attack at 9:20am after completing
> only 1.15% of the cracking session. At that point I had cracked 603 of
> 4716 (12.78%) of the Crypt-MD5 hashes.
That's quite good, and it might have provided some unique passwords (not
cracked by others on our team). I think I only ran the much smaller
wordlist ruleset against these hashes, then went with incremental, and
Simon was using the Markov mode. Even the initial 1% of single crack
mode rules do not fully overlap with wordlist mode rules. One thing you
could have done better was to pass the output from "john ... --stdout"
through "unique", then use the resulting file as a wordlist (to which
you would not apply further rules). The single crack mode ruleset
produces quite a few duplicates, especially when the input wordlist is
not all-lowercase (and password.lst is not), these hashes are slow
enough to compute to make the use of "unique" worthwhile, and the
resulting file would not be too large (since password.lst is small).
Besides, you could have interrupted the "john ... | unique ..." command
after a little while because JtR would not have enough time to complete
running through the resulting wordlist anyway (not against these hashes).
> On a side note, I was only
> making around 6000 guesses a second against the Crypt-MD5 hashes. I'm
> not sure if that was 6000 g/s against all of the hashes, or if I was
> only making about unique 2 g/s. Needless to say, I wasn't making much
> progress...
These hashes are slow to compute (although by far not as slow as the
Blowfish-based crypt ones). JtR reports the number of combinations of
{hash, password} it tries per second. So you need to divide this number
by the number of hashes if you want the rate of different candidate
passwords per second (tried against all hashes). Given your numbers,
you appear to have been testing less than 2 candidate passwords per
second. That's fine (for this hash type and this number of different
salts), although you could have done a bit better by using a 64-bit
build of JtR. Your 6000 c/s on a single core in a 2.2 GHz Core 2 Duo
sounds like it was for a 32-bit build; you'd get something like 9000 c/s
for a 64-bit build, which would process two candidate passwords at a
time. You could have done better yet by using Simon's
john-1.7.5-jumbo-3-md5intrinsics.diff.gz (it's on the wiki), but I
understand that it's tricky (its performance heavily depends on compiler
and its version), you didn't have time, and you had other things to
focus on for the contest. I assume that you ran something else on the
other core in your CPU (although the laptop would get really hot...)
> Before I left to attend the conference Friday morning, I launched
> another attack using the single mode ruleset and the various InsidePro
> "From Queue" dictionaries. I also created a wordlist containing all of
> the extracted lower-alpha characters from the passwords the
> john_user's team had cracked, and uploaded it to the team's server.
> Aka I created an input dictionary by extracting the words, (in a
> fairly naive manner), from the cracked passwords.
These were welcome contributions. I did not use any InsidePro wordlists
at all, so you might have cracked some unique passwords. The use of
wordlists made of other real passwords, with the major exception for the
RockYou list, was relatively ineffective against the contest hashes,
though (compared to what we could expect for real-world passwords) - due
to the way the hashes were generated (apparently, using only RockYou
passwords as real-world password material, which was understandable).
As to the extracted words, I never got around to looking at what you
uploaded, and I don't know if anyone else on our team used those...
although it would have been great if some of us did. Instead, I used my
"extract all substrings" approach at a slightly later time (I'm going to
describe it in the writeup), as well as a manually-created list of the
most common contest-specific words.
> I also created an
> input dictionary of all of the usernames, but this turned out to be
> not very useful. The Kore folks admitted later that they didn't think
> about matching passwords up with usernames until after they had
> created all of the hashes.
Right. The lack of correlation with the usernames was almost apparent
from initial single crack mode runs that we did.
The admin accounts appeared to have been treated specially, though -
often having what looked like manually-chosen passwords (not the usual
stuff generated with Kore's rulesets).
> Lessons Learned and Mistakes Made:
>
> 1) I believe the key to this contest was analyzing the cracked
> passwords and creating custom rulesets based on them. The HashCat and
> CrackHeads groups did this amazingly well,
Agreed.
> (I have no idea what InsidePro's strategy was).
InsidePro's writeup is here:
http://contest.korelogic.com/team_Insidepro.html
It's pretty short, but the strategy appears to be similar.
> 2) While JtR has the most powerful built-in rule creation system,
> based on the listserv posts, it looks like our team had a hard time
> taking advantage of it.
That's true, but I think the problem was primarily that no one truly
focused on figuring out "all" of the patterns. We did have people
capable of encoding already identified patterns. Instead, everyone
tried to keep their computers busy, which was not as important overall
(there was a lot of overlap).
> 3) From my conversations with other people at the conference, JtR's
> config files/rules are pretty much universally
> hated/feared/misunderstood. It's almost comical since many of the
> people I talked to are fluent in several programming/scripting
> languages.
I guess it's my fault, but it is difficult for me to make use of such
feedback. I am only aware of a few things to improve in this area -
such as adding a "mask mode", which would be an easier to understand
concept. I am also aware that people have difficulty distinguishing
preprocessor expressions (and individual special characters that control
the preprocessor) from rules and from individual rule commands. This
confusion kept coming up in websiteaccess' questions and requests for
new rules to be written for him, for example. But I am unaware of a
good way to deal with this. Any suggestions?
> 6) I'm really envious of team CrackHeads's use of Amazon EC2 clusters.
> I wish I had thought of that!
It might have sounded good to you when you only had a laptop, but I
think CrackHeads didn't benefit from their use of EC2 all that much.
They had "enough" of other resources anyway.
> 7) Rather than starting my probabilistic cracker from the most
> probable password first, I should have started from a less probable
> starting point to avoid duplicating everyone else's work. That's one
> serous downside with using it since it doesn't play well with other
> types of password cracking attacks, (aka avoid duplicating work).
This is an issue with pretty much any attack that focuses on getting the
weaker passwords first. I am not convinced that adjusting the starting
point is a good way to deal with the issue.
Thanks again,
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
