[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20210503175342.GA8548@openwall.com>
Date: Mon, 3 May 2021 19:53:42 +0200
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: Re: source of information for John's charset files
On Mon, May 03, 2021 at 04:18:15PM +0200, Solar Designer wrote:
> On Sun, May 02, 2021 at 11:00:34PM -0400, Matt Weir wrote:
> > Side note, I just saw your most recent results of training/running against
> > RockYou. I'm willing to admit I'm wrong if you are getting better results
> > training without dupes. That's just contrary to what I've seen in the past.
> > I might need to run some tests of my own to look into this.
>
> Note: better results when the test set is also without dupes. However,
> I think that's what matters after most dupes are eliminated using a
> wordlist anyway in real-world usage of our tools.
I found an easily publicly available example where training without
dupes produces worse results even when the test set is also without
dupes. Running through RockYou as a wordlist first corrects that.
I took phpbb-withmd5.txt.bz2 from here:
https://wiki.skullsecurity.org/Passwords
phpbb phpbb.txt.bz2 (868,606 bytes) n/a 2009-01 Ordered by commonness
Cracked from md5 by Brandon Enright
(97%+ coverage)
phpbb with count phpbb-withcount.txt.bz2 (872,867 bytes) n/a
phpbb with md5 phpbb-withmd5.txt.bz2 (4,117,887 bytes) n/a
My 3 training sets are: RockYou with dupes, RockYou unique (full 14.3M
this time), HIBP v7 cracked excluding fbobh_* (458M).
Cracked phpbb uniques with incremental mode at 1 billion candidates are:
RockYou with dupes - 34.7%
RockYou unique - 33.6%
HIBP v7 cracked - 30.9%
RockYou wordlist - 39.3%
RockYou wordlist together with:
RockYou with dupes - 51.3%
RockYou unique - 51.5%
HIBP v7 cracked - 50.6%
RockYou wordlist with best64 rules (only, no incremental):
RockYou + best64 - 56.3%
RockYou wordlist with best64 rules (above) together with the incremental
runs (first set of results above):
RockYou with dupes - 60.4%
RockYou unique - 60.5%
HIBP v7 cracked - 60.1%
All of these combined - 62.3%.
Of course, real attacks on fast hashes like this would test many more
candidates and get almost all hashes cracked. These simulations I run
are to see how effective the approaches are in case similar passwords
were used with slow (non-)hashes and/or with many different salts.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
