[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20130425223147.GA20692@openwall.com>
Date: Fri, 26 Apr 2013 02:31:47 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: revised incremental mode and charset files (was: Bleeding-jumbo branch updated from core)
On Thu, Apr 25, 2013 at 10:45:50PM +0200, magnum wrote:
> I have merged latest commits from core CVS into the bleeding git branch.
Thank you!
I've just pushed another related change out to the public CVS
repository: speeding up charset file generation, as this becomes more
important with increased CHARSET_* settings that we're going to use.
There were two commits:
"Optimized the charset file generation algorithms and code ("--make-charset").
On a certain test case, this has improved the time to generate a charset file
from 55.5 to 19.5 seconds while generating the exact same file."
and:
"In cmp_ratio(), stabilize the sorting order differently from the way we
happened to be doing it before. This results in subtly different charset files
being generated for the same input data, but arguably the new stabilization is
more logical, and it helps reduce the number of recalculations and thus speeds
up charset file generation (now 16.5 seconds in the test case mentioned in the
previous commit)."
The mentioned test case is full RockYou (including dupes), CHARSET_MAX
0xFF, and CHARSET_LENGTH 15 (I also tested 20 and 24, with just slightly
longer charset file generation times with the new code). The times are
as measured on bull.
> Worse, the bleeding branch now does not contain any working charset files for Incremental, because the old ones are incompatible (at least for now). Try creating new ones from the Rockyou list, for testing stuff.
>
> Obviously, with [current] latest bleeding version you can't resume an Incremental session started with an earlier version.
Yes. I considered keeping/introducing some backwards compatibility
code, like we had for JtR 1.6's (not a typo) incremental mode revision
until now. This would be reasonably easy if we kept CHARSET_MAX and
CHARSET_LENGTH intact, but clearly we're not going to. Since at least
the default CHARSET_LENGTH will change, which would break compatibility
with old charset files even without code changes, my options were either
to revise/add a lot of code to support charset files with arbitrary
CHARSET_LENGTH settings (different from the current build's) or to drop
the backwards compatibility altogether and simply standardize on new and
much increased default CHARSET_LENGTH (and maybe new CHARSET_MAX as
well). I chose the latter.
What do you think the new CHARSET_LENGTH and CHARSET_MAX defaults should
be? I think the minimum for new defaults is CHARSET_LENGTH 15
(increased from the current/old default of 8) and CHARSET_MAX 0x7E (same
as it is now), but should we go higher?
If we support 8-bit chars by default (increase CHARSET_MAX), then should
the supplied .chr files include those or not? If yes, then where should
the input data come from? RockYou has about 18k strings with 8-bit
chars, but it is unclear how many of those are actual passwords; I think
the percentage of junk is a lot higher within this portion of RockYou
than it is in RockYou overall.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
