[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20181216175005.GA6555@openwall.com>
Date: Sun, 16 Dec 2018 18:50:05 +0100
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Anyone knows what does this fuzzer message mean?
On Sat, Dec 15, 2018 at 10:08:25PM -0200, Claudio Andr?? wrote:
> The format seems to be working fine (to me).
>
> ```
> $ "$JtR" --fuzz --format=sha512crypt
> Fuzzing: sha512crypt, crypt(3) $6$ (rounds=5000) [SHA512 128/128 AVX 2x]...
> Warning: excessive partial hash collisions detected
> Completed
> All 1 formats passed fuzzing test!
> ```
The message comes from the loader when it encouters too many (by
default, more than 1000) same binary_hash()es, which slows down its dupe
hash check and is likely to slow down actual cracking as well (if the
same or a similar binary_hash*() is used within a salt, and the
collisions are seen within the salt as well). During fuzzing, this is
normal, because the fuzzer alters our test vector hashes in many simple
ways, resulting in a set of fake hashes that are not random-looking.
I've just used this loader hack to see the actual collision counts:
+++ b/src/loader.c
@@ -1229,6 +1229,7 @@ static void ldr_load_pw_line(struct db_main *db, char *line)
}
if (++collisions <= LDR_HASH_COLLISIONS_MAX)
continue;
+ continue;
if (john_main_process) {
if (format->params.binary_size)
@@ -1250,6 +1251,12 @@ static void ldr_load_pw_line(struct db_main *db, char *line)
break;
} while ((current_pw = current_pw->next_hash));
+ static int prevmax = 0;
+ if (collisions > prevmax) {
+ fprintf(stderr, "collisions=%d\n", collisions);
+ prevmax = collisions;
+ }
+
if (current_pw) continue;
}
For sha512crypt, the max collision count was over 3000. For some other
formats like raw-md5 and phpass, it was around 100. So these collisions
do occur for many (maybe all) formats, but only cross the threshold for
some. I suggest we suppress these warnings with the following patch:
diff --git a/src/john.c b/src/john.c
index c7181c8..fd15f9e 100644
--- a/src/john.c
+++ b/src/john.c
@@ -1694,6 +1694,13 @@ static void john_run(void)
#ifdef HAVE_FUZZ
else
if (options.flags & FLG_FUZZ_CHK || options.flags & FLG_FUZZ_DUMP_CHK) {
+/*
+ * Suppress dupe hash check because fuzzed ones often result in too many
+ * partial hash collisions.
+ */
+ options.loader.flags |= DB_WORDS;
+ list_init(&single_seed); /* Required for DB_WORDS */
+
ldr_init_database(&database, &options.loader);
exit_status = fuzz(&database);
}
Alternatively, we could export the skip_dupe_checking flag (currently
local to a function in loader.c) and set it from fuzz_test().
Or we could set a flag in the fuzzer and check it from the loader.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
