Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 19 Sep 2015 01:02:20 +0300
From: Solar Designer <>
Subject: Re: current large hash file speeds and bottlenecks (was: ldr_split_line() performance regression)

On Fri, Sep 18, 2015 at 07:57:05PM +0300, Solar Designer wrote:
> --fork=32:
> real    0m17.399s
> user    2m38.080s
> sys     0m19.943s
> time ./john -form=raw-md5 -w=10m.pass -ru=fred-best64 -nolog -v=1 -save-mem=1 -fork=32 29m.txt
> 1709703 passwords are cracked in all cases, like before.
> The next hurdle is "--show", which is still unreasonably slow: takes
> several minutes to display those same passwords we've cracked in ~20
> seconds.  I haven't looked into it yet.  I think this might be a
> regression with some changes in jumbo as I seem to recall "--show"
> working no slower than normal loading of large files before.  I hope
> it's something trivial.

It was mostly several trivial issues, addressed in the attached patch.
I am still not entirely happy with the performance, but I think it's now

[ run]$ time ./john -form=raw-md5 -show 29m.txt | tail -2

1709703 password hashes cracked, 27302555 left

real    0m14.284s
user    0m13.883s
sys     0m0.456s

The patch also bumps up unique's default buffer size.  Ideally, this
should be split into a separate commit.


View attachment "john-huge-show-unique.diff" of type "text/plain" (4527 bytes)

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.