[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20190112212258.GA25254@openwall.com>
Date: Sat, 12 Jan 2019 22:22:59 +0100
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Cc: Denis Burykin <apingis@...nwall.net>
Subject: Re: sha256crypt password cracking on FPGA
Hi,
As many of you are aware, we support descrypt, bcrypt, sha512crypt,
sha256crypt, md5crypt, phpass, and Drupal7 password hash cracking on the
old ZTEX 1.15y quad-FPGA boards. Threads:
https://www.openwall.com/lists/john-users/2016/11/06/1
https://www.openwall.com/lists/john-users/2017/06/25/1
https://www.openwall.com/lists/john-users/2018/07/23/1
https://www.openwall.com/lists/john-users/2018/08/27/11
https://www.openwall.com/lists/john-users/2018/10/12/1
Denis is currently working on improving these existing designs and
removing unnecessary historical differences between them. Denis'
original design for sha256crypt, compared to his earlier sha512crypt
design, was an experiment in not trying to maximize the theoretical
clock rate (which wouldn't be fully achieved in practice anyway) for
simplicity and to pack more SHA-256 cores per chip:
On Mon, Aug 27, 2018 at 06:07:16PM +0200, Solar Designer wrote:
> Denis wrote a good description of the design with some ASCII diagrams,
> currently found here:
>
> https://github.com/magnumripper/JohnTheRipper/tree/bleeding-jumbo/src/ztex/fpga-sha256crypt
>
> Similarly to Denis' design for sha512crypt and Drupal7, the new one for
> sha256crypt uses specialized soft CPU cores along with cryptographic
> cores. However, the specific parameters of those cores changed: while
> the sha512crypt and Drupal7 design used 32-bit 16-way SMT CPU cores, the
> one for sha256crypt uses smaller 16-bit 6-way SMT CPU cores, and while
> the SHA-512 cores handled up to 4 in-flight hashes, the SHA-256 ones
> handle only up to 2. Accordingly, the ratio of SHA-2 to CPU cores was 4
> to 1, and is now 3 to 1. These changes are in part due to SHA-256 being
> smaller and faster (so without the smaller CPU cores the ratio would
> have been even lower), and in part due to Denis not optimizing this for
> maximum theoretical clock rate (per design tools) to the same extent, as
> that clock rate for sha512crypt and Drupal7 turned out to be unreachable
> in practice on the ZTEX boards anyway (to remind, for those hashes the
> toolset reported clock rate was 225 MHz while actual stable under full
> device utilization was up to 160 MHz).
>
> Three SHA-256 cores, one soft CPU core, and memory and glue logic form a
> unit. The SHA-256 cores occupy 2/3 of the unit's area, and the soft CPU
> core occupies 10%. The rest goes primarily to shared SHA-256 context
> logic such as buffering and padding, which isn't in the cores.
>
> 25 units fit in one Spartan-6 LX150 FPGA. This means 25 soft CPU cores,
> 150 hardware threads, 75 SHA-256 cores, up to 150 in-flight SHA-256 per
> FPGA. Four times that per board.
>
> Also included are on-device candidate password generator (for mask mode,
> including in hybrid modes along with a wordlist coming from host, etc.)
> and hash comparator (capable of up to 512 loaded hashes per salt; no
> limit on total loaded hashes as that's handled on host). This is the
> same as Denis' design for sha512crypt and Drupal7 also has.
>
> Per Xilinx tools, this design was supposed to work at 166 MHz. In our
> testing on actual boards, the design works reliably for us at 135 MHz on
> many boards tested, and at 160 MHz on some. The frequency is
> configurable in john.conf, where we set the default to 135 MHz.
Denis has now changed the sha256crypt design to be more similar to those
for sha512crypt (and Drupal7) and md5crypt (and phpass), doubling the
number of threads (to 4 in-flight hashes per SHA-256 core, 12-way SMT
CPU cores), increasing the clock rate (design tools' reported increased
from 166 to 241 MHz, JtR's default increased from 135 to 175 MHz), but
resulting in larger units and thus having to reduce their number (from
25 to 23 units per chip, or from 75 to 69 SHA-256 cores per chip). The
total number of hardware threads in the CPUs and of in-flight hashes
increased from 150 to 276 per chip, or from 600 to 1104 per board. At
our default clock rates for the two designs, this results in an overall
speedup of about 19%. At the design tools' reported clock rates, the
speedup would be over 33%.
Another improvement (not yet implemented in Denis' other designs) is
reduction of power consumption at idle through clock gating. Denis says
one board with the new design consumes just ~0.4A at 12V at idle now, or
around 5W, whereas under full load it consumes ~3.1A, or ~37W. This is
with our new default frequency of 175 MHz.
> As discussed in the Twitter thread below, sha256crypt's performance is
> very sensitive to combination of the salt and password lengths (and this
> is also a reason to avoid using sha256crypt defensively - you get major
> timing leaks of the password length even for realistically small lengths
> such as 7 vs. 8 or 11 vs. 12 characters, with the exact thresholds
> varying by salt length):
>
> https://twitter.com/solardiz/status/1031235063181189120
>
> For consistency with Hashcat benchmarks, I chose to use salt length 8
> and password length 7, generating a test password hash with:
>
> $ perl -e 'print crypt("pass256", "\$5\$saltsalt"), "\n";' > pw-sha256crypt-1
> $ cat pw-sha256crypt-1
> $5$saltsalt$ntUtUcOovI4zhuDuXQTtZ4lD7F8GHhVVRI4q1SIfQN3
>
> Here's a test run against one sha256crypt hash on one board (4 FPGAs) at
> 135 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' pw-sha256crypt-1
> [...]
> Loaded 1 password hash (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Cost 1 (iteration count) is 5000 for all loaded hashes
> Press 'q' or Ctrl-C to abort, almost any other key for status
> pass256 (?)
> 1g 0:00:02:57 DONE (2018-08-27 15:41) 0.005640g/s 112392p/s 112392c/s 112392C/s pass256..pas##u6
Same as above with the new design. One board (4 FPGAs) at 175 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' pw-sha256crypt-1
[...]
Loaded 1 password hash (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Cost 1 (iteration count) is 5000 for all loaded hashes
Press 'q' or Ctrl-C to abort, almost any other key for status
pass256 (?)
1g 0:00:02:29 DONE (2019-01-12 17:44) 0.006685g/s 133221p/s 133221c/s 133221C/s pass256..pas##u6
> Four boards (16 FPGAs), 135 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' pw-sha256crypt-1
> [...]
> Loaded 1 password hash (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Cost 1 (iteration count) is 5000 for all loaded hashes
> Press 'q' or Ctrl-C to abort, almost any other key for status
> pass256 (?)
> 1g 0:00:00:44 DONE (2018-08-27 15:57) 0.02234g/s 445201p/s 445201c/s 445201C/s pass256..pas##u6
>
> Scaling efficiency 445201/112392/4 = 99.0%.
Same as above with the new design - four boards (16 FPGAs), 175 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' pw-sha256crypt-1
[...]
Loaded 1 password hash (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Cost 1 (iteration count) is 5000 for all loaded hashes
Press 'q' or Ctrl-C to abort, almost any other key for status
pass256 (?)
1g 0:00:00:38 DONE (2019-01-12 17:46) 0.02613g/s 528384p/s 528384c/s 528384C/s pass256..pas##U6
Scaling efficiency 528384/133221/4 = 99.1%.
> This is roughly 74% of the speed of one GTX 1080 Ti, which is reported
> to achieve around 600 kH/s in Jeremi Gosney's Hashcat benchmarks:
>
> https://gist.github.com/epixoip/ace60d09981be09544fdd35005051505
>
> Hashtype: sha256crypt $5$, SHA256 (Unix)
>
> Speed.Dev.#1.....: 599.8 kH/s (75.76ms)
> Speed.Dev.#2.....: 593.7 kH/s (76.53ms)
> Speed.Dev.#3.....: 593.1 kH/s (76.59ms)
> Speed.Dev.#4.....: 590.5 kH/s (76.94ms)
> Speed.Dev.#5.....: 596.1 kH/s (76.24ms)
> Speed.Dev.#6.....: 596.2 kH/s (76.22ms)
> Speed.Dev.#7.....: 603.7 kH/s (75.27ms)
> Speed.Dev.#8.....: 601.5 kH/s (75.53ms)
> Speed.Dev.#*.....: 4774.6 kH/s
>
> With lucky ZTEX boards doing this at 160 MHz, it'd be ~88% of a 1080 Ti.
Now we're up to 88% of a 1080 Ti with the four boards above at our new
default of 175 MHz. No need for lucky boards for that.
> (Only two of my four boards tested here are lucky enough. All four
> might pass the one password test, but from more extensive testing I know
> that two would often miss guesses when running at 160 MHz.)
My two lucky boards run the new design at 190 MHz.
> Denis says his board consumes 2.6A at 12V running this at 160 MHz, which
> is 31.2W. Comparing this to atom's "This same hash, running a GTX1080
> and capped at 90W, is doing 355kH/s" (referring to a different hash with
> the same salt length, so should be a valid comparison), we get 383 kH/s
> per 90W for the FPGAs, which is slightly more energy-efficient than the
> power-capped GPU's 355 kH/s.
For the new design it's 322 kH/s per 90W at 175 MHz, which is worse than
the old design's and even slightly worse than the GPU. I guess that's a
price for trying to maximize performance.
> Now to some multi-hash runs for reliability testing:
>
> $ perl -e 'for ($i = 100; $i < 612; $i++) { print crypt("pass$i", "\$5\$saltsalt"), "\n"; }' > pw-sha256crypt
>
> One board (4 FPGAs), 160 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -dev=04A3465XXX -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:05 0.93% (ETA: 15:54:43) 0g/s 130932p/s 130932c/s 67037KC/s pasaa"a..pasat"a
> 52g 0:00:00:23 3.86% (ETA: 15:55:41) 2.195g/s 132574p/s 132574c/s 65162KC/s pasaaVi..pasatVi
> 206g 0:00:01:33 15.29% (ETA: 15:55:53) 2.197g/s 132848p/s 132848c/s 57501KC/s pasaaYc..pasatYc
> 461g 0:00:02:38 25.93% (ETA: 15:55:55) 2.901g/s 132895p/s 132895c/s 42594KC/s pasaa*b..pasat*b
> 512g 0:00:02:43 DONE (2018-08-27 15:48) 3.124g/s 132846p/s 132846c/s 41449KC/s pass577..pas##E7
As the above was one lucky board at its higher than default clock rate,
here's the new design on a lucky board at 190 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -dev=04A3465XXX -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:04 0.71% (ETA: 17:38:20) 0g/s 141568p/s 141568c/s 72483KC/s pasaaFa..pasa9Fa
52g 0:00:00:16 2.84% (ETA: 17:38:20) 3.237g/s 143860p/s 143860c/s 72433KC/s pasaaZ1..pasa9Z1
206g 0:00:01:24 14.89% (ETA: 17:38:19) 2.451g/s 144331p/s 144331c/s 62525KC/s pasaacc..pasa9cc
466g 0:00:02:29 26.42% (ETA: 17:38:19) 3.127g/s 144380p/s 144380c/s 45364KC/s pass417..pasa907
512g 0:00:02:31 DONE (2019-01-12 17:31) 3.388g/s 144323p/s 144323c/s 44812KC/s pass577..pas##R7
> Two boards (8 FPGAs), 160 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 52g 0:00:00:27 9.04% (ETA: 15:54:11) 1.868g/s 264620p/s 264620c/s 125368KC/s pasaaKs..pasa6Ks
> 206g 0:00:00:47 15.42% (ETA: 15:54:16) 4.345g/s 264982p/s 264982c/s 114756KC/s pasaa@...pasa6@c
> 410g 0:00:01:07 22.07% (ETA: 15:54:16) 6.037g/s 264729p/s 264729c/s 96949KC/s pasaa{4..pasa6{4
> 512g 0:00:01:22 DONE (2018-08-27 15:50) 6.194g/s 264657p/s 264657c/s 82757KC/s pass177..pas##D7
Two lucky boards (8 FPGAs), 190 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
52g 0:00:00:20 7.09% (ETA: 17:37:24) 2.585g/s 287220p/s 287220c/s 137348KC/s pasaa$r..pasaf$r
206g 0:00:00:40 14.18% (ETA: 17:37:24) 5.137g/s 288079p/s 288079c/s 127418KC/s pass473..pass333
410g 0:00:01:07 23.76% (ETA: 17:37:24) 6.101g/s 287940p/s 287940c/s 99321KC/s pasaaUu..pasafUu
512g 0:00:01:16 DONE (2019-01-12 17:33) 6.712g/s 287777p/s 287777c/s 89554KC/s pass477..pas##K7
> Four boards (16 FPGAs), 160 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:03 2.13% (ETA: 15:53:17) 0g/s 514183p/s 514183c/s 263262KC/s pasaa11..pasa.11
> 50g 0:00:00:08 5.32% (ETA: 15:53:26) 6.024g/s 521927p/s 521927c/s 254178KC/s pas32%o..pasa.nn
> 168g 0:00:00:21 13.83% (ETA: 15:53:27) 7.835g/s 525335p/s 525335c/s 239972KC/s pass223..pasa.33
> 503g 0:00:00:46 30.32% (ETA: 15:53:28) 10.72g/s 526490p/s 526490c/s 151516KC/s pasaaQp..pasa.Qp
> [...]
> 503g 0:00:02:34 DONE (2018-08-27 15:53) 3.252g/s 526652p/s 526652c/s 48574KC/s pasaa||..pas||}|
>
> Oops, like I said the other two boards don't manage this frequency -
> only 503 out of 512 passwords got cracked. (The longer runtime and
> lower C/s rate is explained by this run having done more work: it
> continued to test other candidate passwords against the remaining 9
> hashes past the point where the previous two runs had stopped upon
> cracking all passwords.)
Four boards (16 FPGAs), 190 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
49g 0:00:00:07 4.96% (ETA: 17:36:49) 6.853g/s 565482p/s 565482c/s 277651KC/s pasaa/o..pasaV/o
149g 0:00:00:17 12.06% (ETA: 17:36:50) 8.652g/s 570220p/s 570220c/s 266987KC/s pasaaTt..pasaVTt
388g 0:00:00:34 24.11% (ETA: 17:36:50) 11.30g/s 572381p/s 572381c/s 203366KC/s pasaa~u..pasaV~u
482g 0:00:00:51 36.17% (ETA: 17:36:50) 9.373g/s 572882p/s 572882c/s 144014KC/s pasaaAA..pasaVAA
[...]
482g 0:00:02:22 DONE (2019-01-12 17:36) 3.385g/s 572024p/s 572024c/s 62708KC/s pas##||..-----
This is more of an oops than we had with a higher than default clock
rate on all of these four boards before - only 482 out of 512 cracked.
> Four boards (16 FPGAs), 135 MHz:
>
> $ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:03 1.60% (ETA: 15:57:39) 0g/s 428910p/s 428910c/s 219602KC/s pasaaBe..pasa.Be
> 115g 0:00:00:19 10.64% (ETA: 15:57:30) 5.888g/s 443625p/s 443625c/s 208548KC/s pass302..pasa.22
> 206g 0:00:00:27 14.89% (ETA: 15:57:32) 7.545g/s 444307p/s 444307c/s 196239KC/s pasaacc..pasa.cc
> 410g 0:00:00:39 21.81% (ETA: 15:57:30) 10.26g/s 444808p/s 444808c/s 167448KC/s pass374..pasa./4
> 512g 0:00:00:49 DONE (2018-08-27 15:55) 10.29g/s 444352p/s 444352c/s 139720KC/s pass477..pas##\7
> Session completed
>
> This worked OK.
Four boards (16 FPGAs), 175 MHz:
$ ./john -form=sha256crypt-ztex -mask='pas?a?a?a?a' -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:02 1.42% (ETA: 17:41:11) 0g/s 502260p/s 502260c/s 257157KC/s pasaaze..pasaVze
104g 0:00:00:15 9.93% (ETA: 17:41:21) 6.757g/s 525432p/s 525432c/s 249505KC/s pass370..pasaVN0
206g 0:00:00:23 14.89% (ETA: 17:41:24) 8.944g/s 526686p/s 526686c/s 230713KC/s pasaacc..pasaVcc
461g 0:00:00:39 25.53% (ETA: 17:41:22) 11.70g/s 528024p/s 528024c/s 175175KC/s pasaabb..pasaVbb
512g 0:00:00:41 DONE (2019-01-12 17:39) 12.29g/s 527108p/s 527108c/s 168067KC/s pass377..pas##K7
Session completed
This still works OK, just faster.
> Now to some wordlist mode runs, using RockYou top 1 million passwords
> sorted for decreasing number of occurrences. (More precisely, 1136144
> passwords to have a consistent cut-off number of occurrences.)
>
> Two boards (8 FPGAs), 160 MHz:
> $ ./john -form=sha256crypt-ztex -w=rtop1m -mask='?w?d' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:01 4.16% (ETA: 16:28:03) 0g/s 180431p/s 180431c/s 92381KC/s scofield1..1403927
> 20g 0:00:00:03 8.41% (ETA: 16:28:14) 5.089g/s 191450p/s 191450c/s 97384KC/s pass116..tiamarie7
> 60g 0:00:00:06 12.70% (ETA: 16:28:26) 9.118g/s 190577p/s 190577c/s 95288KC/s pass146..070319957
> 70g 0:00:00:09 17.02% (ETA: 16:28:31) 7.567g/s 189794p/s 189794c/s 92294KC/s pass154..0116007
> 80g 0:00:00:10 19.21% (ETA: 16:28:31) 7.554g/s 189461p/s 189461c/s 91083KC/s pass217..danny937
> 100g 0:00:00:13 23.57% (ETA: 16:28:34) 7.513g/s 188429p/s 188429c/s 88561KC/s pass107..240425257
> 150g 0:00:00:18 32.32% (ETA: 16:28:34) 7.991g/s 187064p/s 187064c/s 83885KC/s pass167..hanneman7
> 180g 0:00:00:27 45.63% (ETA: 16:28:38) 6.649g/s 185297p/s 185297c/s 78288KC/s pass267..im1ru127
> 210g 0:00:00:35 59.02% (ETA: 16:28:38) 5.920g/s 183839p/s 183839c/s 73620KC/s hickling1..gunpowder17
> 280g 0:00:01:02 DONE (2018-08-27 16:28) 4.480g/s 181778p/s 181778c/s 61995KC/s 060850#..-----
>
> (Like in some other runs, I pressed a key a few times during this one to
> see status. "pass" is seen so often at the start of a range due to
> peculiarity of JtR's internal "formats" interface: when successful
> guesses are found in the range, the same interface returns them.)
>
> Note that the c/s rate is much lower than it was for the 7 character
> mask runs (was 264657c/s, now 181778c/s). That's primarily because of
> sha256crypt's sensitivity to (candidate) password lengths. Our wordlist
> contains many lines longer than 7, and they're not sorted by length.
Two boards (8 FPGAs), 175 MHz:
$ ./john -form=sha256crypt-ztex -w=rtop1m -mask='?w?d' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
60g 0:00:00:07 N/A 7.853g/s 189267p/s 189267c/s 93119KC/s pass147..mattt7
150g 0:00:00:18 N/A 8.064g/s 186580p/s 186580c/s 84023KC/s pass166..john727
280g 0:00:00:47 N/A 5.888g/s 182460p/s 182460c/s 68057KC/s manman781..mafexita7
280g 0:00:01:02 DONE (2019-01-12 17:53) 4.472g/s 181458p/s 181458c/s 61637KC/s 070133#..-----
This is similar to the old design's, but at the new design's default
clock rate rather than the old design's increased clock rate.
Two lucky boards (8 FPGAs), 190 MHz:
$ ./john -form=sha256crypt-ztex -w=rtop1m -mask='?w?d' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
180g 0:00:00:24 N/A 7.349g/s 200751p/s 200751c/s 85189KC/s pass266..lana147
210g 0:00:00:36 N/A 5.772g/s 198735p/s 198735c/s 77506KC/s 32370511..2618197
280g 0:00:00:48 N/A 5.779g/s 196978p/s 196978c/s 70948KC/s gamer33..ero-sennin7
280g 0:00:00:57 DONE (2019-01-12 17:56) 4.856g/s 197036p/s 197036c/s 66928KC/s 070133#..-----
Now this is actually faster than the old design's.
> Let's try sorting them for increasing length:
>
> $ awk '{ print length, $0 }' < rtop1m | sort -n | cut -d' ' -f2- > rtop1m-by-length
>
> $ ./john -form=sha256crypt-ztex -w=rtop1m-by-length -mask='?w?d' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
> [...]
> Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
> Press 'q' or Ctrl-C to abort, almost any other key for status
> 0g 0:00:00:03 8.40% (ETA: 16:34:54) 0g/s 254619p/s 254619c/s 130365KC/s 1618251..1783107
> 0g 0:00:00:05 12.01% (ETA: 16:35:00) 0g/s 257671p/s 257671c/s 131927KC/s 6805021..7533577
> 280g 0:00:00:17 32.63% (ETA: 16:35:11) 16.14g/s 245882p/s 245882c/s 109692KC/s ashes011..baby6217
> 280g 0:00:00:24 42.94% (ETA: 16:35:14) 11.32g/s 223203p/s 223203c/s 88713KC/s pencere1..pipe1237
> 280g 0:00:00:29 49.62% (ETA: 16:35:16) 9.605g/s 215094p/s 215094c/s 81219KC/s 181019611..198602257
> 280g 0:00:00:35 58.90% (ETA: 16:35:17) 7.984g/s 207390p/s 207390c/s 74145KC/s funy65411..giants247
> 280g 0:00:00:51 86.55% (ETA: 16:35:17) 5.455g/s 195479p/s 195479c/s 63139KC/s allahlove11..ashleybabe7
> 280g 0:00:01:00 DONE (2018-08-27 16:35) 4.662g/s 189163p/s 189163c/s 59085KC/s andresydaniela#..-----
>
> This is now slightly faster: 182k to 189k c/s overall, and 191k to 254k
> early on (on smaller password lengths, especially in the second run).
> And in this case the final guess count is reached at least twice sooner.
Two lucky boards (8 FPGAs), 190 MHz:
$ ./john -form=sha256crypt-ztex -w=rtop1m-by-length -mask='?w?d' -dev=04A3465XXX,04A3466XXX -verb=1 pw-sha256crypt
[...]
Loaded 512 password hashes with no different salts (sha256crypt-ztex, crypt(3) $5$ [sha256crypt ZTEX])
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 0:00:00:03 N/A 0g/s 268606p/s 268606c/s 137526KC/s 1203981..1235437
280g 0:00:00:12 N/A 22.70g/s 281459p/s 281459c/s 144107KC/s pass568..solmio7
280g 0:00:00:25 N/A 10.79g/s 234215p/s 234215c/s 91812KC/s 022319881..050525277
280g 0:00:00:38 N/A 7.267g/s 217669p/s 217669c/s 75718KC/s sxcjames1..termleap7
280g 0:00:00:51 N/A 5.474g/s 209196p/s 209196c/s 67530KC/s techn0l0gy1..uruchimaru7
280g 0:00:00:55 DONE (2019-01-12 17:57) 5.046g/s 204778p/s 204778c/s 65023KC/s ventenecesito#..-----
I don't know what happened to the progress indicator on these tests
(we're getting N/A in place of the percentage) - we'll need to look into
that separately.
> We'd appreciate more testing, such as on Royce' larger cluster of ZTEX
> boards maybe. Please post your results as follow-ups to this message.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
