Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Mon, 3 Jul 2017 05:58:29 -0800
From: Royce Williams <royce@...ho.org>
To: john-users@...ts.openwall.com
Subject: Re: bcrypt-opencl on multiple GPUs?

On Mon, Jul 3, 2017 at 2:00 AM, magnum <john.magnum@...hmail.com> wrote:
> On 2017-07-03 07:56, Royce Williams wrote:
>>
>> I tried for the first time to use john bcrypt-opencl on multiple GPUs
>> today, like so:
>>
>> $ ./john --format=bcrypt-opencl --device=gpu --fork=6 -inc=lower
>> -min-len=8 -max-len=8 -mask='?w?l?l?l?l' --progress-every=300
>> --max-run-time=3660 single-bf.hash
>>
>> ... but only one GPU is providing status output:
>>
>> Using default input encoding: UTF-8
>> Loaded 1 password hash (bcrypt-opencl [Blowfish OpenCL])
>> Node numbers 1-6 of 6 (fork)
>> Device 3: GeForce GTX 1080
>> Device 0: GeForce GTX 1080
>> Device 5: GeForce GTX 1080
>> Device 4: GeForce GTX 1080
>> Device 1: GeForce GTX 1080
>> Device 2: GeForce GTX 1080
>> [Build log elided]
>> Press 'q' or Ctrl-C to abort, almost any other key for status
>> 1 0g 0:00:01:16 0.00% (ETA: 2037-12-19 05:32) 0g/s 53.38p/s 53.38c/s
>> 53.38C/s GPU:34C lilluela..lilleoya
>> 1 0g 0:00:02:33  0g/s 53.38p/s 53.38c/s 53.38C/s GPU:35C
>> lillioya..lillbnpa
>> 1 0g 0:00:05:06  0g/s 53.38p/s 53.38c/s 53.38C/s GPU:35C
>> lillnlxa..lillgsne
>> 1 0g 0:00:06:23  0g/s 53.38p/s 53.38c/s 53.38C/s GPU:36C
>> lillpsne..lillrmce
>> 1 0g 0:00:10:13  0g/s 53.39p/s 53.39c/s 53.39C/s GPU:37C
>> lilltywe..lillfhii
>> 1 0g 0:00:11:30  0g/s 53.39p/s 53.39c/s 53.39C/s GPU:37C
>> lillwhii..lillmbti
>> Waiting for 5 children to terminate
>> Session aborted
>
> Strange. Do you have the same problem when pressing a key? What if you send
> some or all of the children a SIGUSR1? Eg. 'pkill -usr1 john'.

Yep, the output above is from me pressing a key in some cases, and
scheduled output in others.

It's also notable that it takes more than 30 seconds between pressing
a key and having status appear.

Either USR1 is being ignored, or it's taking a very long time to
respond; it's been 3 minutes since I sent the USR1, without any change
in output.

Ah ... from looking at the nvidia-smi output, it's clear that there's
only one child of john running, despite the --fork=6:

More info:

$ ./john --format=bcrypt-opencl --device=gpu --fork=6 -inc=lower
-min-len=8 -max-len=8 -mask='?w?l?l?l?l' single-bf.hash

$ uname -a
Linux irongiant 4.4.0-83-generic #106-Ubuntu SMP Mon Jun 26 17:54:43
UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ ./john | head -1
John the Ripper 1.8.0.6-jumbo-1-3129-g087c81e6f OMP [linux-gnu 64-bit XOP-ac]

$ nvidia-smi
Mon Jul  3 05:55:22 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66                 Driver Version: 375.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    On   | 0000:01:00.0      On |                  N/A |
|100%   42C    P2    62W / 150W |    154MiB /  8113MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 1080    On   | 0000:02:00.0      On |                  N/A |
|100%   37C    P0    36W / 150W |     12MiB /  8114MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 1080    On   | 0000:05:00.0      On |                  N/A |
|100%   38C    P0    36W / 150W |     12MiB /  8114MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  GeForce GTX 1080    On   | 0000:06:00.0      On |                  N/A |
|100%   35C    P0    35W / 150W |     12MiB /  8114MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  GeForce GTX 1080    On   | 0000:09:00.0      On |                  N/A |
|100%   36C    P0    36W / 150W |     12MiB /  8114MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  GeForce GTX 1080    On   | 0000:0A:00.0      On |                  N/A |
|100%   33C    P0    35W / 150W |     12MiB /  8114MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1110    G   /usr/lib/xorg/Xorg                              14MiB |
|    0      4047    C   ./john                                         135MiB |
|    1      1110    G   /usr/lib/xorg/Xorg                               7MiB |
|    2      1110    G   /usr/lib/xorg/Xorg                               7MiB |
|    3      1110    G   /usr/lib/xorg/Xorg                               7MiB |
|    4      1110    G   /usr/lib/xorg/Xorg                               7MiB |
|    5      1110    G   /usr/lib/xorg/Xorg                               7MiB |
+-----------------------------------------------------------------------------+


Royce

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.