[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20100620170939.GA31161@openwall.com>
Date: Sun, 20 Jun 2010 21:09:39 +0400
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: auditing SHA-crypt hashes with JtR 1.7.6+ on Linux (Ubuntu, Fedora, etc.), including OpenMP parallelization
Hi,
Earlier today, I posted a tutorial on auditing SHA-crypt hashes on
Solaris 10:
http://www.openwall.com/lists/john-users/2010/06/20/2
The following is a similar tutorial for doing it on Linux. (In fact, I
am posting to the same thread to make it easy to find this new tutorial
for those who happen to find the previous posting first.)
I'll be using Fedora 12 on x86-64. (More specifically, this is an
OpenVZ container on Openwall GNU/*/Linux, instantiated from
fedora-12-x86_64.tar.gz, which was downloaded from
http://download.openvz.org/template/precreated/ , with gcc installed
into it with "yum install gcc" as root.) The machine uses a Core i7 920
CPU at 2.66 GHz (with Turbo Boost to a higher frequency when only 1 or 2
CPU cores are in use).
Here's the version of gcc:
[solar@...ora12 ~]$ rpm -q gcc
gcc-4.4.3-4.fc12.x86_64
[solar@...ora12 ~]$ gcc -v 2>&1 | tail -1
gcc version 4.4.3 20100127 (Red Hat 4.4.3-4) (GCC)
This one is OpenMP-capable (must be 4.2 or newer), which we'll make use
of further down this tutorial.
The password file looks like:
[solar@...ora12 john]$ cat pw-sha-rs904c
test:$6$MDZEL9CQ$ZentsLbLWphRB2./B0xKv1vWPY9IBknYrcD.3SlY5RamKsnzlCSC4ImT3KWY8rXMbodbFA9wDrlf51DT3HgoW1:102:14::/home/test:/bin/sh
test:$6$dofn/L59$nygYCacync7RzPfYjWZ1OO6b8MZDETUzYP2SbJD0PPqXDjQ3caVlR8/O6G2DSMh.6X0Dzwre86QPifkEU22dW/:102:14::/home/test:/bin/sh
robert:$6$U4dzp4gvMho3$hAUA8cpTUdHnNjuMFUtiXzd0NL85RMNihshXdks/7LD5zDVseUawK1JIotk5JVZoyacpHy.Vuja0b9GD2gZ0J.:14527:0:99999:7:::
These are the password file lines posted by Robert (see the Solaris 10
revision of this tutorial), combined into one file. Two lines came from
Solaris 10 systems (and they've been through "unshadow" run on both
passwd and shadow files), the third line came from an Ubuntu system (a
shadow file line only, which is slightly wrong as it relates to JtR's
use of it).
Let's start by extracting a clean John the Ripper 1.7.6 tarball (no
patches) and compiling John:
[solar@...ora12 john]$ tar xvjf john-1.7.6.tar.bz2
john-1.7.6/
john-1.7.6/doc/
john-1.7.6/doc/EXAMPLES
[...]
john-1.7.6/src/x86-sse.h
john-1.7.6/src/DES_vec.pl
john-1.7.6/README
[solar@...ora12 john]$ cd john-1.7.6/src/
[solar@...ora12 src]$ make -j8 linux-x86-64
ln -sf x86-64.h arch.h
[...]
ln -s john ../run/unique
make[1]: Leaving directory `/home/solar/john/john-1.7.6/src'
We've successfully compiled John. Now let's try to use it:
[solar@...ora12 src]$ cd ../run/
[solar@...ora12 run]$ ./john ../../pw-sha-rs904c
Loaded 3 password hashes with 3 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:00:15 79% (1) c/s: 201 trying: testU - test52
password (robert)
password (test)
password (test)
guesses: 3 time: 0:00:00:22 100% (2) c/s: 200 trying: 12345 - missy
I pressed "any key" at 15 seconds to obtain the non-100% status line
above. It could be seen that John was trying username-based passwords
first. Also, it was probably trying passwords with "99999" in them,
because of the shadow file line containing that string in the field
which JtR expects to be the GECOS field, but this is not seen above
(I did not catch it with my keypress). You really ought to use the
included unshadow program to have JtR try the right user-specific
candidate passwords.
Then John proceeded with "pass 2", which is the wordlist mode, and it
tried "password" soon because it's found near the beginning of the
common passwords list in password.lst.
Now let's display the results properly:
[solar@...ora12 run]$ ./john --show ../../pw-sha-rs904c
test:password:102:14::/home/test:/bin/sh
test:password:102:14::/home/test:/bin/sh
robert:password:14527:0:99999:7:::
3 password hashes cracked, 0 left
That's it. But we were only using one out of four CPU cores, and we
were not using the Core i7 CPU's capability to simultaneously run two
threads or processes per core. Let's try to use all eight logical CPUs
that the Linux kernel sees. We need to make an OpenMP-enabled build.
First, clean out the old build:
[solar@...ora12 run]$ cd ../src/
[solar@...ora12 src]$ make clean
Then enable OpenMP. We need to uncomment one of the OMPFLAGS lines near
the beginning of Makefile:
[solar@...ora12 src]$ vi Makefile
We locate the line for gcc and remove the leading "#" character.
Before our edits:
# gcc with OpenMP
#OMPFLAGS = -fopenmp
After the edits:
# gcc with OpenMP
OMPFLAGS = -fopenmp
(the true comment remains with its "#" character, but the variable
assignment is now uncommented).
Now let's make the new build:
[solar@...ora12 src]$ make -j8 linux-x86-64
ln -sf x86-64.h arch.h
[...]
gcc -c -Wall -O2 -fomit-frame-pointer -fopenmp -DHAVE_CRYPT -funroll-loops c3_fmt.c
[...]
ln -s john ../run/unique
make[1]: Leaving directory `/home/solar/john/john-1.7.6/src'
And let's test it:
[solar@...ora12 src]$ cd ../run/
[solar@...ora12 run]$ rm john.pot
[solar@...ora12 run]$ ./john ../../pw-sha-rs904c
Loaded 3 password hashes with 3 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:00:01 18% (1) c/s: 564 trying: Robert99999g - Robert/
password (robert)
password (test)
password (test)
guesses: 3 time: 0:00:00:06 100% (2) c/s: 746 trying: 12345 - missy
This has completed in 6 seconds instead of 22. With more precise timing
(used by John internally) the speedup was "only" 3.7x in this case (200 c/s
to 746 c/s). The system appeared to be under some light load unrelated
to John, though. Another test run was slightly faster:
guesses: 3 time: 0:00:00:05 100% (2) c/s: 778 trying: 12345 - missy
This is a 3.9x speedup from parallelization, assuming that the 200 c/s
single process run could use all CPU time it wanted to.
Some speedup might be possible with "OMP_WAIT_POLICY=PASSIVE":
[solar@...ora12 run]$ OMP_WAIT_POLICY=PASSIVE ./john ../../pw-sha-rs904c
Loaded 3 password hashes with 3 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:00:02 51% (1) c/s: 796 trying: Drr99999 - rbert99999
password (robert)
password (test)
password (test)
guesses: 3 time: 0:00:00:05 100% (2) c/s: 785 trying: 12345 - missy
The effect of this setting on overall JtR performance is
system-specific. On another CPU (such as without SMT), gcc's default of
ACTIVE might be faster.
The "--show" option works as usual:
[solar@...ora12 run]$ ./john --show ../../pw-sha-rs904c
test:password:102:14::/home/test:/bin/sh
test:password:102:14::/home/test:/bin/sh
robert:password:14527:0:99999:7:::
3 password hashes cracked, 0 left
Finally, here's how "unshadow" may be used. As root, make the shadow
file temporarily available to our non-root account, assuming that the
home directories for "root" and "solar" are on the same filesystem:
[root@...ora12 ~]# pwd
/root
[root@...ora12 ~]# umask 077
[root@...ora12 ~]# cp /etc/shadow .
[root@...ora12 ~]# chown solar: shadow
[root@...ora12 ~]# ln shadow ~solar/
[root@...ora12 ~]# rm shadow
rm: remove regular file `shadow'? y
The "rm shadow" command removed just one of two hard-links to the shadow
file copy.
Under the non-root account, process the system's /etc/passwd file along
with the shadow file with "unshadow":
[solar@...ora12 run]$ umask 077
[solar@...ora12 run]$ chmod 600 ~/shadow
[solar@...ora12 run]$ ./unshadow /etc/passwd ~/shadow > pw-local
[solar@...ora12 run]$ shred -u ~/shadow
This assumes that no changes were made to /etc/passwd since we grabbed a
copy of /etc/shadow and until we ran the unshadow command. "chmod 600"
is needed because /etc/shadow is mode 0 on Fedora 12 (at least on this
specific install), which gets inherited in the copy. So we need to make
the file readable to "unshadow" and writable to "shred". "umask 077" is
needed to ensure that our newly-created files are not readable to other
users (the default umask may be much more relaxed).
Now let's run John as usual:
[solar@...ora12 run]$ ./john pw-local
Loaded 2 password hashes with 2 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:00:05 0% (2) c/s: 784 trying: kim - olivier
guesses: 0 time: 0:00:00:14 4% (2) c/s: 812 trying: Canela - Dammit
guesses: 0 time: 0:00:00:20 6% (2) c/s: 817 trying: scruffies - xcountries
guesses: 0 time: 0:00:02:17 37% (2) c/s: 824 trying: sydney? - minnie?
guesses: 0 time: 0:00:02:26 41% (2) c/s: 824 trying: nsbt - rbrts
Session aborted
or maybe:
[solar@...ora12 run]$ OMP_WAIT_POLICY=PASSIVE ./john pw-local
Loaded 2 password hashes with 2 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:00:09 3% (2) c/s: 808 trying: nicholas - Buster
guesses: 0 time: 0:00:04:25 76% (2) c/s: 828 trying: Foster0 - Logical0
guesses: 0 time: 0:00:04:42 80% (2) c/s: 828 trying: 7rancid - 9michelle
Session aborted
My passwords are not getting cracked this easily. Also, we're getting
slightly better performance with longer runs (these hashes use the same
iteration counts as Robert's, so a direct comparison is valid this
time) ... and the effect of "OMP_WAIT_POLICY=PASSIVE" is not obvious.
I was pressing Ctrl-C to interrupt these sessions. It is possible to
continue an interrupted session in JtR's usual way:
[solar@...ora12 run]$ OMP_WAIT_POLICY=PASSIVE ./john --restore
Loaded 2 password hashes with 2 different salts (generic crypt(3) [?/64])
guesses: 0 time: 0:00:04:44 80% (2) c/s: 827 trying: 9alexis - 9passwd
guesses: 0 time: 0:00:04:50 82% (2) c/s: 827 trying: 9softball - 5fuckme
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
