[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20110428033452.GA23067@openwall.com>
Date: Thu, 28 Apr 2011 07:34:52 +0400
From: Solar Designer <solar@...nwall.com>
To: announce@...ts.openwall.com, john-users@...ts.openwall.com
Subject: JtR 1.7.7 & 1.7.7-jumbo-1, Intel AVX & AMD XOP support
Hi,
I've just released John the Ripper 1.7.7, as well as 1.7.7-jumbo-1 and
DES/OpenMP patches updated to this new version.
http://www.openwall.com/john/
http://www.openwall.com/john/#contrib
http://download.openwall.net/pub/projects/john/1.7.7/
http://openwall.info/wiki/john/patches
This release has been sponsored by Rapid7, a leading provider of unified
vulnerability management and penetration testing solutions:
http://www.rapid7.com
I would also like to thank Sebastian Rother for his assistance in
getting the AVX support tested on a Core i7-2600 3.4 GHz CPU (Sandy
Bridge). Here are some performance numbers:
http://www.openwall.com/lists/john-users/2011/03/22/2
On one CPU core (non-OpenMP build):
Benchmarking: Traditional DES [128/256 BS AVX]... DONE
Many salts: 4819K c/s real, 4867K c/s virtual
Only one salt: 4080K c/s real, 4080K c/s virtual
-omp-des-4 (better suited for multi-salt; on 4 cores, 8 threads):
Benchmarking: Traditional DES [128/256 BS AVX]... DONE
Many salts: 19243K c/s real, 2429K c/s virtual
Only one salt: 9682K c/s real, 2060K c/s virtual
-omp-des-7 (better suited for single-salt; on 4 cores, 8 threads):
Benchmarking: Traditional DES [128/256 BS AVX]... DONE
Many salts: 17006K c/s real, 2358K c/s virtual
Only one salt: 14404K c/s real, 2211K c/s virtual
The changes in 1.7.7 (since 1.7.6) are as follows:
* Added Intel AVX and AMD XOP instruction sets support for bitslice DES
(with C compiler intrinsics). New make targets: linux-x86-64-avx,
linux-x86-64-xop, linux-x86-avx, and linux-x86-xop (these require recent
versions of GCC and GNU binutils).
* A "dummy" "format" is now supported (plaintext passwords encoded in
hexadecimal and prefixed with "$dummy$") - for faster testing and tuning
of custom wordlists, rule sets, .chr files, and external modes on
already known or artificial passwords, as well as for testing of future
and modified versions of John itself.
* Apache "$apr1$" MD5-based password hashes are now supported along with
the FreeBSD-style MD5-based crypt(3) hashes that were supported
previously. Hashes of both of these types may be loaded for cracking
simultaneously.
* The "--salts" option threshold is now applied before removal of
previously cracked hashes for consistent behavior with interrupted and
continued sessions. (Suggested by magnum.)
* The "Idle = Y" setting (which is the default) is now ignored for
OpenMP-enabled hash types when the actual number of threads is greater
than 1. (Unfortunately, it did not work right at least with GNU libgomp
on Linux.)
* When a cracking session terminates or is interrupted, John will now
warn the user if the cracked passwords printed to the terminal while
cracking are potentially incomplete. It will advise the user to use the
"--show" option to see the complete set of cracked passwords with proper
post-processing.
* When loading hashes specified on a line on their own (feature
introduced in 1.7.6), the loader will now ignore leading and trailing
whitespace.
* Unless a hash type is forced from the command line, the loader will
now print warnings about additional hash types seen in the input files
(beyond the hash type autodetected initially).
* For use primarily by the jumbo patch (and later by future enhancements
to the official versions as well), the loader now includes logic to warn
the user of ambiguous hash encodings (e.g. LM vs. NTLM vs. raw-MD5, all
of which may be represented as 32 hexadecimal characters) and of
excessive partial hash collisions, which it works around (these are
typically caused by an incomplete implementation of a new hash type).
* The "unique" and "unshadow" programs have been made significantly faster.
* "DateTime", "Repeats", "Subsets", "AtLeast1-Simple",
"AtLeast1-Generic", and "Policy" external mode samples have been added
to the default john.conf.
* The self-tests have been enhanced to detect more kinds of program bugs.
* A few minor bug fixes and enhancements were made.
Changes to the jumbo patch have been conservative. Besides the update
to 1.7.7, the jumbo patch has been revised to make proper use of 1.7.7's
ability to detect ambiguous hash encodings. Here's what it says when
invoked against the KoreLogic contest hashes with the first two lines
of the KoreLogic-provided file removed (such that the first hash seen
is an LM hash):
Warning: detected hash type "lm", but the string is also recognized as "mscash"
Use the "--format=mscash" option to force loading these as that type instead
Warning: detected hash type "lm", but the string is also recognized as "mscash2"
Use the "--format=mscash2" option to force loading these as that type instead
Warning: detected hash type "lm", but the string is also recognized as "lotus5"
Use the "--format=lotus5" option to force loading these as that type instead
Warning: only loading hashes of type "lm", but also saw type "des"
Use the "--format=des" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "bf"
Use the "--format=bf" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "md5"
Use the "--format=md5" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "ssha"
Use the "--format=ssha" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "nsldap"
Use the "--format=nsldap" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "oracle"
Use the "--format=oracle" option to force loading hashes of that type instead
Warning: only loading hashes of type "lm", but also saw type "pix-md5"
Use the "--format=pix-md5" option to force loading hashes of that type instead
"md5a" ("Apache MD5") is now gone as a separate "format" since those
hashes are now supported by the official JtR (and thus in jumbo as well)
via the main/shared MD5-based crypt(3) code.
Also gone is the "--salt-list" option. It was poorly implemented, and
was arguably of little use even if it were implemented better.
Additionally, I applied the following contributed patches:
john-1.7.6-jumbo-12-alainesp-nt-scale-1.diff - speedup, by Alain Espinosa
john-1.7.6-jumbo-12-hash_sizes.diff - speedup, by magnum
john-1.7.6-jumbo-12-wordistfix.patch - bugfix, by Joshua J. Drake
mkv-warnings-fix.diff - compiler warnings fix, by Dhiru Kholia
rawSHA1_fmt_faster-1.diff - speedup, by JimF
In the source code, OpenMP-enabled "formats" are now expected to set the
FMT_OMP flag, which I took care of for those currently in jumbo.
I mention this for authors of JtR patches.
Please provide "user" feedback on john-users (opinions about the
changes, testing results, feature requests, custom builds, etc.) and
"developer" feedback on john-dev.
I think that many development-related topics actually belong to
john-users as long as they don't go into discussing source code changes
in detail - e.g., bug reports and feature requests are more appropriate
for the users list.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
