[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20080825070111.GA32056@openwall.com>
Date: Mon, 25 Aug 2008 11:01:11 +0400
From: Solar Designer <solar@...nwall.com>
To: john-users@...ts.openwall.com
Subject: jumbo patch for 1.7.3.1
Hi,
I've just released the jumbo patch updated to JtR 1.7.3.1, with newly
contributed modules added and lots of bugs fixed (both in the newly
added code and in code previously found in jumbo patches), and also with
some generic improvements that I originally introduced in JtR Pro. The
current revision is 1.7.3.1-all-2, linked right from JtR homepage and
also found in the contrib/ FTP directory:
http://www.openwall.com/john/
ftp://ftp.openwall.com/pub/projects/john/contrib/
It has been briefly tested on all four combinations of 32-bit / 64-bit
and little-endian / big-endian, as well as with / without generic / MMX /
SSE2 assembly code on x86 and x86-64.
Some more detail on the changes, besides the update to JtR 1.7.3.1:
I have revised Alain's NTLM x86-64 assembly code to use instruction
pointer relative addressing, which made it work on Mac OS X (originally
in JtR Pro). Thus, the extra macosx-* make targets introduced in 1.7.3+
are now also usable with the jumbo patch (tested on 10.5). However, I
did not revise Alain's and others' x86 and x86-64 assembly codes for
compilation with Sun's assembler. Thus, the extra solaris-x86-* make
targets introduced in 1.7.3+ are not supposed to work when the jumbo
patch is applied, at least not yet.
The NTLM "loader hack" previously found in jumbo patches has been
replaced with more correct code from JtR Pro. However, it is surrounded
by pieces of code derived from the older "loader hack" for the various
NET*LM (challenge/response) "formats".
The Mac OS X 10.4+ salted SHA-1 hashes support code has been replaced
with my own, from JtR Pro. It is not very fast yet (it uses OpenSSL),
but it is cleaner. The corresponding "format label" is now "XSHA" (or
"xsha" if you like), and only all-uppercase strings will be accepted
for the hashes (48 hex digits), because that's how Mac OS X itself
encodes those hashes.
The OpenSSL-derived implementation of SHA-1 has been dropped, in favor
of using one from whatever version of OpenSSL we're linking against.
We already had a dependency on OpenSSL in the jumbo patches anyway.
Several "formats" have been revised to use OpenSSL's newer names and
calling conventions for the DES routines. The code still builds and
works with OpenSSL 0.9.7, but now it should also build and work with
later/future versions, and with some distribution vendor adjusted ones
that dropped the backwards compatibility stuff earlier (e.g., NetBSD).
Many contributed additions and fixes have been merged:
- sap friend's "patch for SAP-passwords (BCODE & PASSCODE)",
john-1.7.2-SAPLover-1.diff, with minor bug fixes, and sap_prepare.pl is
now added to the "run" directory;
- Samuel Monux's "NetscreenOS passwords", john-1.7.2-netscreen-2.diff,
with minor changes, and netscreen.py is now added to the "run" directory;
- the portion of Simon Marechal's "working with many passwords",
john-1.7.2-largepasswordset-1.diff, responsible for faster processing of
successful guesses;
- Bucsay Balazs and Peter Kasza's "raw-md5 module improvement" and
revised "new MYSQL323 module", john-1.7.3-rawMD5_fast-2.diff,
john-1.7.3.1-rawmd5-bigendian.patch, and john-1.7.3-MYSQL_fast-2.diff,
with minor bug fixes and a symbol clash resolved;
- JoMo-Kun's bug fix to his loader hack, "NetLM / NetNTLM loader.c
Patch", loader.c.diff.
Oh, did I mention bug fixes? ;-) Well, the most common bug, of those I
dealt with this time, was to declare something as an array of char, then
cast portions of it to (int *) or the like (and dereference). Indeed,
this works on x86, but it does not guarantee proper alignment - so
things break on some builds on architectures with strict alignment
requirements, and performance is sometimes non-optimal even on x86.
An even worse variation of the bug is to inconsistently cast to (int *)
and (long *). This works on 32-bit, sometimes works on 64-bit
little-endian, but breaks badly (or rather nicely, because it is easy to
reproduce) on 64-bit big-endian. The fix I used this time was to change
the initial declaration to be based on a large enough integer data type
(such as ARCH_WORD_32), then cast to (char *) or the like in places
where the code depended on that. Other bugs included out-of-bounds
array accesses - on one occasion it was a cross-function off-by-one
nicely spotted at compile time by gcc 4.3.1 (impressive), on another it
was an issue with OpenSSL DES routines processing data in whole DES
blocks, whereas the buffers were not a multiple of DES block size.
Speaking of new code, sapB_fmt.c incorrectly declared final_key[] as
BINARY_SIZE+1 instead of BINARY_SIZE*2 bytes. A bug that cost me half
an hour to figure out was the KRB5_* stuff making an assumption about
the size of DES key schedule structs, which may differ across OpenSSL
versions and builds.
I'm sure there are plenty of bugs left in the jumbo patch, but at least
all self-tests pass on all machines and make targets that I used for
testing this time. Not surprisingly, not a single bug was found in
"official" JtR code during this testing session. ;-)
I know there's still plenty of stuff to consider and maybe merge from
Simon Marechal's "myjohn". Simon - would you be so kind to submit it as
patches against 1.7.3.1-all-2 (or whatever will be latest at the time)?
When submitting patches, please pay attention to these guidelines:
http://openwall.info/wiki/how-to-make-patches
Also being made available at the same time is "revision 1" of the patch
for 1.7.3.1, which is a mere update to this newer version of JtR -
without any additions or fixes. It is only available via FTP, under
contrib/historical/ - because it is not meant for actual use, but rather
to maintain some "poor man's revision history". (Maybe we'll introduce
proper revision control for contributed patches eventually, in one way
or another. Meanwhile, keeping older and intermediate revisions under
historical/ should do. Right now, only "official" JtR code is in our
CVS repository, along with other Openwall software.)
And last but not least - Thank You to all who have contributed code or
provided bug reports or fixes.
Alexander
--
To unsubscribe, e-mail john-users-unsubscribe@...ts.openwall.com and reply
to the automated confirmation request that will be sent to you.
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
