Follow @Openwall on Twitter for new release announcements and other news

The following changes have been made between John 1.8.0 and 1.9.0:

* Increased the interleaving for bcrypt on x86-64 from 2x to 3x for a major speedup on CPUs without SMT. Unfortunately, this sometimes results in a minor performance regression when running multiple threads on CPUs with SMT.
* Recognize the $2b$ bcrypt prefix.
* In the generic crypt(3) format, detect descrypt with valid vs. invalid salts as separate id's for our heuristics on supported hash types.
* Introduced a number of optimizations for faster handling of large password hash files, including loading, cracking, and "--show". Some of these use more memory than before, yet in a more efficient manner.
* Benchmark using all-different candidate passwords of length 7 by default.
* Dropped undocumented special handling of "Mc" in 'c' and 'C' rule commands.
* Dropped undocumented limitation of the 'M' and 'Q' rule commands where they would sometimes memorize/check only up to the current hash type's length limit yet this optimization wouldn't necessarily be transparent (e.g., if a later command would extract a substring from above the hash type's length limit and bring it to within the limit).
* Implemented special-case handling of repeated rule commands '$', '^', '[', ']', '{', and '}', as well as faster handling of the 'D' command.
* When built with "--fork" support, disallow session names with all-digit suffixes since these clash with those produced by "--fork".
* Forward SIGTERM to --fork'ed children.
* Set stdout to line buffered (rather than potentially fully buffered), except for "--stdout", "--show", and auxiliary programs such as "unshadow".
* On Windows, restore normal processing of Ctrl-C in case our parent (such as Johnny the GUI) had disabled it.
* Added linux-x86*-avx512 and linux-x86*-avx2 make targets, which use respectively AVX-512 and AVX2 for bitslice DES.
* Added linux-mic make target for Intel MIC (first generation Xeon Phi, aka Knights Corner), which uses its 512-bit SIMD intrinsics for bitslice DES. (For second generation Xeon Phi, aka Knights Landing, use linux-x86-64-avx512.)
* Added linux-arm64le, linux-arm32le-neon, and linux-arm32le make targets. (The first two of these make use of ASIMD or NEON for bitslice DES.)
* Added linux-sparc64 make target.
* Made a minor optimization to MMX and SSE2 assembly code for LM hash.
* Dropped Ultrix and SCO support.
* Don't probe for alternate config file names (like john.ini when on Unix).
* "DokuWiki" external mode sample has been added to the default john.conf.
* Fixed operator precedence in the external mode compiler to be the same as C.
* Fixed an out of bounds write bug in the external mode virtual machine.
* Fixed a bug introduced in version 1.7.4 in the wordlist rules engine, where some sequences of rule commands could overflow a word buffer.
* Fixed a bug where unaligned access SSE/AVX instructions would unnecessarily be generated by GCC 4.6+ in the bitslice DES code in non-OpenMP builds.
* Fixed a bug where "Warning: no OpenMP support for this hash type" could be printed in "--stdout" mode.
* Made assorted other bugfixes, portability and documentation enhancements.

The following changes have been made between John 1.7.9.8 and 1.8.0:

* Revised the incremental mode to let the current character counts grow for each character position independently, with the aim to improve efficiency in terms of successful guesses per candidate passwords tested.
* Revised the pre-defined incremental modes, as well as external mode filters that are used to generate .chr files.
* Added makechr, a script to (re-)generate .chr files.
* Enhanced the status reporting to include four distinct speed metrics (g/s, p/s, c/s, and C/s).
* Added the "--fork=N" and "--node=MIN[-MAX]/TOTAL" options for trivial parallel and distributed processing.
* In the external mode compiler, treat character literals as unsigned.
* Renamed many of the formats.
* Updated the documentation.
* Relaxed the license for many source files to cut-down BSD.
* Relaxed the license for John the Ripper as a whole from GPLv2 (exact version) to GPLv2 or newer with optional OpenSSL and unRAR exceptions.
* Assorted other changes have been made.

The following changes have been made between John 1.7.9 and 1.7.9.8:

* Enhanced the support for DES-based tripcodes by making use of the bitslice DES implementation and supporting OpenMP parallelization.
* Implemented bitmaps for fast initial comparison of computed hashes against those loaded for cracking. This is applied before hash table lookups, and it allows for the use of smaller hash tables (thereby saving memory) while achieving the same or greater speed that larger hash tables previously did. The speed increase is due to improved locality of reference (where only the smaller bitmap is accessed all the time, whereas the larger hash table behind it is only accessed for a percentage of comparisons and additionally it is smaller than it would otherwise need to be).
* Tuned the bitmap and hash table sizes and thresholds based on testing on saltless hashes on a Core 2'ish CPU.
* When cracking LM hashes, don't store the ASCII encodings of the hashes in memory, but instead reconstruct them from the binary hashes for writing into john.pot when a password gets cracked.
* With 32-bit x86 builds and at least MMX enabled, the "two hashes at a time" code for bcrypt is now enabled for GCC 4.2 and newer. This change is made based on benchmark results for different builds made with different versions of GCC on CPUs ranging from Pentium 3 to Core i7. Unfortunately, there's a known performance regression with this change on Atom. Previously, this code was only enabled for x86-64 and/or OpenMP-enabled builds.
* The formats interface has been enhanced to better support GPU implementations (in jumbo), as well as fast hashes on multi-CPU systems (not yet made use of).
* Assorted minor corrections to Cygwin builds were made.
* Fixed a bug in the Keyboard external mode (uninitialized variables on "--restore" or when minlength is greater than 1).
* Enhanced the generic crypt(3) format to handle possible NULL returns from crypt() and crypt_r().
* Updated the FAQ.

The following changes have been made between John 1.7.8 and 1.7.9:

* Added optional parallelization of the MD5-based crypt(3) code with OpenMP.
* Added optional parallelization of the bitslice DES code with OpenMP.
* Replaced the bitslice DES key setup algorithm with a faster one, which significantly improves performance at LM hashes, as well as at DES-based crypt(3) hashes when there's just one salt (or very few salts).
* Optimized the DES S-box x86-64 (16-register SSE2) assembly code.
* Added support for 10-character DES-based tripcodes (not optimized yet).
* Added support for the "$2y$" prefix of bcrypt hashes.
* Added two more hash table sizes (16M and 128M entries) for faster processing of very large numbers of hashes per salt (over 1M).
* Added two pre-defined external mode variables: "abort" and "status", which let an external mode request the current cracking session to be aborted or the status line to be displayed, respectively.
* Made some minor optimizations to external mode function calls and virtual machine implementation.
* The "--make-charset" option now uses floating-point rather than 64-bit integer operations, which allows for larger CHARSET_* settings in params.h.
* Added runtime detection of Intel AVX and AMD XOP instruction set extensions, with optional fallback to an alternate program binary.
* In OpenMP-enabled builds, added support for fallback to a non-OpenMP build when the requested thread count is 1.
* Added relbench, a Perl script to compare two "john --test" benchmark runs, such as for different machines, "make" targets, C compilers, optimization options, or/and versions of John the Ripper.
* Additional public lists of "top N passwords" have been merged into the bundled common passwords list, and some insufficiently common passwords were removed from the list.
* Many minor enhancements and a few bug fixes were made.

The following changes have been made between John 1.7.7 and 1.7.8:

* The bitslice DES S-box expressions have been replaced with those generated by Roman Rusakov specifically for John the Ripper. The corresponding assembly code for x86 with MMX, SSE2, and for x86-64 with SSE2 has been re-generated. For other CPUs and for AVX/XOP, C compilers do a reasonably good job of generating the code from the supplied C source files (with intrinsics where relevant). The S-box expressions that we were using before had a 21% larger gate count, so theoretically this could provide a 21% speedup. In practice, though, a 12% to 14% speedup at DES-based crypt(3) hashes is typical. This effort has been sponsored by Rapid7: https://www.rapid7.com
* Corrected support for bcrypt (OpenBSD Blowfish) hashes of passwords containing non-ASCII characters (that is, characters with the 8th bit set). Added support for such hashes produced by crypt_blowfish up to 1.0.4, which contained a sign extension bug (inherited from older versions of John). The old buggy behavior may be enabled per-hash, using the "$2x$" prefix.
* The external mode virtual machine's performance has been improved through additional multi-op instructions matching common instruction sequences (assign-pop and some triple- and quad-push VM instructions were added).
* A few minor bug fixes and enhancements were made.

The following changes have been made between John 1.7.6.1 and 1.7.7:

* Added Intel AVX and AMD XOP instruction sets support for bitslice DES (with C compiler intrinsics). New make targets: linux-x86-64-avx, linux-x86-64-xop, linux-x86-avx, and linux-x86-xop (these require recent versions of GCC and GNU binutils).
* A "dummy" "format" is now supported (plaintext passwords encoded in hexadecimal and prefixed with "$dummy$") - for faster testing and tuning of custom wordlists, rule sets, .chr files, and external modes on already known or artificial passwords, as well as for testing of future and modified versions of John itself.
* Apache "$apr1$" MD5-based password hashes are now supported along with the FreeBSD-style MD5-based crypt(3) hashes that were supported previously. Hashes of both of these types may be loaded for cracking simultaneously.
* The "--salts" option threshold is now applied before removal of previously cracked hashes for consistent behavior with interrupted and continued sessions.
* The "Idle = Y" setting (which is the default) is now ignored for OpenMP-enabled hash types when the actual number of threads is greater than 1. (Unfortunately, it did not work right at least with GNU libgomp on Linux.)
* When a cracking session terminates or is interrupted, John will now warn the user if the cracked passwords printed to the terminal while cracking are potentially incomplete. It will advise the user to use the "--show" option to see the complete set of cracked passwords with proper post-processing.
* When loading hashes specified on a line on their own (feature introduced in 1.7.6), the loader will now ignore leading and trailing whitespace.
* Unless a hash type is forced from the command line, the loader will now print warnings about additional hash types seen in the input files (beyond the hash type autodetected initially).
* For use primarily by the jumbo patch (and later by future enhancements to the official versions as well), the loader now includes logic to warn the user of ambiguous hash encodings (e.g. LM vs. NTLM vs. raw-MD5, all of which may be represented as 32 hexadecimal characters) and of excessive partial hash collisions, which it works around (these are typically caused by an incomplete implementation of a new hash type).
* The "unique" and "unshadow" programs have been made significantly faster.
* "DateTime", "Repeats", "Subsets", "AtLeast1-Simple", "AtLeast1-Generic", and "Policy" external mode samples have been added to the default john.conf.
* The self-tests have been enhanced to detect more kinds of program bugs.
* A few minor bug fixes and enhancements were made.

The following changes have been made between John 1.7.6 and 1.7.6.1:

* Corrected a logic error introduced in JtR 1.7.4.2: in "single crack" mode, we need a salt's key buffer even when we have no words corresponding to that salt's hashes to base candidate passwords on. We need this buffer to hold other salts' successful guesses for testing against this salt's hashes.

The following changes have been made between John 1.7.5.1 and 1.7.6:

* Generic crypt(3) support (enabled with "--format=crypt") has been added for auditing password hash types supported by the system but not yet supported by John's own optimized cryptographic routines (such as SHA-crypt and SunMD5).
* Optional parallelization of the above has been implemented by means of OpenMP along with glibc's crypt_r(3) or Solaris' MT-safe crypt(3C).
* Optional parallelization of John's own optimized code for the OpenBSD-style Blowfish-based crypt(3) (bcrypt) hashes with OpenMP has been added.
* A more suitable version of 32-bit x86 assembly code for Blowfish is now chosen on Core i7 and similar CPUs (when they happen to run a 32-bit build).
* More optimal DES S-box expressions for PowerPC with AltiVec (making use of the conditional select operation) contributed by Dumplinger Boy (Dango-Chu) have been integrated.
* The bitslice DES C source code has been reworked to allow for the use of arbitrary SIMD intrinsics, which was previously only implemented for AltiVec as a special case.
* Support for SSE2 and MMX intrinsics with bitslice DES (as an alternative to the supplied assembly code) has been added (currently only enabled for SSE2 on x86-64 when compiling with GCC 4.4+).
* Support for mixed-type longer virtual vectors (such as SSE2+MMX, SSE2+ALU, AltiVec+ALU, and other combinations) with bitslice DES has been added (not enabled by default yet, primarily intended for easy benchmarks on future CPUs, with future compiler versions, with even more SIMD instruction sets, and with different DES S-box expressions that might be available in the future).
* The obsolete 32-bit SPARC assembly implementation of DES has been dropped.
* The loader will now detect password hashes specified on a line on their own, not only as part of an /etc/passwd or PWDUMP format file.
* When run in "--stdin" mode and reading candidate passwords from a terminal (to be typed by the user), John will no longer mess with the terminal settings.
* John will now restore terminal settings not only on normal termination or interrupt, but also when forcibly interrupted with two Ctrl-C keypresses.

The following changes have been made between John 1.7.5 and 1.7.5.1:

* A new numeric variable has been added to the word mangling rules engine: "p" for position of the character last found with the "/" or "%" commands.

The following changes have been made between John 1.7.4.2 and 1.7.5:

* Support for the use of "--format" along with "--show" or "--make-charset" has been added.
* The choice of .rec and .log filenames for custom session names has been made more intuitive.
* Support for "\r" (character lists with repeats) and "\p0" (reference to the immediately preceding character list/range) has been added to the word mangling rules preprocessor.
* The undefined and undocumented behavior of some subtle word mangling rules preprocessor constructs has been changed to arguably be more sensible.
* Some bugs were fixed, most notably JtR crashing on no password hashes loaded (bug introduced in 1.7.4.2).

The following changes have been made between John 1.7.4 and 1.7.4.2:

* Major performance improvements for processing of very large password files or sets of files, especially with salt-less or same-salt hashes, achieved primarily through introduction of two additional hash table sizes (64K and 1M entries), changes to the loader, and smarter processing of successful guesses (to accommodate getting thousands of hashes successfully cracked per second).
* Many default buffer and hash table sizes have been increased and thresholds for the use of hash tables lowered, meaning that John will now tend to use more memory to achieve better speed (unless it is told not to with the "--save-memory" option).
* Some previously missed common website passwords found on public lists of "top N passwords" have been added to the bundled common passwords list.
* Some bugs introduced in 1.7.4 and affecting wordlist mode's elimination of consecutive duplicate candidate passwords have been fixed.

The following changes have been made between John 1.7.3.4 and 1.7.4:

* Support for back-references and "parallel" ranges has been added to the word mangling rules preprocessor.
* The notion of numeric variables (to be used for character positions and substring lengths along with numeric constants supported previously) has been introduced into the rules engine. Two pre-defined variables ("l" for initial or updated word's length and "m" for initial or memorized word's last character position) and 11 user-defined variables ("a" through "k") have been added. Additionally, there's a new numeric constant: "z" for "infinite" position or length.
* New rule commands have been added: "A" (append, insert, or prefix with a string), "X" (extract a substring from memory and insert), "v" (subtract and assign to a numeric variable).
* New rule reject flags have been added: ":" (no-op, for use along with the "parallel" ranges feature of the preprocessor) and "p" (reject unless word pair commands are allowed, for sharing of the same ruleset between "single crack" and wordlist modes).
* Processing of word mangling rules has been made significantly faster in multiple ways (caching of the current length, less copying of data, code and data placement changes for better branch prediction and L1 cache usage, compiler-friendly use of local variables, code micro-optimizations, removal of no-op rule commands in an initial pass).
* The default rulesets for "single crack" and wordlist modes have been revised to make use of the new features, for speed, to produce fewer duplicates, and to attempt additional kinds of candidate passwords (such as for years 2010 through 2019 with "year-based" rules).
* The idle priority emulation code has been optimized for lower overhead when there appears to be no other demand for CPU time.
* The default for the Idle setting has been changed from N to Y.

The following changes have been made between John 1.7.3.1 and 1.7.3.4:

* "make check" has been implemented (for Unix-like systems only).
* The "--test" option will now take an optional argument - the duration of each benchmark in seconds.
* Section .note.GNU-stack has been added to all assembly files to avoid the stack area unnecessarily being made executable on Linux systems that use this mechanism.
* Some very minor bugs that did not affect normal operation have been fixed.
* Some unimportant compiler warnings have been fixed, a source code comment has been made more verbose and more complete.

The following changes have been made between John 1.7.3 and 1.7.3.1:

* Corrected the x86 assembly files for building on Mac OS X.
* Merged in some generic changes from JtR Pro.

The following changes have been made between John 1.7.2 and 1.7.3:

* Two Blowfish-based crypt(3) hashes may now be computed in parallel for much better performance on modern multi-issue CPUs with a sufficient number of registers (e.g., x86-64).
* Bitslice DES assembly code for x86-64 has been converted to use instruction pointer relative addressing (needed for Mac OS X support).
* New make targets: macosx-universal, macosx-x86-64, solaris-x86-64-cc, solaris-x86-64-gcc, solaris-x86-sse2-cc, solaris-x86-sse2-gcc, solaris-x86-mmx-cc, solaris-x86-mmx-gcc, solaris-x86-any-cc, linux-ia64; other changes to the Makefile.
* Minor bug fixes.
* "DumbForce" and "KnownForce" external mode samples have been added to the default john.conf.

The following changes have been made between John 1.7.1 and 1.7.2:

* Bitslice DES assembly code for x86-64 making use of the 64-bit mode extended SSE2 with 16 XMM registers has been added for better performance at DES-based crypt(3) hashes with x86-64 builds on AMD processors.
* New make target for FreeBSD/x86-64.

The following changes have been made between John 1.7.0.2 and 1.7.1:

* Bitslice DES code for x86 with SSE2 has been added for better performance at DES-based crypt(3) hashes on Pentium 4 and SSE2-capable AMD processors.
* Assorted high-level changes have been applied to improve performance on current x86-64 processors.
* New make target for NetBSD/SPARC64.
* Minor source code cleanups.

The following changes have been made between John 1.7 and 1.7.0.2:

* Minor bug and portability fixes.
* Better handling of certain uncommon scenarios and improper uses of John.
* Bonus: "Keyboard" cracker included in the default john.conf (john.ini) that will try sequences of adjacent keys on a keyboard as passwords.

The following changes have been made between John 1.6 and 1.7:

* Bitslice DES code for x86 with MMX: more than twice faster than older non-bitslice MMX code.
* Bitsliced the LM hash code as well: now several times faster.
* Significant improvements to the generic bitslice DES code: +20% on RISC.
* PowerPC G4+ AltiVec support (Mac OS X and Linux): effective 128-bitness for bitslice DES, resulting in huge speedups.
* First attempt at generic vectorization support for bitslice DES.
* Two MD5 hashes at a time for extra ILP on RISC: up to +80% on Alpha EV5+.
* Generic Blowfish x86 assembly code in addition to the original Pentium version: +15% on the Pentium Pro family (up to and including Pentium III), +20% on AMD K6 (Pentium 4 and newer AMD CPUs are more happy running the original Pentium code for Blowfish).
* Verbose logging of events to the global or a session-specific log file.
* Better idle priority emulation with POSIX.1b (POSIX.4) scheduling calls.
* System-wide installation support for *BSD ports and Linux distributions.
* AIX, DU/Tru64 C2, HP-UX tcb files support in unshadow.
* New make targets for Linux/x86-64, Linux/PowerPC, FreeBSD/Alpha, OpenBSD/x86-64, OpenBSD/Alpha, OpenBSD/SPARC, OpenBSD/SPARC64, OpenBSD/PowerPC, OpenBSD/PA-RISC, OpenBSD/VAX, NetBSD/VAX, Solaris/SPARC64, Mac OS X (PowerPC and x86), SCO, BeOS.
* Bug and portability fixes, and new bugs.
* Bonus: "Strip" cracker included in the default john.conf (john.ini).

$Owl: Owl/packages/john/john/doc/CHANGES,v 1.117 2019/04/12 01:03:53 solar Exp $