[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150421210839.GA9777@openwall.com>
Date: Wed, 22 Apr 2015 00:08:39 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Johnny: core/jumbo differences
On Tue, Apr 21, 2015 at 05:18:22PM +0300, Aleksey Cherepanov wrote:
> I've tried to write down exhaustive list of differences between john
> core and john jumbo. I don't know everything, so feel free to comment.
>
> This list does not cover usage of the differences in Johnny. I'll make
> a follow up with ideas.
Some ideas for all those options, not prioritized. I am not sure what
should fall under "jumbo support" and "*2john". What options are
needed in johnny to be not so limited for jumbo?
What are show-stoppers that prevent johnny from being used much?
After working through jumbo options, I don't think that jumbo is very
different: there are 3 new cracking modes, there are a lot of options
for tweaking, there is --list= interface to improve interactive gui
features. That's all, is not it?
Having -fork, I'd say Johnny lacks ability to attack one file with
several attacks at the same time, attack multiple files with one/many
attacks (BTW john can attack several files in one invocation: like
`john *.pw`). Also I fing inconvenient the way newly cracked passwords
are shown: user have to monitor the log page. It is not very different
from cmd line john (or at least it differs from my expectations from
the gui).
BTW I found quite confusing: when short wordlist finishes its work,
just start/pause buttons switches, there is no big banner "the attack
is finished!!!!".
If the log is so important then it may be shown all the time as a
small region with 1-5 lines of the log. The log may be not from cli
but from events to show only really important things to the user.
(There is no border between fantasies and really important features in
this text.)
> * Common CLI options without differences
> * Common CLI Options
>
> The format:
> core option
> jumbo option
> comments
>
> ----------------------------------------
>
> --single "single crack" mode
>
> --single[=SECTION] "single crack" mode
>
> It is possible to choose Rules section to run with single mode.
> [List.Rules:Single] is the default in both versions (so --single is
> equivalent to --single=Single; though parameters are not case
> sensitive).
and
> --rules enable word mangling rules for wordlist mode
>
> --rules[=SECTION] enable word mangling rules for wordlist modes
>
> $ ./core-1.8.0/run/john --rules=NT --wordlist=~/d/t.pw ~/d/t.pw
> Extra parameter for option: "--rules=NT"
>
> Core does not allow rules section to be specified. Both versions
> default to [List.Rules:Wordlist]
Currently johnny does not allow to choose rules section for single
mode and wordlist mode. `john --list=rules` can be used to get the
list of rules sections. Though single mode has additional rules, I am
not sure how they affect wordlist mode and if rules for wordlist mode
are good for single mode.
> ----------------------------------------
>
> --wordlist=FILE --stdin wordlist mode, read words from FILE or stdin
>
> --wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
> --pipe like --stdin, but bulk reads, and allows rules
>
> $ ./core-1.8.0/run/john --wordlist ~/d/t.pw
> Option requires a parameter: "--wordlist"
>
> The default wordlist is used by default in jumbo:
> [Options]
> # Wordlist file name, to be used in batch mode
> Wordlist = $JOHN/password.lst
>
> Also jumbo has --stdin and --pipe options to read candidates from
> standard input.
It may be possible to put john's default wordlist as the default for
this option in johnny. It may be useful to remember wordlists used by
user between restarts of johnny.
> ----------------------------------------
>
> --show show cracked passwords
>
> --show[=LEFT] show cracked passwords [if =LEFT, then uncracked]
>
> In jumbo --show can print remaining hashes but it drops dupes (when
> there is 1 canonical hash for 2 users and the hash is in different
> forms and/or there are different gecos).
>
> ----------------------------------------
>
> --salts=[-]N load salts with[out] at least N passwords only
>
> --salts=[-]COUNT[:MAX] load salts with[out] COUNT [to MAX] hashes
That's an option to choose hashes. Like other options to choose hashes
to be loaded, it may be implemented as filter for table view.
> ----------------------------------------
>
> --format=NAME force hash type NAME: descrypt/bsdicrypt/md5crypt/
> bcrypt/LM/AFS/tripcode/dummy/crypt
>
> --format=NAME force hash of type NAME. The supported formats can
> be seen with --list=formats and --list=subformats
>
> Format list is much larger in jumbo, also it is possible to specify
> hash type for generic crypt through --subformat option.
List of format names may be loaded on the fly with `john
--list=formats`. Though usually it is not needed to specify the
format. There are several cases it is needed:
- file contain hashes of different formats, user wants to attack
different part of file (running john against file does mean that
all hashes are under attack),
- hashes are loaded incorrectly: they may be loaded as several formats
and the default format is not what user wants,
- to choose format's implementation (regular vs -ng, cpu vs -gpu, and
others).
> * Jumbo-only CLI options
>
> --loopback[=FILE] like --wordlist, but fetch words from a .pot file
It seems to be a separate mode. It may be implemented as a checkbox
for wordlist mode.
> --dupe-suppression suppress all dupes in wordlist (and force preload)
It affects performance only. It may be made as a setting. Or some more
clever heuristic may be implemented: use dupe suppression only for
slow hashes.
> --prince[=FILE] PRINCE mode, read words from FILE
> --mask=MASK mask mode using MASK
> --markov[=OPTIONS] "Markov" mode (see doc/MARKOV)
The cracking modes not supported by johnny yet. Rexgen is one too.
It may be useful to make "feature testing" determing capabilities of
john in use.
> --pot=NAME pot file to use
> --list=WHAT list capabilities, see --list=help or doc/OPTIONS
> --help print usage summary, just like running the command
> without any parameters
> --config=FILE use FILE instead of john.conf or john.ini
> --mem-file-size=SIZE size threshold for wordlist preload (default 5 MB)
It may be done as a setting.
> --progress-every=N emit a status line every N seconds
> --crack-status emit a status line whenever a password is cracked
It may be useful for johnny. Though there may be buffering.
> --log-stderr log to screen instead of file
It may be helpful to see log in output with option to write found
passwords to log because this way it possible to connect the passwords
with rules used.
> --costs=[-]C[:M][,...] load salts with[out] cost value Cn [to Mn] for
> tunable cost parameters, see doc/OPTIONS
> (comma separated list of values/ranges per param.)
Another hash selector.
> --field-separator-char=C use 'C' instead of the ':' in input and pot files
This option allows to load hashes that are separated not with colon.
There are formats that use or allow colon as part of them. Johnny
won't open such files nicely.
Hm, theoretically there may be a hash with all possible characters
alone on a line in input file... Not sure what then.
> --max-run-time=N gracefully exit after this many seconds
It may be showed to user to limit the work.
> --stress-test[=TIME] loop self tests forever
Johnny may have a mode/page for benchmarks.
> --encoding=NAME input encoding (eg. UTF-8, ISO-8859-1). See also
> doc/ENCODING and --list=hidden-options.
> --input-encoding=NAME input encoding (alias for --encoding)
> --internal-encoding=NAME encoding used in rules/masks (see doc/ENCODING)
> --target-encoding=NAME output encoding (used by format, see doc/ENCODING)
Encoding stuff.
> --format=CLASS valid classes: dynamic, cpu
> --subformat=FORMAT pick a benchmark format for --format=crypt
> --mkpc=N request a lower max. keys per crypt
> --min-length=N request a minimum candidate length
> --max-length=N request a maximum candidate length
> --fix-state-delay=N performance tweak, see doc/OPTIONS
> --nolog disables creation and writing to john.log file
> --bare-always-valid=C if C is 'Y' or 'y', then the dynamic format will
> always treat bare hashes as valid
> --keep-guessing try more candidates for cracked hashes (ie. search
> for plaintext collisions)
> --regen-lost-salts=N regenerate lost salts (see doc/OPTIONS)
> --mkv-stats=FILE "Markov" stats file (see doc/MARKOV)
> --reject-printable reject printable binaries
> --verbosity=N change verbosity (1-5, default 3)
> --skip-self-tests skip self tests
There are options to tweak john. There may be "advanced user" checkbox
to show all of them.
I am not sure that it is good to put some of such options to settings
page because it seems natural to separate johnny's settings and john's
options to be applied always. Also john has options in the config file.
Hm, some options have settings in config file:
# Emit a status line whenever a password is cracked (this is the same as
# passing the --crack-status option flag to john). NOTE: if this is set
# to true here, --crack-status will toggle it back to false.
CrackStatus = N
> * Config file
>
> In Jumbo config files may include other files. Core john has only one
> file.
>
> John built from sources searches config file (and other files) in run/
> directory, john installed from package most usually looks into ~/.john
> and /etc/john . While distros usually use john core, there distros
> with jumbo. Also this behaviour may depend on distro.
>
> With jumbo, you can get path to the folder with --list=build-info option:
> $ jumbo/run/john --list=build-info
> [...]
> $JOHN is jumbo/run/
> [...]
The setting to use with --log-stderr in trick mentioned above:
# Write cracked passwords to the log file (default is just the user name)
LogCrackedPasswords = N
> * Utilities and tools
>
> In core
> john - john
> mailer - "script to send mail to all users whose passwords got cracked."
> makechr - utility to make .chr files (incremental mode) from current .pot
> relbench - statistics for benchmarks
> unafs
> unique - filter to remove dupes in wordlist saving order
> unshadow
>
> unafs and unshadow may fall into *2john category.
>
>
> BTW Jumbo has bash auto-completions.
>
>
> In jumbo there are various utilities:
>
> *2john converters (including symlinks):
>
> These tools usually read file(s) and print hashes to stdout in a form
> john understands them. Some utilities perform quite non-trivial
> parsing using pulled-in third party libraries.
>
> 1password2john.py
> 7z2john.py
[...]
> wpapcap2john
> zip2john
>
> undrop
These utilities may be used to "import" files. So to crack a .7z file
with password you don't apply john to the file, you make a hash using
7z2john.py and then apply john on the hash.
It may be tricky to choose the tool automatically. Though some files
have simple connection between their extensions and the tools (.7z
files have .7z on the end, though not all files are so).
The other way to pick the tools is to get the file's format by
magic/signature. On Linux, it is possible to call `file` utility.
Though it may be unsafe (though running the utilities on user's files
may be unsafe).
The utilities are written in various languages. While johnny does not
have anything to do with it, such zoo may be a problem to package john
(e.g. to prepare a bundle john + johnny + all dependencies).
> There are more tools:
>
> base64conv - raw/hex/base64(mime/crypt/cryptBS) converter
> genincstats.rb - make statistics from .inc file
> leet.pl - rules generator for leet speak
> netscreen.py - netscreen hasher
> hextoraw.pl - unhex for pass_gen.pl
> pass_gen.pl - dynamic formats hasher
> sha-dump.pl - ldap dumper
> sha-test.pl - ldap sha1 hasher
> calc_stat - some statistics for wordlist
>
> benchmark-unify
> # John the Ripper benchmark output conversion tool, revision 1
>
> netntlm.pl
> # The purpose of this script is to aid with cracking a LM/NTLM
> # challenge/response set, when part of the password is known. It
> # was written with John's NetLM/NetNTLM formats and "halflmchall"
> # Rainbow Tables in mind.
>
> unrule.pl
> # Extract basewords from list of plains. Based on an embryo from epixoip.
> # ./unrule.pl < rockyou.lst > basewords.lst
>
> (I don't know at all)
> genmkvpwd
> mkvcalcproba
> SIPdump
> tgtsnarf
> cprepair
They does not seem very important now.
Thanks!
--
Regards,
Aleksey Cherepanov
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
