[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150422180559.GC22988@openwall.com>
Date: Wed, 22 Apr 2015 21:05:59 +0300
From: Aleksey Cherepanov <lyosha@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Johnny: core/jumbo differences
On Wed, Apr 22, 2015 at 10:17:46AM +0300, Shinnok wrote:
> After reading your quite good breakdown and your question regarding show stoppers, I think Johnny could potentially differentiate itself from CLI usage in the following ways:
> 1. Makes session management a breeze
> 2. Makes concurrent cracking management a breeze(on a single machine)
> 3. Makes wordlist management easier(mulitple, concat, dedup, reseed from pots, etc..)
> 4. Makes hash management(inclusion, exclusion, dedup, concat, merge, recognition) easier
> 5. Simplifies the path towards testing several attacks modes quickly(rules, formats, modes)
>
> See my other thoughts embedded bellow.
>
> > On Apr 22, 2015, at 12:08 AM, Aleksey Cherepanov <lyosha@...nwall.com> wrote:
> >
> > On Tue, Apr 21, 2015 at 05:18:22PM +0300, Aleksey Cherepanov wrote:
> >> I've tried to write down exhaustive list of differences between john
> >> core and john jumbo. I don't know everything, so feel free to comment.
> >>
> >> This list does not cover usage of the differences in Johnny. I'll make
> >> a follow up with ideas.
> >
> > Some ideas for all those options, not prioritized. I am not sure what
> > should fall under "jumbo support" and "*2john". What options are
> > needed in johnny to be not so limited for jumbo?
> >
> > What are show-stoppers that prevent johnny from being used much?
> >
> > After working through jumbo options, I don't think that jumbo is very
> > different: there are 3 new cracking modes, there are a lot of options
> > for tweaking, there is --list= interface to improve interactive gui
> > features. That's all, is not it?
> >
> > Having -fork, I'd say Johnny lacks ability to attack one file with
> > several attacks at the same time, attack multiple files with one/many
> > attacks (BTW john can attack several files in one invocation: like
> > `john *.pw`). Also I fing inconvenient the way newly cracked passwords
> > are shown: user have to monitor the log page. It is not very different
> > from cmd line john (or at least it differs from my expectations from
> > the gui).
>
> I think this answers my question regarding multiple john instances on a single machine question in the other thread.
> I gather it that there's a need to attack a hash set(or multiple sets) using different attacks at the same time in a hope that one will yield confirmation or results sooner than another, something like profiling for the correct attack, is this correct?
Rather yes. Though I am not sure how important it is.
But there may be small attacks (small wordlists and/or small rules
sets) that may be easier to dispatch at the same time. Though in this
case, user may want a queue of attacks.
> > BTW I found quite confusing: when short wordlist finishes its work,
> > just start/pause buttons switches, there is no big banner "the attack
> > is finished!!!!".
> >
> > If the log is so important then it may be shown all the time as a
> > small region with 1-5 lines of the log. The log may be not from cli
> > but from events to show only really important things to the user.
> >
> > (There is no border between fantasies and really important features in
> > this text.)
> >
> >
> >> * Common CLI options without differences
> >
> >> * Common CLI Options
> >>
> >> The format:
> >> core option
> >> jumbo option
> >> comments
> >>
> >> ----------------------------------------
> >>
> >> --single "single crack" mode
> >>
> >> --single[=SECTION] "single crack" mode
> >>
> >> It is possible to choose Rules section to run with single mode.
> >> [List.Rules:Single] is the default in both versions (so --single is
> >> equivalent to --single=Single; though parameters are not case
> >> sensitive).
> >
> > and
> >
> >> --rules enable word mangling rules for wordlist mode
> >>
> >> --rules[=SECTION] enable word mangling rules for wordlist modes
> >>
> >> $ ./core-1.8.0/run/john --rules=NT --wordlist=~/d/t.pw ~/d/t.pw
> >> Extra parameter for option: "--rules=NT"
> >>
> >> Core does not allow rules section to be specified. Both versions
> >> default to [List.Rules:Wordlist]
> >
> > Currently johnny does not allow to choose rules section for single
> > mode and wordlist mode. `john --list=rules` can be used to get the
> > list of rules sections. Though single mode has additional rules, I am
> > not sure how they affect wordlist mode and if rules for wordlist mode
> > are good for single mode.
> >
> >> ----------------------------------------
> >>
> >> --wordlist=FILE --stdin wordlist mode, read words from FILE or stdin
> >>
> >> --wordlist[=FILE] --stdin wordlist mode, read words from FILE or stdin
> >> --pipe like --stdin, but bulk reads, and allows rules
> >>
> >> $ ./core-1.8.0/run/john --wordlist ~/d/t.pw
> >> Option requires a parameter: "--wordlist"
> >>
> >> The default wordlist is used by default in jumbo:
> >> [Options]
> >> # Wordlist file name, to be used in batch mode
> >> Wordlist = $JOHN/password.lst
> >>
> >> Also jumbo has --stdin and --pipe options to read candidates from
> >> standard input.
> >
> > It may be possible to put john's default wordlist as the default for
> > this option in johnny. It may be useful to remember wordlists used by
> > user between restarts of johnny.
>
> Agree.
>
> >
> >> ----------------------------------------
> >>
> >> --show show cracked passwords
> >>
> >> --show[=LEFT] show cracked passwords [if =LEFT, then uncracked]
> >>
> >> In jumbo --show can print remaining hashes but it drops dupes (when
> >> there is 1 canonical hash for 2 users and the hash is in different
> >> forms and/or there are different gecos).
> >>
> >> ----------------------------------------
> >>
> >> --salts=[-]N load salts with[out] at least N passwords only
> >>
> >> --salts=[-]COUNT[:MAX] load salts with[out] COUNT [to MAX] hashes
> >
> > That's an option to choose hashes. Like other options to choose hashes
> > to be loaded, it may be implemented as filter for table view.
> >
> >> ----------------------------------------
> >>
> >> --format=NAME force hash type NAME: descrypt/bsdicrypt/md5crypt/
> >> bcrypt/LM/AFS/tripcode/dummy/crypt
> >>
> >> --format=NAME force hash of type NAME. The supported formats can
> >> be seen with --list=formats and --list=subformats
> >>
> >> Format list is much larger in jumbo, also it is possible to specify
> >> hash type for generic crypt through --subformat option.
> >
> > List of format names may be loaded on the fly with `john
> > --list=formats`. Though usually it is not needed to specify the
> > format. There are several cases it is needed:
> > - file contain hashes of different formats, user wants to attack
> > different part of file (running john against file does mean that
> > all hashes are under attack),
> > - hashes are loaded incorrectly: they may be loaded as several formats
> > and the default format is not what user wants,
> > - to choose format's implementation (regular vs -ng, cpu vs -gpu, and
> > others).
> >
> >
> >
> >> * Jumbo-only CLI options
> >>
> >> --loopback[=FILE] like --wordlist, but fetch words from a .pot file
> >
> > It seems to be a separate mode. It may be implemented as a checkbox
> > for wordlist mode.
> >
> >> --dupe-suppression suppress all dupes in wordlist (and force preload)
> >
> > It affects performance only. It may be made as a setting. Or some more
> > clever heuristic may be implemented: use dupe suppression only for
> > slow hashes.
> >
> >> --prince[=FILE] PRINCE mode, read words from FILE
> >> --mask=MASK mask mode using MASK
> >> --markov[=OPTIONS] "Markov" mode (see doc/MARKOV)
> >
> > The cracking modes not supported by johnny yet. Rexgen is one too.
> >
> > It may be useful to make "feature testing" determing capabilities of
> > john in use.
> >
> >> --pot=NAME pot file to use
> >> --list=WHAT list capabilities, see --list=help or doc/OPTIONS
> >> --help print usage summary, just like running the command
> >> without any parameters
> >> --config=FILE use FILE instead of john.conf or john.ini
> >
> >
> >
> >> --mem-file-size=SIZE size threshold for wordlist preload (default 5 MB)
> >
> > It may be done as a setting.
> >
> >> --progress-every=N emit a status line every N seconds
> >> --crack-status emit a status line whenever a password is cracked
> >
> > It may be useful for johnny. Though there may be buffering.
> >
> >> --log-stderr log to screen instead of file
> >
> > It may be helpful to see log in output with option to write found
> > passwords to log because this way it possible to connect the passwords
> > with rules used.
>
> I think the best interface for Johnny is john's stdout and stdin. Monitoring files is not that efficient for our use case.
IO buffering may disappoint you...
> >> --costs=[-]C[:M][,...] load salts with[out] cost value Cn [to Mn] for
> >> tunable cost parameters, see doc/OPTIONS
> >> (comma separated list of values/ranges per param.)
> >
> > Another hash selector.
> >
> >> --field-separator-char=C use 'C' instead of the ':' in input and pot files
> >
> > This option allows to load hashes that are separated not with colon.
> > There are formats that use or allow colon as part of them. Johnny
> > won't open such files nicely.
> >
> > Hm, theoretically there may be a hash with all possible characters
> > alone on a line in input file... Not sure what then.
> >
> >> --max-run-time=N gracefully exit after this many seconds
> >
> > It may be showed to user to limit the work.
> >
> >> --stress-test[=TIME] loop self tests forever
> >
> > Johnny may have a mode/page for benchmarks.
>
> Agreed. It was part of the original design.
It is not very important. Though it may be visually attractive and be
used to "promote" johnny.
> >> --encoding=NAME input encoding (eg. UTF-8, ISO-8859-1). See also
> >> doc/ENCODING and --list=hidden-options.
> >> --input-encoding=NAME input encoding (alias for --encoding)
> >> --internal-encoding=NAME encoding used in rules/masks (see doc/ENCODING)
> >> --target-encoding=NAME output encoding (used by format, see doc/ENCODING)
> >
> > Encoding stuff.
> >
> >> --format=CLASS valid classes: dynamic, cpu
> >> --subformat=FORMAT pick a benchmark format for --format=crypt
> >> --mkpc=N request a lower max. keys per crypt
> >> --min-length=N request a minimum candidate length
> >> --max-length=N request a maximum candidate length
> >> --fix-state-delay=N performance tweak, see doc/OPTIONS
> >> --nolog disables creation and writing to john.log file
> >> --bare-always-valid=C if C is 'Y' or 'y', then the dynamic format will
> >> always treat bare hashes as valid
> >> --keep-guessing try more candidates for cracked hashes (ie. search
> >> for plaintext collisions)
> >> --regen-lost-salts=N regenerate lost salts (see doc/OPTIONS)
> >> --mkv-stats=FILE "Markov" stats file (see doc/MARKOV)
> >> --reject-printable reject printable binaries
> >> --verbosity=N change verbosity (1-5, default 3)
> >> --skip-self-tests skip self tests
> >
> > There are options to tweak john. There may be "advanced user" checkbox
> > to show all of them.
> >
> > I am not sure that it is good to put some of such options to settings
> > page because it seems natural to separate johnny's settings and john's
> > options to be applied always. Also john has options in the config file.
> >
> > Hm, some options have settings in config file:
> > # Emit a status line whenever a password is cracked (this is the same as
> > # passing the --crack-status option flag to john). NOTE: if this is set
> > # to true here, --crack-status will toggle it back to false.
> > CrackStatus = N
>
> This should be the priority order for options:
> 1. Any option set in the GUI(default or explicit) and thus translated to an argument for john cli - first tier
> 2. Any option not set/used in the GUI should use the default from john's config file - second tier
> 3. Any option not set in both GUI and config file is trusted to be handled nice by john - third tier
>
> Is there any reason to suspect that john might not respect this?
John:
- john has config file,
- Some values may be overwritten from command line.
Johnny, at the moment:
- settings:
- have hard-coded defaults,
- on them it applies config,
- then some settings are passed to john,
- options:
- passed to john.
Options and settings do not overlap.
Hm, an editor may be made for john's settings from config (the results
is stored there). This way, the settings apply to cmd invocations too,
but the settings should be ported when user chooses other john.
Thanks!
--
Regards,
Aleksey Cherepanov
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
