[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20120127204835.GB8898@openwall.com>
Date: Sat, 28 Jan 2012 00:48:35 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: unique using more than 2 GB RAM
magnum, Jim -
Here's a wish list item:
We should make "unique" able to use more than 2 GB of memory. As it
turns out, "unique" at 2 GB is about twice slower than "sort -u -S 14G"
(on a 16 GB RAM machine), although of course this may vary by input
data. Maybe "unique" should start using 40-bit offsets (good for up to
1 TB of RAM). I am concerned that it will become slightly less
efficient (in terms of both speed and memory usage) when this new
functionality is not being made use of, though.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
