john-dev - Undefined behavior in rpp.c: left shift of 1 by 63 places cannot be represented in type 'long'

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <56747030.70009@mailbox.org>
Date: Fri, 18 Dec 2015 21:44:32 +0100
From: Frank Dittrich <frank.dittrich@...lbox.org>
To: john-dev@...ts.openwall.com
Subject: Undefined behavior in rpp.c: left shift of 1 by 63 places cannot be
 represented in type 'long'

Solar,

in 2013 you fixed similar errors found by Alexander Cherepanov in 
DES_Std.c, see commit 223d6f53b8490ce2a074f9e933188e69fdcc11c4 in john's 
master branch.

Now I found the same problem in rpp.c:

rpp.c:32:33: runtime error: left shift of 1 by 63 places cannot be 
represented in type 'long'

This is the offending statement:
ARCH_WORD mask = (ARCH_WORD)1 << (c % ARCH_BITS);


Just in case you want to reproduce, these are my Makefile changes to see 
these errors in a linux-x86-64-avx build:

diff --git a/src/Makefile b/src/Makefile
index 9d0e364..70eec41 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -8,7 +8,7 @@
  # There's ABSOLUTELY NO WARRANTY, express or implied.
  #

-CC = gcc
+CC = clang
  AS = $(CC)
  LD = $(CC)
  CPP = $(CC)
@@ -28,11 +28,11 @@ OMPFLAGS =
  #OMPFLAGS = -fopenmp -D_FORTIFY_SOURCE=0
  # Sun Studio with OpenMP (set the OMP_NUM_THREADS env var at runtime)
  #OMPFLAGS = -xopenmp
-CFLAGS = -c -Wall -Wdeclaration-after-statement -O2 
-fomit-frame-pointer $(OMPFLAGS)
+CFLAGS = -c -Wall -Wdeclaration-after-statement -O2 
-fomit-frame-pointer $(OMPFLAGS) -fsanitize=undefined 
-fno-sanitize=alignment
  # CFLAGS for use on the main john.c file only
  CFLAGS_MAIN = $(CFLAGS)
  ASFLAGS = -c $(OMPFLAGS)
-LDFLAGS = -s $(OMPFLAGS)
+LDFLAGS = -s $(OMPFLAGS) -fsanitize=undefined -fno-sanitize=alignment
  OPT_NORMAL = -funroll-loops
  # Remove the "-Os" if you're using an ancient version of gcc
  OPT_INLINE = -Os -funroll-loops -finline-functions

There's also a github issue (in bleeding-jumbo that's in line 34):
https://github.com/magnumripper/JohnTheRipper/issues/1963

Frank

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.