libc-coord - [RFC] Resolve inconsistent stack protection implementation between GCC and run-time

Follow @Openwall on Twitter for new release announcements and other news
Message-ID: <CAMe9rOosO3j77Vw=C0dS=KpKvgRJLc_Uj=zFZRrY+seeLhOyXw@mail.gmail.com>
Date: Wed, 27 May 2026 12:34:53 +0800
From: "H.J. Lu" <hjl.tools@...il.com>
To: GCC Patches <gcc-patches@....gnu.org>, libc-coord@...ts.openwall.com, 
	LLVM Dev <llvm-dev@...ts.llvm.org>
Cc: Andrew Pinski <pinskia@...il.com>, Iain Sandoe <idsandoe@...glemail.com>, 
	Joseph Myers <josmyers@...hat.com>, Jason Merrill <jason@...hat.com>
Subject: [RFC] Resolve inconsistent stack protection implementation between
 GCC and run-time

There are some inconsistency issues with stack protection implementation:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=125466

which was added to GCC 4.1 by

commit 7d69de618e732d343228a07d797a30e39a6363f4
Author: Richard Henderson <rth@...hat.com>
Date:   Mon Jun 27 00:41:16 2005 -0700

    c-cppbuiltin.c (c_cpp_builtins): Add __SSP_ALL__ and __SSP__.

The initial commit placed the stack protection guard in libgcc2.c.
Later, it was moved to libssp by

commit 77008252819720c987f11b3dade670e2b3ba09b8
Author: Jakub Jelinek <jakub@...hat.com>
Date:   Sat Jul 2 10:52:21 2005 +0200

    Makefile.def (target_modules): Add libssp.

If TARGET_LIBC_PROVIDES_SSP is defined, libssp is unused and the stack
protection run-time in the C library is used instead.

When -mstack-protector-guard=global is used, the stack protection canary
is placed in an external variable, __stack_chk_guard, whose type is
documented as a pointer and libssp provides __stack_chk_guard as a
pointer.  However, most, if not all, of the C libraries which provide
the stack protection run-time, including glibc and Linux kernel, define
__stack_chk_guard as uintptr_t, not a pointer.

This discrepancy hasn't caused any major issue since uintptr_t and pointer
have the same size.  However, since __stack_chk_guard isn't visible to
the front-end, symbol visibility on __stack_chk_guard in the stack
protection run-time source is ignored.  When __stack_chk_guard was
declared in the C/C++ front-ends by

commit c05b5e5d8cb660ed43159d66fd669c20746d6bea
Author:     H.J. Lu <hjl.tools@...il.com>
AuthorDate: Fri Sep 12 18:52:39 2025 -0700
Commit:     H.J. Lu <hjl.tools@...il.com>
CommitDate: Tue May 5 06:01:23 2026 +0800

    c/c++: Declare stack protection guard as a global symbol

it caused the build issue on Darwin

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=125226

sice it declares __stack_chk_guard as uintptr_t by calling

  lang_hooks.types.type_for_mode (ptr_mode, 1);

to get an integer type for __stack_chk_guard which is declared as a
global symbol of type uintptr_t.  For 32-bit systems, uintptr_t may
be either unsigned int or unsigned long int.  On 32-bit Darwin, we get

$ cat /tmp/x.c
__UINTPTR_TYPE__ __stack_chk_guard = 0x1000;
$ ./xgcc -B./ -S /tmp/x.c -m32
/tmp/x.c:1:18: error: conflicting types for ‘__stack_chk_guard’; have ‘long unsi
gned int’
    1 | __UINTPTR_TYPE__ __stack_chk_guard = 0x1000;
      |                  ^~~~~~~~~~~~~~~~~
cc1: note: previous declaration of ‘__stack_chk_guard’ with type ‘unsigned int’
$

since lang_hooks.types.type_for_mode returns unsigned int while Darwin's
uintptr_t is unsigned long int.  Also we got

cc1: warning: nested extern declaration of '__stack_chk_guard' [-Wnested-externs
]

with -Wnested-externs warning in C since __stack_chk_guard is declared in
function scope, not in file scope.

Another issue with libssp is that the target attribute

__attribute__ ((optimize ("stack-protector-all")))

enables stack protection without -fstack-protector.  As the result, if
libssp is required, we get undefined symbol error at link-time since
libssp isn't used without -fstack-protector:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=125205

To resolve these issues, we can do

1. Link with libssp unconditionally if TARGET_LIBC_PROVIDES_SSP is
undefined.  This should fix the undefined symbol error with

__attribute__ ((optimize ("stack-protector-all")))

2. Move __stack_chk_guard to file scope to avoid -Wnested-externs warning
in C.

3. Change libssp to declare __stack_chk_guard as unsigned long int:

unsigned long int __stack_chk_guard = 0;

for Darwin or add LANG_HOOKS_TYPE_FOR_MODE_KIND to specify signed or
unsigned integer type for pointer and update default_stack_protect_guard
to call

  lang_hooks.types.type_for_mode_kind
    (ptr_mode, 1, KIND_IS_INTEGER_FOR_POINTER);

to get unsigned integer type for pointer.

-- 
H.J.
Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.