Check if an ICE is reproducible

Sometimes gcc crashes due to external reasons not directly related to gcc:

• out-of-memory condition (check dmesg)
• bad RAM modules (sys-apps/memtest86+)

Make sure the above is not your case.

Look at build.log with ICE and try to resume the build to see if the same command causes the same ICE. Usually command looks something like make --jobs=8 --load-average=8 CPPFLAGS= CFLAGS= LDFLAGS=.

Go to Working directory (specified in build.log as well) and rerun build command. You should see exact command and the failure.

Our running example will be ICE on dev-lang/python-3.6.5:

running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers
building 'cmath' extension
gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c /dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c -o build/temp.linux-i686-3.6/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.o
In file included from /dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c:11:0:
/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h: In function 'cmath_acos':
/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h:45:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.gentoo.org/> for instructions.

The build fails every time LANG=C make is reran. Yay!

Extract exact command

If build system already prints exact command just re-execute it (make sure you are in correct directory). If build system does not print the command you will have to use other methods of getting command like running strace or passing verbose flags manually.

In case of python command is printed as-is:

In file included from /dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/cmathmodule.c:11:0:
/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h: In function 'cmath_acos':
/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Modules/clinic/cmathmodule.c.h:45:1: internal compiler error: Segmentation fault
 }
 ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://bugs.gentoo.org/> for instructions.

Extract self-contained source (use -save-temps)

Add -save-temps to gcc command on a previous step to extract preprocessed sample. foo.c will create foo.i (or foo.ii for c++).

Obligatory python example:

... same failure ...

The preprocessed file is in cmathmodule.i now. You can get more tips at https://gcc.gnu.org/bugs/

Expand -march=native, exact gcc version and other system-specific options

Some compiler options like -march=native are depend on the environment. You need to resolve them into exact options. One way to do it is to query gcc for expansion. Pick your arch instead of arch=sandybridge to avoid diff on march= value. That will minimize the diff.

--- /dev/fd/63	2020-06-10 22:26:45.832792880 +0100
+++ /dev/fd/62	2020-06-10 22:26:45.832792880 +0100
@@ -70,3 +70,3 @@
-  --param=l1-cache-line-size= 		32
-  --param=l1-cache-size=      		64
-  --param=l2-cache-size=      		512
+  --param=l1-cache-line-size= 		64
+  --param=l1-cache-size=      		32
+  --param=l2-cache-size=      		8192
--- /dev/fd/61	2020-06-10 22:26:45.856792856 +0100
+++ /dev/fd/60	2020-06-10 22:26:45.856792856 +0100
@@ -16 +16 @@
-  -maes                       		[disabled]
+  -maes                       		[enabled]
@@ -26 +26 @@
-  -mavx                       		[disabled]
+  -mavx                       		[enabled]
...

Here -march=native is substituted for -march=sandybridge --param=l1-cache-line-size=64 --param=l1-cache-size=32 --param=l2-cache-size=8192 -maes -mavx .... Make sure bug is reproducible after substitution as well.

Report bug on bugs.gentoo.org

File a bug at https://bugs.gentoo.org/ against toolchain@gentoo.org and provide a few details:

• gcc -v output
• attach .i reproducer

[bonus] minimize needed flags to reproduce failure

You can do similar expansion for optimizer options as well: LANG=C gcc -O2 -Q --help=optimizers

To expand -O3 into -O2 + extra you can use diff -U0 <(LANG=C gcc -O2 -Q --help=optimizers) <(LANG=C gcc -O3 -Q --help=optimizers)

Try to find minimal set of flags needed to trigger the crash by removing some expanded options.

[bonus] minimize self-contained source using cvise

Now to the fun part! Let's shrink .i down to a manageable size with help of https://github.com/marxin/cvise.

1. root #emerge --ask dev-util/cvise
2. write a test.sh tester:
   FILE test.sh

   <syntaxhighlight lang="bash">LANG=C gcc-7.3.0 -pthread -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -march=prescott -mfpmath=sse -O2 -pipe -fomit-frame-pointer -fwrapv -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I./Include -I. -I/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5/Include -I/dev/shm/portage/dev-lang/python-3.6.5/work/Python-3.6.5 -c cmathmodule.i -o cmathmodule.o \
   >gcc_out.txt 2>&1
   grep "internal compiler error" gcc_out.txt >/dev/null 2>&1</syntaxhighlight>

3. validate tester
   user $./test.sh

   user $echo $?

   0

4. Reduce!

===< 2430 >===
running 4 interestingness tests in parallel
===< pass_includes :: 0 >===
===< pass_unifdef :: 0 >===
===< pass_comments :: 0 >===
===< pass_blank :: 0 >===
(1.2 %, 733906 bytes)
(27.4 %, 539234 bytes)
===< pass_clang_binsrch :: replace-function-def-with-decl >===
(29.5 %, 523834 bytes)
(30.2 %, 518466 bytes)
(32.0 %, 505350 bytes)
...
          ******** cmathmodule.i ********

typedef struct {
  double a
} b;
b c;
d, e, f;
g() {
  _setjmp();
  b h;
  if (e)
    h.a = copysign(f, d);
  c = h;
}

Done!

cmathmodule.i contains the same shrunk example.

You can also pass the tester command directly to cvise without creating a shell script, e.g.:

[bonus] Check if bug still exists on vanilla gcc master

Building gcc locally could look like that:

Look at configure options of gentoo's gcc -v to get more options to add to defaults. Be careful: some configure options like --enable-default-pie --enable-default-ssp can change code generation enough to trigger or hide a bug.

[bonus] Report bug on gcc.gnu.org/bugzilla

File the bug at https://gcc.gnu.org/bugzilla/ with attached minimal reproducer and gcc -v detals.

[bonus] Extract gcc backtrace

Example session would be:

(gdb) set follow-fork-mode child 
(gdb) break internal_error
(gdb) run
...
Thread 2.1 "cc1" received signal SIGSEGV, Segmentation fault.
[Switching to process 17286]
0x085375d4 in lra_eliminate_reg_if_possible(rtx_def**) ()
(gdb) bt
#0  0x085375d4 in lra_eliminate_reg_if_possible(rtx_def**) ()
...

[bonus] Fix gcc bug

By now you have a source tree and command how to build it!

A few starting points for you:

• https://gcc.gnu.org/contribute.html for general notes on coding tips and guidelines
• https://gcc.gnu.org/onlinedocs/

Good luck!