14940 – PCH largefile test fails on various platforms (original) (raw)

Description Ian Lance Taylor 2004-04-13 13:57:22 UTC

The PCH largefile test, in gcc/testsuite/gcc.dg/pch/pch.exp, fails using the default mmap implementation on i386-unknown-netbsdelf1.6.1.

It also fails using the Solaris implementation on sparc-sun-solaris2.9. It fails on i686-pc-cygwin, although actually most of the PCH tests fail on that platform.

A failure in the PCH largefile test for a given platform means that PCH is unstable for that platform. A PCH will work for most source code files but will fail for large source code files. The user can not easily predict when a PCH will work and when it will fail. When a PCH fails, there is no easy way to compile the file without removing the PCH so that gcc does not find it.

On i386-unknown-netbsdelf1.6.1, and probably other platforms, a direct consequence is that qt-x11-free-3.3.0 can not be built. Those sources test whether PCH works, and use it if they do. The test passes, but some of the source files are large. Those source files fail to build.

Comment 1 Drea Pinski 2004-04-13 16:09:15 UTC

Confirmed.

Comment 2 Geoff Keating 2004-04-13 18:56:12 UTC

The Cygwin problems are 11341.

I've added the two BSD maintainers to the CC.

Comment 3 Loren Rittle 2004-04-14 17:52:52 UTC

Confirmed on i386-unknown-freebsd4.9 with yesterday's tree-ssa build.

Comment 4 Drea Pinski 2004-07-17 04:09:22 UTC

*** Bug 16599 has been marked as a duplicate of this bug. ***

Comment 5 Drea Pinski 2004-07-17 04:10:43 UTC

Actually this is target independent and only host dependent.

Comment 6 dave 2004-07-17 04:21:41 UTC

Subject: Re: PCH largefile test fails on various platforms

Actually this is target independent and only host dependent.

Do you think we could set or fallback to MAP_FIXED? If we can't, the mmap call can fail on all hosts and the problem is not host dependent.

Dave

Comment 7 Ian Lance Taylor 2004-07-17 14:20:54 UTC

Subject: Re: PCH largefile test fails on various platforms

Some hosts override the default mmap call. See, e.g., config/host-linux.c. So whether this works is indeed host dependent.

Comment 8 Ian Lance Taylor 2004-07-17 14:43:32 UTC

I should add that always using MAP_FIXED will cause the new mmap to clobber any existing memory maps, which will lead to silent failure if the GC code (or the library malloc code) happens to have mapped any portion of that memory space. So using MAP_FIXED does not work in general. For an example of how it can be used on some systems, see gcc/config/host-solaris.c, which uses mincore() to explicitly check whether anything else is mapped in that address space. But note that that code does not work on, e.g., GNU/Linux, because mincore() does not work as expected--it does not detect anonymous maps.

Comment 9 Eric Botcazou 2004-07-17 15:43:03 UTC

Note also that, although the mincore() trick is functional on Solaris, it doesn't solve the problem in general because the largefile test fails on all versions I tested (Solaris 2.5.1, 2.6, 7, 8 and 9).

Subject: Re: PCH largefile test fails on various platforms

------- Additional Comments From ebotcazou at gcc dot gnu dot org 2004-07-17 15:43 ------- Note also that, although the mincore() trick is functional on Solaris, it doesn't solve the problem in general because the largefile test fails on all versions I tested (Solaris 2.5.1, 2.6, 7, 8 and 9).

Is that because a page is mapped in the PCH memory region, or because the mmap call with MAP_FIXED fails to return the desired address?

HP-UX doesn't have mincore. It might be possible to write a routine to probe the region. However, it's not going to fix the problem if extant pages can be mapped in the PCH memory region.

Dave

Comment 11 Eric Botcazou 2004-07-18 10:00:41 UTC

Is that because a page is mapped in the PCH memory region, or because the mmap call with MAP_FIXED fails to return the desired address?

The former IIRC.

HP-UX doesn't have mincore. It might be possible to write a routine to probe the region. However, it's not going to fix the problem if extant pages can be mapped in the PCH memory region.

Btw, the test fails on PA/Linux too?

Comment 12 Ian Lance Taylor 2004-07-18 12:25:52 UTC

Subject: Re: PCH largefile test fails on various platforms

"ebotcazou at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

Btw, the test fails on PA/Linux too?

Add an appropriate definition of TRY_EMPTY_VM_SPACE in gcc/config/host-linux.c.

Ian

Subject: Re: PCH largefile test fails on various platforms

Btw, the test fails on PA/Linux too?

Yes. I had one libstdc++ testsuite run recently where this problem caused nearly all the tests to fail. It's not just a problem with largefile.c.

Dave

Comment 15 Drea Pinski 2004-08-21 17:05:26 UTC

*** Bug 17131 has been marked as a duplicate of this bug. ***

Subject: Re: PCH largefile test fails on various platforms

I have noted in testing in the past week that changing kernels on hppa-linux can break PCH. Rerunning the libstdc++-v3 testsuite after a kernel change sometimes causes nearly all the tests to fail.

Dave

Comment 17 Jon Grimm 2004-10-12 19:56:22 UTC

patch for powerpc64-linux-gnu: http://gcc.gnu.org/ml/gcc-patches/2004-10/msg01037.html

Index: gcc/config/host-linux.c

RCS file: /home/jgrimm/gcc-cvs/gcc/gcc/config/host-linux.c,v retrieving revision 1.2 diff -c -3 -p -r1.2 host-linux.c *** gcc/config/host-linux.c 24 Apr 2004 00:33:18 -0000 1.2 --- gcc/config/host-linux.c 12 Oct 2004 21:35:47 -0000

*** 75,80 **** --- 75,82 ----

define TRY_EMPTY_VM_SPACE 0x8000000000

#elif defined(s390)

define TRY_EMPTY_VM_SPACE 0x60000000

Comment 18 James A. Morrison 2004-11-04 04:57:39 UTC

Also works for sparc-linux and largefile.c on sparc64-linux. Unfortunatly, there are other pch failures on sparc64-linux.

Comment 19 Eric Botcazou 2004-11-04 07:09:13 UTC

Also works for sparc-linux and largefile.c on sparc64-linux. Unfortunatly, there are other pch failures on sparc64-linux.

I'm a bit skeptical about using the same value for the 32-bit and the 64-bit flavors, but I'm not a specialist of Linux on the SPARC at all. I'd suggest to contact David S. Miller (see the MAINTAINERS file).

Comment 24 Joseph S. Myers 2005-02-11 01:54:03 UTC

Also fails on ia64-hpux.

Subject: Re: PCH largefile test fails on various platforms

------- Additional Comments From jsm28 at gcc dot gnu dot org 2005-02-11 01:54 ------- Also fails on ia64-hpux.

This is now "fixed" on the PA. You might look at pa-host.c to if the adopted work around might help other ports.

Dave

Comment 26 Eric Botcazou 2005-02-13 11:52:46 UTC

This is now "fixed" on the PA. You might look at pa-host.c to if the adopted work around might help other ports.

Alas, that doesn't work on SPARC/Solaris.

Subject: Re: PCH largefile test fails on various platforms

Log message: ./ PR pch/14400 Backport from mainline:

With this change largefile.c now fails on hppa2.0w-hp-hpux11.00 on 3.4. So, we now have a 3.4 regression. I guess the 4.0 PA fix needs to be backported.

Dave

Comment 30 Ian Lance Taylor 2005-08-04 17:54:06 UTC

My patch added the largefile test to the 3.4 branch. I don't think that failing a new test can be called a regression. I expect that the compiler would have failed the test before, too, it's just that nobody knew about it.

Failing the largefile test means that pre-compiled headers will fail occasionally and effectively unpredictably. That's why I added the test: so that at least people will know for which targets PCH is unreliable.

I would be in favor of backporting to 3.4 any HP/UX fix which fixes the largefile test. But since I don't think it's a regression I don't think it's on my plate to do that. I will probably do it if nobody else does, but not very soon.

Subject: Re: PCH largefile test fails on various platforms

My patch added the largefile test to the 3.4 branch. I don't think that failing a new test can be called a regression. I expect that the compiler would have failed the test before, too, it's just that nobody knew about it.

Ok.

I would be in favor of backporting to 3.4 any HP/UX fix which fixes the largefile test. But since I don't think it's a regression I don't think it's on my plate to do that. I will probably do it if nobody else does, but not very soon.

I'll try to find the time to test the change in the next few days.

Dave

Comment 33 Brett Albertson 2005-12-14 19:24:54 UTC

I think I'm seeing this same problem on Solaris x86.

Executing on host: /var/tmp/gcc_svn/gcc20051214/gcc/xgcc -B/var/tmp/gcc_svn/gcc2 0051214/gcc/ largefile.c -O0 -g -I. -S -o largefile.s (timeout = 300) largefile.c:1: fatal error: had to relocate PCH^M compilation terminated.^M compiler exited with status 1 output is: largefile.c:1: fatal error: had to relocate PCH^M compilation terminated.^M

FAIL: largefile.c -O0 -g (test for excess errors) Excess errors: largefile.c:1: fatal error: had to relocate PCH

Should the SPARC fix apply for x86 also?

Brett Albertson

Comment 34 Mark Mitchell 2007-03-27 04:31:48 UTC

Just a confirmation that I see this test failing with GCC 4.1 on Solaris as well. As Eric says in Comment #11, the symptoms are that, in host-solaris.c the first call to mmap succeeds (but at the wrong address). Then, mincore reports that the pages we want are already mapped, so we can't try the MAP_FIXED approach.

Comment 35 Mark Mitchell 2007-05-08 22:05:40 UTC

Ian Taylor suggests:

The way to fix this is to add a HOST_HOOKS_GT_PCH_GET_ADDRESS to host-solaris.c. That hook should try to allocate the space in some address area that is normally free on Solaris. See the use of TRY_EMPTY_VM_SPACE in host-linux.c.

Comment 36 Rainer Orth 2010-03-01 12:44:54 UTC

Subject: Bug 14940

Author: ro Date: Mon Mar 1 12:44:33 2010 New Revision: 157141

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=157141 Log: gcc: PR pch/14940 * config/host-solaris.c (HOST_HOOKS_GT_PCH_GET_ADDRESS): Redefine to sol_gt_pch_get_address. (TRY_EMPTY_VM_SPACE): Define for all combinations of 32 and 64-bit, SPARC and x86. (sol_gt_pch_get_address): New function.

gcc/testsuite:
[PR pch/14940](show%5Fbug.cgi?id=14940 "NEW - PCH largefile test fails on various platforms")
* gcc.dg/pch/pch.exp: Don't XFAIL largefile.c on i?86-*-solaris2.10.

Modified: trunk/gcc/ChangeLog trunk/gcc/config/host-solaris.c trunk/gcc/testsuite/gcc.dg/pch/pch.exp

Comment 39 Rainer Orth 2010-06-02 17:28:48 UTC

Subject: Bug 14940

Author: ro Date: Wed Jun 2 17:28:24 2010 New Revision: 160176

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=160176 Log: Backport from mainline: 2010-03-01 Rainer Orth <ro@CeBiTec.Uni-Bielefeld.DE>

[PR pch/14940](show%5Fbug.cgi?id=14940 "NEW - PCH largefile test fails on various platforms")
* config/host-solaris.c (HOST_HOOKS_GT_PCH_GET_ADDRESS): Redefine
to sol_gt_pch_get_address.
(TRY_EMPTY_VM_SPACE): Define for all combinations of 32 and
64-bit, SPARC and x86.
(sol_gt_pch_get_address): New function.

Modified: branches/gcc-4_4-branch/gcc/ChangeLog branches/gcc-4_4-branch/gcc/config/host-solaris.c branches/gcc-4_4-branch/gcc/testsuite/gcc.dg/pch/pch.exp

Comment 43 Rainer Orth 2011-02-15 09:49:22 UTC

Author: ro Date: Tue Feb 15 09:49:14 2011 New Revision: 170166

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=170166 Log: PR pch/14940 * config/alpha/host-osf.c: New file. * config/alpha/x-osf: New file. * config.host (alpha*-dec-osf*): Use it.

Added: trunk/gcc/config/alpha/host-osf.c trunk/gcc/config/alpha/x-osf Modified: trunk/gcc/ChangeLog trunk/gcc/config.host

Comment 44 Rainer Orth 2011-02-15 09:52:04 UTC

Author: ro Date: Tue Feb 15 09:51:58 2011 New Revision: 170167

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=170167 Log: PR pch/14940 * config/alpha/host-osf.c: New file. * config/alpha/x-osf: New file. * config.host (alpha*-dec-osf*): Use it.

Added: branches/gcc-4_5-branch/gcc/config/alpha/host-osf.c branches/gcc-4_5-branch/gcc/config/alpha/x-osf Modified: branches/gcc-4_5-branch/gcc/ChangeLog branches/gcc-4_5-branch/gcc/config.host

Comment 45 Rainer Orth 2011-02-15 09:58:04 UTC

Author: ro Date: Tue Feb 15 09:58:01 2011 New Revision: 170168

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=170168 Log: PR pch/14940 * config/alpha/host-osf.c: New file. * config/alpha/x-osf: New file. * config.host (alpha*-dec-osf*): Use it.

Added: branches/gcc-4_4-branch/gcc/config/alpha/host-osf.c branches/gcc-4_4-branch/gcc/config/alpha/x-osf Modified: branches/gcc-4_4-branch/gcc/ChangeLog branches/gcc-4_4-branch/gcc/config.host

Comment 47 Martin Richter 2015-06-27 18:28:15 UTC

Patch for i386-mingw32:

diff --git a/gcc/config/i386/host-mingw32.c b/gcc/config/i386/host-mingw32.c index aa17378..7ee6d0b 100644 --- a/gcc/config/i386/host-mingw32.c +++ b/gcc/config/i386/host-mingw32.c @@ -42,9 +42,6 @@ static size_t mingw32_gt_pch_alloc_granularity (void); static inline void w32_error(const char*, const char*, int, const char*); -/* FIXME: Is this big enough? */ -static const size_t pch_VA_max_size = 128 * 1024 * 1024;

/* Granularity for reserving address space. */ static size_t va_granularity = 0x10000;

@@ -86,9 +83,6 @@ static void * mingw32_gt_pch_get_address (size_t size, int) { void* res;

/* FIXME: We let system determine base by setting first arg to NULL. Allocating at top of available address space avoids unnecessary @@ -148,7 +142,7 @@ mingw32_gt_pch_use_address (void *addr, size_t size, int fd,

/* Offset must be also be a multiple of allocation granularity for this to work. We can't change the offset. */

Comment 49 Martin Richter 2015-07-07 10:07:45 UTC

(In reply to Martin Richter from comment #47)

My apologies, that patch is incorrect - VirtualAlloc still uses pch_VA_max_size instead of size. Some discussion about this bug and the reasoning behind the patch can be found here: https://sourceforge.net/p/mingw-w64/bugs/382/

Now the correct patch:

diff --git a/gcc/config/i386/host-mingw32.c b/gcc/config/i386/host-mingw32.c index aa17378..631d9c4 100644 --- a/gcc/config/i386/host-mingw32.c +++ b/gcc/config/i386/host-mingw32.c @@ -42,9 +42,6 @@ static size_t mingw32_gt_pch_alloc_granularity (void); static inline void w32_error(const char*, const char*, int, const char*); -/* FIXME: Is this big enough? */ -static const size_t pch_VA_max_size = 128 * 1024 * 1024;

/* Granularity for reserving address space. */ static size_t va_granularity = 0x10000;

@@ -86,9 +83,6 @@ static void * mingw32_gt_pch_get_address (size_t size, int) { void* res;

/* FIXME: We let system determine base by setting first arg to NULL. Allocating at top of available address space avoids unnecessary @@ -98,7 +92,7 @@ mingw32_gt_pch_get_address (size_t size, int) If we allocate at bottom we need to reserve the address as early as possible and at the same point in each invocation. */

Comment 50 Martin Richter 2015-07-09 18:47:20 UTC

The following patch fixes segfault when gt_pch_use_address fails (returns -1). fatal_error now correctly shows an error message and terminates the program. I have basicly only reordered reads, and placed them after the file mapping itself. Global pointers are changed only after gt_pch_use_address succeeds, so in case of failure they still contain valid addresses.

This patch is meant for the master branch. However, it should not be hard to modify it for others.

diff --git a/gcc/ggc-common.c b/gcc/ggc-common.c index 5096837..f741f2c 100644 --- a/gcc/ggc-common.c +++ b/gcc/ggc-common.c @@ -599,7 +599,9 @@ gt_pch_restore (FILE *f) size_t i; struct mmap_info mmi; int result;

@@ -631,12 +637,35 @@ gt_pch_restore (FILE *f) if (result == 0) { if (fseek (f, mmi.offset, SEEK_SET) != 0 - || fread (mmi.preferred_base, mmi.size, 1, f) != 1) - fatal_error (input_location, "can%'t read PCH file: %m");

Comment 51 Julien Ruffin 2020-11-11 15:30:53 UTC

I can confirm Martin's patches in #49 and #50 work on GCC 9.2.0. They solve silent segfaults when using large PCHs (> 128MB).

Comment 53 LIU Hao 2022-05-10 04:52:19 UTC

The patch no longer applies to GCC 12.

Comment 55 LIU Hao 2022-05-10 05:05:25 UTC

(In reply to Andrew Pinski from comment #54)

(In reply to LIU Hao from comment #53)

The patch no longer applies to GCC 12.

Right because I think this issue has been fixed by r12-5855-g747380f47da0da .

No, at least not completely. In 'gcc/config/i386/host-mingw32.cc' there is still the 128MiB hard limit:

/* FIXME: Is this big enough?  */
static const size_t pch_VA_max_size  = 128 * 1024 * 1024;

Comment 56 Drea Pinski 2022-05-10 05:15:09 UTC

(In reply to LIU Hao from comment #55)

(In reply to Andrew Pinski from comment #54)

(In reply to LIU Hao from comment #53)

The patch no longer applies to GCC 12.

Right because I think this issue has been fixed by r12-5855-g747380f47da0da .

No, at least not completely. In 'gcc/config/i386/host-mingw32.cc' there is still the 128MiB hard limit:

/* FIXME: Is this big enough?  */
static const size_t pch_VA_max_size  = 128 * 1024 * 1024;

Right, some hosts need to be changed. Looks like only darwin and linux was fixed.

See r12-5857-g0d5db1dd65b452 for the change done for darwin. Maybe a similar change is needed for Mingw.

Comment 57 Jakub Jelinek 2022-05-10 08:51:38 UTC

(In reply to Andrew Pinski from comment #56)

Right, some hosts need to be changed. Looks like only darwin and linux was fixed.

Yeah, I said that clearly that other ports need to be changed if they want to take advantage of it. I don't have access to those targets, can't test and am not familiar with them, it is up to the maintainers of those targets or anyone that is willing to test it...

Comment 62 Yvan Roux 2026-01-16 14:17:11 UTC

We're doing canadian cross build of arm-none-eabi toolchain. Building GCC version r15.2.0-477-gc3e93759b51 on old LTS Ubuntu version work fine, when switching to Ubuntu 22.04 with these versions of the tools we have the PCH issue:

Binutils 2.38-3ubuntu1+9build1 GCC 10.3.0-14ubuntu1+24.3 Mingw 8.0.0-1

Applying your patch fixes the problem.

Comment 64 Jakub Jelinek 2026-01-16 16:17:47 UTC

LGTM.

Comment 68 LIU Hao 2026-02-06 08:25:14 UTC

(In reply to Torbjorn SVENSSON from comment #67)

Looks like releases/gcc-15 never got the https://gcc.gnu.org/cgit/gcc/commit/ ?id=16e301adf7b653c72d5c3cae1c7287877f3f20ff patch, while the other branches do. Is it okay if I pick it for releases/gcc-15?

I'm no GCC maintainer, but I think it's fine.