GCC selftest improvements

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GCC selftest improvements

gcc - Dev mailing list
TLDR: I'd like to propose adding a dependency on a modern unit testing framework to make it easier to write unit tests within GCC. Before I spend much more time on it, what sort of buy-in should I get? Are there any people in particular I should work more closely with as I make this change?
 
Terminology: Within GCC, there are two types of tests in place: unit tests and regression tests. The unit tests have been written with a home-grown selftest framework and run as part of the build process. Any failures to a unit test results in no compiler being produced. The regression tests, on the other hand, run after build, and use the separate DejaGnu framework. In this email, I am only concerning myself with the unit tests, and throughout the remainder of the email, any mention of tests refers to these.
 
Working on GCC, I wanted to add some new unit tests to my feature as I went, but I noticed that there is a good deal of friction involved. Right now, adding new unit tests requires writing the test method, then modifying a second place in the code to call said test method, repeating as necessary until getting all the way to either the selftest.c file or the target hook. There is also no way to do test setup/teardown automatically. Everything is manual.
 
I'd like to propose adding a dependency on a modern open-source unit testing framework as an enhancement to the current self test system. I have used Catch2 (https://github.com/catchorg/Catch2, Boost Software License 1.0) with great success in the past. I experimented with adding it to GCC and converting a handful of tests to use Catch2. Although I only converted a small number of tests, I didn't see any performance impact during selftest. As a bonus, while doing so, I actually found that one test that I had written previously wasn't actually being run, because I had failed to manually call it.
 
Some nice things that Catch2 provides are better error reporting (see below for a comparison), ease of adding new tests (just include the header and write a TEST_CASE(), as opposed to the manual plumbing required right now), extension points for adding custom comparisons (I could see this being very useful to expand on the current rtl test macros), and the ability to run a subset of the tests without recompiling. It is also easy to integrate Catch2 with the existing self-test framework.
 
If this path seems useful to others, I'm happy to pursue it further. A list of work items I see are:
 
1. Convert more tests to verify the claim that build performance is not degraded
2. Update the docs to list Catch2 as the new recommended way to write unit tests
3. If all of the target self-tests are converted, then we can remove the target test hook. Similar for the lang test hook.
 
One thing that would make Catch2 an even more slam-dunk case was if we were able to enable exceptions for the check builds. Then, running the unit tests could report multiple failures at the same time instead of just aborting at the first one. That said, even without enabling exceptions, Catch2 is on par with the current selftest in terms of terminating at the first failure.
 
Another option is to use a test framework that doesn't use exceptions, such as Google Test (https://github.com/google/googletest, BSD 3-Clause "New" or "Revised" License). I personally think Catch2 is more flexible, or I would lead with Google Test. For example, in Catch2, shared setup is done in place with the tests itself, having each subtest be a nested SECTION, where-as in GTest, you have to write a test class that derives from ::test and overrides SetUp(). In addition, the sections in Catch2 can be nested further, allowing several related tests to build on each other.
 
Here is some sample output for the case where all the tests are passing:
===============================================================================
All tests passed (25 assertions in 5 test cases)
 
And here is the output when a test fails:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
is a Catch v2.9.2 host application.
Run with -? for options
 
-------------------------------------------------------------------------------
test_set_range
-------------------------------------------------------------------------------
../../gcc/bitmap.c:2661
...............................................................................
../../gcc/bitmap.c:2668: FAILED:
  REQUIRE( 6 == bitmap_count_bits (b) )
with expansion:
  6 == 5
 
Catch will terminate because it needed to throw an exception.
The message was: Test failure requires aborting test!
terminate called without an active exception
../../gcc/bitmap.c:2668: FAILED:
  {Unknown expression after the reported line}
due to a fatal error condition:
  SIGABRT - Abort (abnormal termination) signal
===============================================================================
test cases: 2 | 1 passed | 1 failed
assertions: 5 | 3 passed | 2 failed
cc1: internal compiler error: Aborted
<long callstack>
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
 
(Note that at the moment it doesn't know the name of our application or it would have prefixed "is a Catch..." with our app name).
 
Compare that to the output of the current test framework:
../../gcc/bitmap.c:2669: test_set_range: FAIL: ASSERT_EQ ((6), (bitmap_count_bits (b)))
cc1: internal compiler error: in fail, at selftest.c:47
/bin/bash ../../gcc/../move-if-change tmp-macro_list macro_list
echo timestamp > s-macro_list
<long callstack>
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <https://gcc.gnu.org/bugs/> for instructions.
 
Thanks,
 
Andrew
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jonathan Wakely-4
On Thu, 24 Oct 2019 at 21:50, Andrew Dean via gcc <[hidden email]> wrote:
>
> TLDR: I'd like to propose adding a dependency on a modern unit testing framework to make it easier to write unit tests within GCC. Before I spend much more time on it, what sort of buy-in should I get? Are there any people in particular I should work more closely with as I make this change?

Dave Malcolm added the self test framework. He's subscribed to the
list, but I've CC'd him on this mail too.
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

David Malcolm
In reply to this post by gcc - Dev mailing list
On Thu, 2019-10-24 at 20:50 +0000, Andrew Dean via gcc wrote:

> TLDR: I'd like to propose adding a dependency on a modern unit
> testing framework to make it easier to write unit tests within GCC.
> Before I spend much more time on it, what sort of buy-in should I
> get? Are there any people in particular I should work more closely
> with as I make this change?
>  
> Terminology: Within GCC, there are two types of tests in place: unit
> tests and regression tests. The unit tests have been written with a
> home-grown selftest framework and run as part of the build process.
> Any failures to a unit test results in no compiler being produced.
> The regression tests, on the other hand, run after build, and use the
> separate DejaGnu framework. In this email, I am only concerning
> myself with the unit tests, and throughout the remainder of the
> email, any mention of tests refers to these.
>  
> Working on GCC, I wanted to add some new unit tests to my feature as
> I went, but I noticed that there is a good deal of friction involved.
> Right now, adding new unit tests requires writing the test method,
> then modifying a second place in the code to call said test method,
> repeating as necessary until getting all the way to either the
> selftest.c file or the target hook. There is also no way to do test
> setup/teardown automatically. Everything is manual.
>  
> I'd like to propose adding a dependency on a modern open-source unit
> testing framework as an enhancement to the current self test system.
> I have used Catch2 (https://github.com/catchorg/Catch2, Boost
> Software License 1.0) with great success in the past. I experimented
> with adding it to GCC and converting a handful of tests to use
> Catch2. Although I only converted a small number of tests, I didn't
> see any performance impact during selftest. As a bonus, while doing
> so, I actually found that one test that I had written previously
> wasn't actually being run, because I had failed to manually call it.
>  
> Some nice things that Catch2 provides are better error reporting (see
> below for a comparison), ease of adding new tests (just include the
> header and write a TEST_CASE(), as opposed to the manual plumbing
> required right now), extension points for adding custom comparisons
> (I could see this being very useful to expand on the current rtl test
> macros), and the ability to run a subset of the tests without
> recompiling. It is also easy to integrate Catch2 with the existing
> self-test framework.
>  
> If this path seems useful to others, I'm happy to pursue it further.
> A list of work items I see are:
>  
> 1. Convert more tests to verify the claim that build performance is
> not degraded
> 2. Update the docs to list Catch2 as the new recommended way to write
> unit tests
> 3. If all of the target self-tests are converted, then we can remove
> the target test hook. Similar for the lang test hook.
>  
> One thing that would make Catch2 an even more slam-dunk case was if
> we were able to enable exceptions for the check builds. Then, running
> the unit tests could report multiple failures at the same time
> instead of just aborting at the first one. That said, even without
> enabling exceptions, Catch2 is on par with the current selftest in
> terms of terminating at the first failure.
>  
> Another option is to use a test framework that doesn't use
> exceptions, such as Google Test (https://github.com/google/googletest
> , BSD 3-Clause "New" or "Revised" License). I personally think Catch2
> is more flexible, or I would lead with Google Test. For example, in
> Catch2, shared setup is done in place with the tests itself, having
> each subtest be a nested SECTION, where-as in GTest, you have to
> write a test class that derives from ::test and overrides SetUp(). In
> addition, the sections in Catch2 can be nested further, allowing
> several related tests to build on each other.
>  
> Here is some sample output for the case where all the tests are
> passing:
> =====================================================================
> ==========
> All tests passed (25 assertions in 5 test cases)
>  
> And here is the output when a test fails:
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ~~~~~~~~~~
> is a Catch v2.9.2 host application.
> Run with -? for options
>  
> -------------------------------------------------------------------
> ------------
> test_set_range
> -------------------------------------------------------------------
> ------------
> ../../gcc/bitmap.c:2661
> .....................................................................
> ..........
> ../../gcc/bitmap.c:2668: FAILED:
>   REQUIRE( 6 == bitmap_count_bits (b) )
> with expansion:
>   6 == 5
>  
> Catch will terminate because it needed to throw an exception.
> The message was: Test failure requires aborting test!
> terminate called without an active exception
> ../../gcc/bitmap.c:2668: FAILED:
>   {Unknown expression after the reported line}
> due to a fatal error condition:
>   SIGABRT - Abort (abnormal termination) signal
> =====================================================================
> ==========
> test cases: 2 | 1 passed | 1 failed
> assertions: 5 | 3 passed | 2 failed
> cc1: internal compiler error: Aborted
> <long callstack>
> Please submit a full bug report,
> with preprocessed source if appropriate.
> Please include the complete backtrace with any bug report.
> See <https://gcc.gnu.org/bugs/> for instructions.
>  
> (Note that at the moment it doesn't know the name of our application
> or it would have prefixed "is a Catch..." with our app name).
>  
> Compare that to the output of the current test framework:
> ../../gcc/bitmap.c:2669: test_set_range: FAIL: ASSERT_EQ ((6),
> (bitmap_count_bits (b)))
> cc1: internal compiler error: in fail, at selftest.c:47
> /bin/bash ../../gcc/../move-if-change tmp-macro_list macro_list
> echo timestamp > s-macro_list
> <long callstack>
> Please submit a full bug report,
> with preprocessed source if appropriate.
> Please include the complete backtrace with any bug report.
> See <https://gcc.gnu.org/bugs/> for instructions.

Thanks for your email, it looks interesting.  Is your code somewhere we
can see it?

I'm the author of gcc's selftest framework (and I use it heavily e.g.
for testing the diagnostics subsystem [1]).

It went through substantial changes during review.

I looked over my notes; for reference, here's a summary of the history
of the patches:

v1: https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00765.html
  "[PATCH 00/17] RFC: Addding a unit testing framework to gcc"
     Used Google Test framework, was a dummy "frontend"

v2: https://gcc.gnu.org/ml/gcc-patches/2015-06/msg01224.html
  "[PATCH/RFC]: unittesting v2: as a plugin (was Re: [PATCH 00/17] RFC:
Addding a unit testing framework to gcc)"
    Still Google Test, as a plugin rather than a frontend

v3: https://gcc.gnu.org/ml/gcc-patches/2015-10/msg02947.html
  "[PATCH 00/16] Unit tests framework (v3)"
    Still Google Test, as a plugin built by plugin.exp within DejaGnu
tests, with a custom gtest reporter
    Some discussion about gtest:
      https://gcc.gnu.org/ml/gcc-patches/2015-10/msg03215.html
   
v4: https://gcc.gnu.org/ml/gcc-patches/2015-11/msg02379.html
  "[PATCH 00/15] Unittests framework v4: -fself-test"
    Done via "-fself-test" via compiling a dummy file in DejaGnu tests
    I believe it was at this point that I switched to a custom API that
resembles gtest, rather than gtest itself.

v5: https://gcc.gnu.org/ml/gcc-patches/2016-06/msg00082.html
  "[PATCH 00/21] Add -fself-test framework for fast, early unit-testing
(unittests v5)"
    Done via "-fself-test" at each of the 3 stages of bootstrap.

v6: https://gcc.gnu.org/ml/gcc-patches/2016-06/msg00210.html
  "[PATCH 00/16] v6 of -fself-test/unit-testing patch"
    Switched to "abort on first failure"
    Eliminated runner class, and from self-registrating tests to manual
test invocation

v7: https://gcc.gnu.org/ml/gcc-patches/2016-06/msg00298.html
   "[PATCH] Selftest framework (v7)"
   (one combined patch)

v8: approved; committed v8 to trunk as r237144 (for gcc 7):
   https://gcc.gnu.org/ml/gcc-patches/2016-06/msg00410.html

Notable followups:

2016-07-11:
  * r238209: "Support running the selftests under valgrind"
    * https://gcc.gnu.org/ml/gcc-patches/2016-07/msg00437.html

2017-12-11:
  * r255563: "Expensive selftests: torture testing for fix-it boundary
conditions (PR c/82050)"
    * https://gcc.gnu.org/ml/gcc-patches/2017-11/msg02459.html
    (some tests run as a DejaGnu-built plugin)

2018-04-30:
  * r259782: "selftest: remove "Yoda ordering" in assertions"
    * https://gcc.gnu.org/ml/gcc-patches/2018-04/msg01323.html

2018-10-17:
  * r265240: "Run selftests for C++ as well as C"
    * https://gcc.gnu.org/ml/gcc-patches/2018-10/msg00802.html

I think the consensus during review was that I was over-engineering
things, and that each iteration from v1 to v8 made the code simpler and
involved less C++ "magic", closer to C.  Whether that's still the
consensus I don't know.  Different people within the GCC dev community
have different comfort levels with C++, and my initial version (using
gtest) was probably too "magic" for some.  Maybe people here are more
comfortable with C++ now?

GCC has some rather unique requirements, in that we support a great
many build configurations, some of which are rather primitive - for
example, requiring just C++98 with exceptions disabled, in that we want
to be able to be bootstrappable on relatively "ancient" configurations.
IIRC auto-registration of tests requires that the build configuration
have a sufficiently sane implementation of C++ - having globals with
non-trivial ctors tends to be problematic when dealing with early
implementations of C++.

Personally I don't find the manual registration of tests to be a pain,
but it would be nice to have more readable errors on failures.  There's
probably a case for more verbose test output.  (generally I immediately
just do "make selftest-gdb" on failures; the issue is if it suddenly
starts failing on a build I don't have access to)

I suspect that exceptions would be a deal-breaker; does Catch2 support
-fno-exceptions?

As for setup/teardown, I've been able to do that "manually" using RAII-
style classes in test functions.

Thanks again for your email; hope this is constructive.
Dave

[1] see e.g. the selftests in gcc/input.c and gcc/diagnostic-show-
locus.c

Reply | Threaded
Open this post in threaded view
|

RE: GCC selftest improvements

gcc - Dev mailing list
> From: David Malcolm <[hidden email]>
> Sent: Thursday, October 24, 2019 11:18 PM
> On Thu, 2019-10-24 at 20:50 +0000, Andrew Dean via gcc wrote:
> Thanks for your email, it looks interesting.  Is your code somewhere we can see
> it?
It can be -- what is the preferred way to share the code? Though to be honest I can summarize the changes pretty quickly:
1. Add catch.hpp (the single include header from the Catch2 project) and a small wrapper header around catch.hpp that temporarily fixes up some macros that GCC defines to replace c library functions and does nothing if !CHECKING_P
2. Modify test methods like so:
- void test_this_thing ()
+ TEST_CASE("test this thing")
And
- ASSERT_EQ (a, b);
+ REQUIRE (a == b);
3. Invoke the Catch2 test runner during selftest like:
Catch::Session ().run ();
4. Remove the manual invocations of the test methods, as the TEST_CASE macro takes care of self-registration.

> I think the consensus during review was that I was over-engineering things, and
> that each iteration from v1 to v8 made the code simpler and involved less C++
> "magic", closer to C.  Whether that's still the consensus I don't know.  Different
> people within the GCC dev community have different comfort levels with C++,
> and my initial version (using
> gtest) was probably too "magic" for some.  Maybe people here are more
> comfortable with C++ now?

Here's hoping! Looks like you had a very similar starting point as what I suggested here.

> GCC has some rather unique requirements, in that we support a great many
> build configurations, some of which are rather primitive - for example,
> requiring just C++98 with exceptions disabled, in that we want to be able to be
> bootstrappable on relatively "ancient" configurations.
> IIRC auto-registration of tests requires that the build configuration have a
> sufficiently sane implementation of C++ - having globals with non-trivial ctors
> tends to be problematic when dealing with early implementations of C++.

Is C++98 the limit of what we can use in GCC? If so, that immediately eliminates Catchv1 (C++03), Catch2 (C++11+) and GTest (C++11)
 
> Personally I don't find the manual registration of tests to be a pain, but it would
> be nice to have more readable errors on failures.  There's probably a case for
> more verbose test output.  (generally I immediately just do "make selftest-gdb"
> on failures; the issue is if it suddenly starts failing on a build I don't have access
> to)

I didn't know about selftest-gdb. That will come in handy. My ideal programming style, is to write a new test, watch it fail for the expected reason, then write the production code to make it pass. Having to attach a debugger to validate/investiigate failures slows down the process, as does having to write the additional code to invoke the new test methods, if only by a little bit.
 
> I suspect that exceptions would be a deal-breaker; does Catch2 support -fno-
> exceptions?

Yes, Catch2 supports -fno-exceptions, though not like GTest, which was built to not use exceptions at all. Catch2 stops running tests at the first failure and gives the output as shown in the original email.

> As for setup/teardown, I've been able to do that "manually" using RAII- style
> classes in test functions.

Yes, I have added some RAII classes to assist in testing as well. I just think it will be even better if it were easier to do.
Reply | Threaded
Open this post in threaded view
|

RE: GCC selftest improvements

gcc - Dev mailing list
[Andrew]

| > GCC has some rather unique requirements, in that we support a great many
| > build configurations, some of which are rather primitive - for example,
| > requiring just C++98 with exceptions disabled, in that we want to be able to
| be
| > bootstrappable on relatively "ancient" configurations.
| > IIRC auto-registration of tests requires that the build configuration have a
| > sufficiently sane implementation of C++ - having globals with non-trivial
| ctors
| > tends to be problematic when dealing with early implementations of C++.
|
| Is C++98 the limit of what we can use in GCC? If so, that immediately
| eliminates Catchv1 (C++03), Catch2 (C++11+) and GTest (C++11)

C++98 was what Diego, Lawrence, Benjamin, Ian, and myself could agreed to back in 2011-2012 when C++11 got just out as a C++ standard, so we couldn't pick C++11 as we didn't have enough G++ out there to count on.

I would expect the situation to have drastically changed - with very handy and popular features such as 'constexpr' (especially with the C++14 relaxation), lambdas and range-for.

Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?

-- Gaby

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Eric Gallager
On 10/25/19, Gabriel Dos Reis via gcc <[hidden email]> wrote:

> [Andrew]
>
> | > GCC has some rather unique requirements, in that we support a great
> many
> | > build configurations, some of which are rather primitive - for example,
> | > requiring just C++98 with exceptions disabled, in that we want to be
> able to
> | be
> | > bootstrappable on relatively "ancient" configurations.
> | > IIRC auto-registration of tests requires that the build configuration
> have a
> | > sufficiently sane implementation of C++ - having globals with
> non-trivial
> | ctors
> | > tends to be problematic when dealing with early implementations of C++.
> |
> | Is C++98 the limit of what we can use in GCC? If so, that immediately
> | eliminates Catchv1 (C++03), Catch2 (C++11+) and GTest (C++11)
>
> C++98 was what Diego, Lawrence, Benjamin, Ian, and myself could agreed to
> back in 2011-2012 when C++11 got just out as a C++ standard, so we couldn't
> pick C++11 as we didn't have enough G++ out there to count on.
>
> I would expect the situation to have drastically changed - with very handy
> and popular features such as 'constexpr' (especially with the C++14
> relaxation), lambdas and range-for.
>
> Jason, Jonathan - is the situation on the terrain really that dire that
> C++11 (or C++14) isn't at all available for platforms that GCC is
> bootstrapped from?
>
> -- Gaby
>

Nicholas Krause was also wanting to move to C++11 recently:
https://gcc.gnu.org/ml/gcc/2019-10/msg00110.html (this month)
https://gcc.gnu.org/ml/gcc/2019-09/msg00228.html (last month)
As I said in that thread, I'd want to try just toggling -Wnarrowing
from off to on first before going full C++11. So, GCC 10 would be
C++98 + -Wnarrowing, and then GCC 11 could be full C++11. Plus then
the GCC version numbers would also line up with the version of C++
being used.

Eric
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jeff Law
In reply to this post by gcc - Dev mailing list
On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:

> [Andrew]
>
> | > GCC has some rather unique requirements, in that we support a great many
> | > build configurations, some of which are rather primitive - for example,
> | > requiring just C++98 with exceptions disabled, in that we want to be able to
> | be
> | > bootstrappable on relatively "ancient" configurations.
> | > IIRC auto-registration of tests requires that the build configuration have a
> | > sufficiently sane implementation of C++ - having globals with non-trivial
> | ctors
> | > tends to be problematic when dealing with early implementations of C++.
> |
> | Is C++98 the limit of what we can use in GCC? If so, that immediately
> | eliminates Catchv1 (C++03), Catch2 (C++11+) and GTest (C++11)
>
> C++98 was what Diego, Lawrence, Benjamin, Ian, and myself could agreed to back in 2011-2012 when C++11 got just out as a C++ standard, so we couldn't pick C++11 as we didn't have enough G++ out there to count on.
>
> I would expect the situation to have drastically changed - with very handy and popular features such as 'constexpr' (especially with the C++14 relaxation), lambdas and range-for.
>
> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
The argument that I'd make is that's relatively uncommon (I know, I know
AIX) that bootstrapping in those environments may well require first
building something like gcc-9.

I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
we have any good mechanism for making this kind of technical decision
when there isn't consensus.

jeff

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Richard Biener-2
On October 28, 2019 8:40:03 PM GMT+01:00, Jeff Law <[hidden email]> wrote:

>On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
>> [Andrew]
>>
>> | > GCC has some rather unique requirements, in that we support a
>great many
>> | > build configurations, some of which are rather primitive - for
>example,
>> | > requiring just C++98 with exceptions disabled, in that we want to
>be able to
>> | be
>> | > bootstrappable on relatively "ancient" configurations.
>> | > IIRC auto-registration of tests requires that the build
>configuration have a
>> | > sufficiently sane implementation of C++ - having globals with
>non-trivial
>> | ctors
>> | > tends to be problematic when dealing with early implementations
>of C++.
>> |
>> | Is C++98 the limit of what we can use in GCC? If so, that
>immediately
>> | eliminates Catchv1 (C++03), Catch2 (C++11+) and GTest (C++11)
>>
>> C++98 was what Diego, Lawrence, Benjamin, Ian, and myself could
>agreed to back in 2011-2012 when C++11 got just out as a C++ standard,
>so we couldn't pick C++11 as we didn't have enough G++ out there to
>count on.
>>
>> I would expect the situation to have drastically changed - with very
>handy and popular features such as 'constexpr' (especially with the
>C++14 relaxation), lambdas and range-for.
>>
>> Jason, Jonathan - is the situation on the terrain really that dire
>that C++11 (or C++14) isn't at all available for platforms that GCC is
>bootstrapped from?
>The argument that I'd make is that's relatively uncommon (I know, I
>know
>AIX) that bootstrapping in those environments may well require first
>building something like gcc-9.
>
>I'd really like to see us move to C++11 or beyond.  Sadly, I don't
>think
>we have any good mechanism for making this kind of technical decision
>when there isn't consensus.

Well, we just do it?

Richard.

>jeff

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jeff Law
On 10/28/19 1:42 PM, Richard Biener wrote:
>>
>> I'd really like to see us move to C++11 or beyond.  Sadly, I don't
>> think
>> we have any good mechanism for making this kind of technical decision
>> when there isn't consensus.
>
> Well, we just do it?
For some reason I thought you were against such a change.  Was I wrong?

jeff

Reply | Threaded
Open this post in threaded view
|

RE: GCC selftest improvements

gcc - Dev mailing list
| -----Original Message-----
| From: Jeff Law <[hidden email]>
| Sent: Monday, October 28, 2019 12:44 PM
| To: Richard Biener <[hidden email]>; [hidden email];
| Gabriel Dos Reis <[hidden email]>; Andrew Dean
| <[hidden email]>; David Malcolm <[hidden email]>;
| [hidden email]; [hidden email];
| [hidden email]; Jonathan Wakely <[hidden email]>
| Subject: Re: GCC selftest improvements
|
| On 10/28/19 1:42 PM, Richard Biener wrote:
| >>
| >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't
| >> think
| >> we have any good mechanism for making this kind of technical decision
| >> when there isn't consensus.
| >
| > Well, we just do it?
| For some reason I thought you were against such a change.  Was I wrong?

I suspect Andrew would be happy to contribute patch and any follow up.

-- Gaby

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Segher Boessenkool
In reply to this post by Jeff Law
On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> > Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> The argument that I'd make is that's relatively uncommon (I know, I know
> AIX) that bootstrapping in those environments may well require first
> building something like gcc-9.
>
> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> we have any good mechanism for making this kind of technical decision
> when there isn't consensus.

Which GCC version will be required to work as bootstrap compiler?  Will
4.8.5 be enough?


Segher
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jeff Law
On 10/28/19 2:27 PM, Segher Boessenkool wrote:

> On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
>> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
>>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
>> The argument that I'd make is that's relatively uncommon (I know, I know
>> AIX) that bootstrapping in those environments may well require first
>> building something like gcc-9.
>>
>> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
>> we have any good mechanism for making this kind of technical decision
>> when there isn't consensus.
>
> Which GCC version will be required to work as bootstrap compiler?  Will
> 4.8.5 be enough?
I'd say gcc-9.  What would we gain by making it 4.8 or anything else
that old?

jeff

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jakub Jelinek
On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:

> On 10/28/19 2:27 PM, Segher Boessenkool wrote:
> > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> >>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> >> The argument that I'd make is that's relatively uncommon (I know, I know
> >> AIX) that bootstrapping in those environments may well require first
> >> building something like gcc-9.
> >>
> >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> >> we have any good mechanism for making this kind of technical decision
> >> when there isn't consensus.
> >
> > Which GCC version will be required to work as bootstrap compiler?  Will
> > 4.8.5 be enough?
> I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> that old?

That is not a good idea, it will make it much harder to build gcc because
not everybody has gcc-9 built as a system compiler.
The previous minimum requirement of 4.1 is perhaps too old now that 4.8 is
something we could require and gain through that C++11 support, but we
shouldn't follow Rust with "you can only build it with 6 weeks old previous
release and nothing else".
As discussed earlier, we gain most through C++11 support, there is no need
to jump to C++17 or C++20 as requirement.

        Jakub

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Iain Sandoe
In reply to this post by Jeff Law
Jeff Law <[hidden email]> wrote:

> On 10/28/19 2:27 PM, Segher Boessenkool wrote:
>> On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
>>> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
>>>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
>>> The argument that I'd make is that's relatively uncommon (I know, I know
>>> AIX) that bootstrapping in those environments may well require first
>>> building something like gcc-9.
>>>
>>> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
>>> we have any good mechanism for making this kind of technical decision
>>> when there isn't consensus.
>>
>> Which GCC version will be required to work as bootstrap compiler?  Will
>> 4.8.5 be enough?
> I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> that old?

We’d have to use something older than 9 on earl(ier) Darwin since 9 will not
bootstrap with the system-provided tools.

ISTM that using a well-baked stable closed branch would be reasonable?

(so 7.5 or earlier, assuming that the decision is made after 7.5 rolls)

Iain

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Andrew Pinski-2
In reply to this post by Jakub Jelinek
On Mon, Oct 28, 2019 at 2:47 PM Jakub Jelinek <[hidden email]> wrote:

>
> On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:
> > On 10/28/19 2:27 PM, Segher Boessenkool wrote:
> > > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> > >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> > >>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> > >> The argument that I'd make is that's relatively uncommon (I know, I know
> > >> AIX) that bootstrapping in those environments may well require first
> > >> building something like gcc-9.
> > >>
> > >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> > >> we have any good mechanism for making this kind of technical decision
> > >> when there isn't consensus.
> > >
> > > Which GCC version will be required to work as bootstrap compiler?  Will
> > > 4.8.5 be enough?
> > I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> > that old?
>
> That is not a good idea, it will make it much harder to build gcc because
> not everybody has gcc-9 built as a system compiler.
> The previous minimum requirement of 4.1 is perhaps too old now that 4.8 is
> something we could require and gain through that C++11 support, but we
> shouldn't follow Rust with "you can only build it with 6 weeks old previous
> release and nothing else".
> As discussed earlier, we gain most through C++11 support, there is no need
> to jump to C++17 or C++20 as requirement.

Just a quick note.
RHEL/CentOS 7 uses GCC 4.8 as the system compiler.  Requiring a new
compiler to compile GCC 10 will not work for me.
I normally bootstrap GCC 10 and then build a GCC 10 cross compiler.
Having to have an extra compiler inbetween is problematic for me.

Thanks,
Andrew

>
>         Jakub
>
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Jeff Law
On 10/28/19 3:52 PM, Andrew Pinski wrote:

> On Mon, Oct 28, 2019 at 2:47 PM Jakub Jelinek <[hidden email]> wrote:
>>
>> On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:
>>> On 10/28/19 2:27 PM, Segher Boessenkool wrote:
>>>> On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
>>>>> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
>>>>>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
>>>>> The argument that I'd make is that's relatively uncommon (I know, I know
>>>>> AIX) that bootstrapping in those environments may well require first
>>>>> building something like gcc-9.
>>>>>
>>>>> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
>>>>> we have any good mechanism for making this kind of technical decision
>>>>> when there isn't consensus.
>>>>
>>>> Which GCC version will be required to work as bootstrap compiler?  Will
>>>> 4.8.5 be enough?
>>> I'd say gcc-9.  What would we gain by making it 4.8 or anything else
>>> that old?
>>
>> That is not a good idea, it will make it much harder to build gcc because
>> not everybody has gcc-9 built as a system compiler.
>> The previous minimum requirement of 4.1 is perhaps too old now that 4.8 is
>> something we could require and gain through that C++11 support, but we
>> shouldn't follow Rust with "you can only build it with 6 weeks old previous
>> release and nothing else".
>> As discussed earlier, we gain most through C++11 support, there is no need
>> to jump to C++17 or C++20 as requirement.
>
> Just a quick note.
> RHEL/CentOS 7 uses GCC 4.8 as the system compiler.  Requiring a new
> compiler to compile GCC 10 will not work for me.
At this point RHEL/CentOS 7 is ancient.  Time to move forward IMHO :-)


I can live with an older compiler, but I'd prefer we move forward as
much as possible.  4.8 is like living in the stone age.

jeff

Reply | Threaded
Open this post in threaded view
|

RE: GCC selftest improvements

gcc - Dev mailing list
In reply to this post by Andrew Pinski-2


| -----Original Message-----
| From: Andrew Pinski <[hidden email]>
| Sent: Monday, October 28, 2019 2:52 PM
| To: Jakub Jelinek <[hidden email]>
| Cc: Jeff Law <[hidden email]>; Segher Boessenkool
| <[hidden email]>; Gabriel Dos Reis <[hidden email]>;
| Andrew Dean <[hidden email]>; David Malcolm
| <[hidden email]>; [hidden email]; [hidden email];
| [hidden email]; [hidden email]; Jonathan Wakely
| <[hidden email]>
| Subject: Re: GCC selftest improvements
|
| On Mon, Oct 28, 2019 at 2:47 PM Jakub Jelinek <[hidden email]> wrote:
| >
| > On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:
| > > On 10/28/19 2:27 PM, Segher Boessenkool wrote:
| > > > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
| > > >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
| > > >>> Jason, Jonathan - is the situation on the terrain really that dire that
| C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped
| from?
| > > >> The argument that I'd make is that's relatively uncommon (I know, I
| know
| > > >> AIX) that bootstrapping in those environments may well require first
| > > >> building something like gcc-9.
| > > >>
| > > >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
| > > >> we have any good mechanism for making this kind of technical
| decision
| > > >> when there isn't consensus.
| > > >
| > > > Which GCC version will be required to work as bootstrap compiler?  Will
| > > > 4.8.5 be enough?
| > > I'd say gcc-9.  What would we gain by making it 4.8 or anything else
| > > that old?
| >
| > That is not a good idea, it will make it much harder to build gcc because
| > not everybody has gcc-9 built as a system compiler.
| > The previous minimum requirement of 4.1 is perhaps too old now that 4.8
| is
| > something we could require and gain through that C++11 support, but we
| > shouldn't follow Rust with "you can only build it with 6 weeks old previous
| > release and nothing else".
| > As discussed earlier, we gain most through C++11 support, there is no need
| > to jump to C++17 or C++20 as requirement.
|
| Just a quick note.
| RHEL/CentOS 7 uses GCC 4.8 as the system compiler.  Requiring a new
| compiler to compile GCC 10 will not work for me.
| I normally bootstrap GCC 10 and then build a GCC 10 cross compiler.
| Having to have an extra compiler inbetween is problematic for me.
|
| Thanks,
| Andrew

I would think C++14 gives you a good compromise, as you have access to key C++11 functionalities with a less crippled constexpr support.

-- Gaby

Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Segher Boessenkool
In reply to this post by Jeff Law
On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:

> On 10/28/19 2:27 PM, Segher Boessenkool wrote:
> > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> >>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> >> The argument that I'd make is that's relatively uncommon (I know, I know
> >> AIX) that bootstrapping in those environments may well require first
> >> building something like gcc-9.
> >>
> >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> >> we have any good mechanism for making this kind of technical decision
> >> when there isn't consensus.
> >
> > Which GCC version will be required to work as bootstrap compiler?  Will
> > 4.8.5 be enough?
> I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> that old?

Many systems do not have a system compiler newer than this *four years old*
one.  GCC 4.8 is the first GCC version that supports all of C++11, which is
the only reason it would be even near acceptable to require something this
*new*.


Segher
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Richard Biener-2
In reply to this post by Jakub Jelinek
On Mon, Oct 28, 2019 at 10:47 PM Jakub Jelinek <[hidden email]> wrote:

>
> On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:
> > On 10/28/19 2:27 PM, Segher Boessenkool wrote:
> > > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> > >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> > >>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> > >> The argument that I'd make is that's relatively uncommon (I know, I know
> > >> AIX) that bootstrapping in those environments may well require first
> > >> building something like gcc-9.
> > >>
> > >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> > >> we have any good mechanism for making this kind of technical decision
> > >> when there isn't consensus.
> > >
> > > Which GCC version will be required to work as bootstrap compiler?  Will
> > > 4.8.5 be enough?
> > I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> > that old?
>
> That is not a good idea, it will make it much harder to build gcc because
> not everybody has gcc-9 built as a system compiler.
> The previous minimum requirement of 4.1 is perhaps too old now that 4.8 is
> something we could require and gain through that C++11 support, but we
> shouldn't follow Rust with "you can only build it with 6 weeks old previous
> release and nothing else".
> As discussed earlier, we gain most through C++11 support, there is no need
> to jump to C++17 or C++20 as requirement.

Yes, I've agreed to raise the requirement to GCC 4.8 which provides
C++11 support.

For convenience we could also provide a configure-time hint if the host compiler
doesn't have C++11 support or is older than 4.8.2 (I think .1 has some issues).
Rather than only running into some obscure errors later on.

Richard.

>         Jakub
>
Reply | Threaded
Open this post in threaded view
|

Re: GCC selftest improvements

Richard Biener-2
In reply to this post by Segher Boessenkool
On Mon, Oct 28, 2019 at 11:12 PM Segher Boessenkool
<[hidden email]> wrote:

>
> On Mon, Oct 28, 2019 at 03:41:13PM -0600, Jeff Law wrote:
> > On 10/28/19 2:27 PM, Segher Boessenkool wrote:
> > > On Mon, Oct 28, 2019 at 01:40:03PM -0600, Jeff Law wrote:
> > >> On 10/25/19 6:01 PM, Gabriel Dos Reis wrote:
> > >>> Jason, Jonathan - is the situation on the terrain really that dire that C++11 (or C++14) isn't at all available for platforms that GCC is bootstrapped from?
> > >> The argument that I'd make is that's relatively uncommon (I know, I know
> > >> AIX) that bootstrapping in those environments may well require first
> > >> building something like gcc-9.
> > >>
> > >> I'd really like to see us move to C++11 or beyond.  Sadly, I don't think
> > >> we have any good mechanism for making this kind of technical decision
> > >> when there isn't consensus.
> > >
> > > Which GCC version will be required to work as bootstrap compiler?  Will
> > > 4.8.5 be enough?
> > I'd say gcc-9.  What would we gain by making it 4.8 or anything else
> > that old?
>
> Many systems do not have a system compiler newer than this *four years old*
> one.  GCC 4.8 is the first GCC version that supports all of C++11, which is
> the only reason it would be even near acceptable to require something this
> *new*.

Agreed.  Note we're even shipping new service packs for SLE12 which has that
"ancient" compiler version (OTOH there _is_ a fully supported GCC 9 available
for SLE12 as well).

So, if we want C++11 then fine.  But requiring GCC 9+ isn't going to fly.  IIRC
GCC 6 is first having -std=c++14 by default, but unless there's a compelling
reason to use C++14 in GCC I'd rather not do it at this point.

Removing all the workarounds in the tree we have for GCC 4.[12].x would of
course be nice.

But I have to update the testers that still use GCC 4.1.x as host compiler :P

Richard.

>
> Segher
12