how to build and test uClinux toolchains

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

how to build and test uClinux toolchains

Christophe Lyon-2
Hi,

While reviewing one of my patches about FDPIC support for ARM, Richard
raised the concern of testing the patch on other uClinux targets [1].

I looked at uclinux.org and at the GCC maintainers file, but it's
still not obvious to me which uClinux targets are currently supported?

I'd like to do a basic:
- build toolchain for target XXX, make check
- apply my GCC patch
- build toolchain for target XXX, make check again
- compare the results to make sure I didn't break anything

I'd appreciate hints on which targets offer to do that easily enough,
using a simulator (qemu?).

Thanks,

Christophe


[1] https://gcc.gnu.org/ml/gcc-patches/2018-10/msg00736.html
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Jim Wilson-2
On 10/16/18 7:19 AM, Christophe Lyon wrote:
> While reviewing one of my patches about FDPIC support for ARM, Richard
> raised the concern of testing the patch on other uClinux targets [1].
>
> I looked at uclinux.org and at the GCC maintainers file, but it's
> still not obvious to me which uClinux targets are currently supported?

You should try asking the uclinux developers.

I tried looking at uclinux, and as far as I can tell, the best supported
targets are arm and m68k/coldfire.  crosstools-ng only supports one
uclinux target for instance, which is m68k.  qemu has m68k support, so
you could try that.  The other uclinux ports seem to be one time efforts
with no long term maintenance.  I see a lot of dead links on the
uclinux.org site, and a lot of stuff that hasn't been updated since 2004.

I see that buildroot has obvious blackfin (bfin), m68k, and xtensa
uclinux support.  But blackfin.uclinux.org says the uclinux port was
deprecated in 2012.  m68k as mentioned above should be usable.  It
appears that xtensa uclinux is still alive and usable.
     http://wiki.linux-xtensa.org/index.php/UClinux
There may be other uclinux targets that are usable but don't have
obvious patches to enable them.

Jim
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Max Filippov
Hello,

On Tue, Oct 16, 2018 at 1:54 PM Jim Wilson <[hidden email]> wrote:
>
> On 10/16/18 7:19 AM, Christophe Lyon wrote:
> > While reviewing one of my patches about FDPIC support for ARM, Richard
> > raised the concern of testing the patch on other uClinux targets [1].
> >
> > I looked at uclinux.org and at the GCC maintainers file, but it's
> > still not obvious to me which uClinux targets are currently supported?

I'm trying to keep nommu xtensa alive.

> I see that buildroot has obvious blackfin (bfin), m68k, and xtensa
> uclinux support.  But blackfin.uclinux.org says the uclinux port was
> deprecated in 2012.  m68k as mentioned above should be usable.  It
> appears that xtensa uclinux is still alive and usable.
>      http://wiki.linux-xtensa.org/index.php/UClinux

Probably the easiest way to get all xtensa toolchain parts correctly it
by using existing buildroot configuration. E.g. the following configuration
may be used to build uclinux xtensa toolchain for the dc233c core:
https://git.buildroot.net/buildroot/tree/configs/qemu_xtensa_lx60_nommu_defconfig

Also bFLT executable format is currently not supported for linux-user
xtensa QEMU. The following branch adds that support:
https://github.com/OSLL/qemu-xtensa/commits/xtensa-bflt

qemu-xtensa built from this QEMU then may be registered as a binfmt
handler for bFLT executable images allowing to run gcc tests that want
to run target binaries.
--
Thanks.
-- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Christophe Lyon-2
On Sun, 21 Oct 2018 at 04:06, Max Filippov <[hidden email]> wrote:

>
> Hello,
>
> On Tue, Oct 16, 2018 at 1:54 PM Jim Wilson <[hidden email]> wrote:
> >
> > On 10/16/18 7:19 AM, Christophe Lyon wrote:
> > > While reviewing one of my patches about FDPIC support for ARM, Richard
> > > raised the concern of testing the patch on other uClinux targets [1].
> > >
> > > I looked at uclinux.org and at the GCC maintainers file, but it's
> > > still not obvious to me which uClinux targets are currently supported?
>
> I'm trying to keep nommu xtensa alive.
>
> > I see that buildroot has obvious blackfin (bfin), m68k, and xtensa
> > uclinux support.  But blackfin.uclinux.org says the uclinux port was
> > deprecated in 2012.  m68k as mentioned above should be usable.  It
> > appears that xtensa uclinux is still alive and usable.
> >      http://wiki.linux-xtensa.org/index.php/UClinux
>
> Probably the easiest way to get all xtensa toolchain parts correctly it
> by using existing buildroot configuration. E.g. the following configuration
> may be used to build uclinux xtensa toolchain for the dc233c core:
> https://git.buildroot.net/buildroot/tree/configs/qemu_xtensa_lx60_nommu_defconfig
>
OK, thanks for your suggestion. I think I managed to build it.
Now, how/where can I run 'make check' for gcc?
I do not see the GCC build tree.

> Also bFLT executable format is currently not supported for linux-user
> xtensa QEMU. The following branch adds that support:
> https://github.com/OSLL/qemu-xtensa/commits/xtensa-bflt
>
> qemu-xtensa built from this QEMU then may be registered as a binfmt
> handler for bFLT executable images allowing to run gcc tests that want
> to run target binaries.
Do you have the magic commands for this?

Thanks,

Christophe

> --
> Thanks.
> -- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Max Filippov
On Fri, Oct 26, 2018 at 6:51 AM Christophe Lyon
<[hidden email]> wrote:
> On Sun, 21 Oct 2018 at 04:06, Max Filippov <[hidden email]> wrote:
> > Probably the easiest way to get all xtensa toolchain parts correctly it
> > by using existing buildroot configuration. E.g. the following configuration
> > may be used to build uclinux xtensa toolchain for the dc233c core:
> > https://git.buildroot.net/buildroot/tree/configs/qemu_xtensa_lx60_nommu_defconfig
> >
> OK, thanks for your suggestion. I think I managed to build it.
> Now, how/where can I run 'make check' for gcc?
> I do not see the GCC build tree.

The gcc build tree is usually in the build/host-gcc-final in the buildroot
build tree. But that's gcc version selected in the buildroot, you probably
want a different version. Usually after the buildroot toolchain is ready I
build gcc separately using binutils and sysroot produced by the buildroot.
I have a few examples here:

  http://wiki.osll.ru/doku.php/etc:users:jcmvbkbc:gcc-xtensa-call0

Please note that you'd need to apply gcc part of the xtensa overlay to
your gcc source for it to correctly generate code for that configuration.

I've run the tests with the current gcc trunk and a lot of execution
tests related to TLS (which is expected) and exceptions (which I
didn't expect) are failing. I'm looking at it.

> > Also bFLT executable format is currently not supported for linux-user
> > xtensa QEMU. The following branch adds that support:
> > https://github.com/OSLL/qemu-xtensa/commits/xtensa-bflt
> >
> > qemu-xtensa built from this QEMU then may be registered as a binfmt
> > handler for bFLT executable images allowing to run gcc tests that want
> > to run target binaries.
> Do you have the magic commands for this?

If you build QEMU from the link above you can use the following command
to register binfmt handler for bFLT binaries assuming that you've installed
it into $QEMU_PREFIX:

  sudo scripts/qemu-binfmt-conf.sh --qemu-path=$QEMU_PREFIX/bin --flat 'xtensa'

The --flat switch is not final, it will likely change before it's accepted to
the QEMU mainline.
--
Thanks.
-- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Christophe Lyon-2
On Fri, 26 Oct 2018 at 19:54, Max Filippov <[hidden email]> wrote:

>
> On Fri, Oct 26, 2018 at 6:51 AM Christophe Lyon
> <[hidden email]> wrote:
> > On Sun, 21 Oct 2018 at 04:06, Max Filippov <[hidden email]> wrote:
> > > Probably the easiest way to get all xtensa toolchain parts correctly it
> > > by using existing buildroot configuration. E.g. the following configuration
> > > may be used to build uclinux xtensa toolchain for the dc233c core:
> > > https://git.buildroot.net/buildroot/tree/configs/qemu_xtensa_lx60_nommu_defconfig
> > >
> > OK, thanks for your suggestion. I think I managed to build it.
> > Now, how/where can I run 'make check' for gcc?
> > I do not see the GCC build tree.
>
> The gcc build tree is usually in the build/host-gcc-final in the buildroot
> build tree. But that's gcc version selected in the buildroot, you probably
> want a different version. Usually after the buildroot toolchain is ready I
> build gcc separately using binutils and sysroot produced by the buildroot.
> I have a few examples here:
>
>   http://wiki.osll.ru/doku.php/etc:users:jcmvbkbc:gcc-xtensa-call0
>
> Please note that you'd need to apply gcc part of the xtensa overlay to
> your gcc source for it to correctly generate code for that configuration.
>
> I've run the tests with the current gcc trunk and a lot of execution
> tests related to TLS (which is expected) and exceptions (which I
> didn't expect) are failing. I'm looking at it.
>

I'm not sure if I followed the instructions correctly:
make qemu_xtensa_lx60_nommu_defconfig
make all
which built:
./output/host/bin/xtensa-buildroot-uclinux-uclibc-gcc (which is 7.3)
then I tried to follow the wiki above:
export TOOLCHAIN=$PWD/output
PATH=$TOOLCHAIN/host/bin:$PATH ..../gcc/configure [...]

I also built qemu from the branch you mentioned,
and used the "run it on linux-user QEMU" section in the wiki

I see many execution errors, now realizing that I didn't
do what you said above: "apply gcc part of the xtensa overlay".
But what is this? Do you mean the patches in buildroot/packages/gcc/8.2.0 ?
I tried to apply 0004-gcc-xtensa-fix-NAND-code-in-xtensa_expand_atomic.patch
but patch says it's already applied (I'm using GCC trunk)

And while writing this reply, I'm just realizing that buildroot builds
for uclinux-uclibc-gcc, while the wiki uses linux-uclibc :(
Does the wiki need an update wrt target name?

> > > Also bFLT executable format is currently not supported for linux-user
> > > xtensa QEMU. The following branch adds that support:
> > > https://github.com/OSLL/qemu-xtensa/commits/xtensa-bflt
> > >
> > > qemu-xtensa built from this QEMU then may be registered as a binfmt
> > > handler for bFLT executable images allowing to run gcc tests that want
> > > to run target binaries.
> > Do you have the magic commands for this?
>
> If you build QEMU from the link above you can use the following command
> to register binfmt handler for bFLT binaries assuming that you've installed
> it into $QEMU_PREFIX:
>
>   sudo scripts/qemu-binfmt-conf.sh --qemu-path=$QEMU_PREFIX/bin --flat 'xtensa'
>
> The --flat switch is not final, it will likely change before it's accepted to
> the QEMU mainline.
> --
> Thanks.
> -- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Christophe Lyon-2
On Wed, 31 Oct 2018 at 16:43, Christophe Lyon
<[hidden email]> wrote:

>
> On Fri, 26 Oct 2018 at 19:54, Max Filippov <[hidden email]> wrote:
> >
> > On Fri, Oct 26, 2018 at 6:51 AM Christophe Lyon
> > <[hidden email]> wrote:
> > > On Sun, 21 Oct 2018 at 04:06, Max Filippov <[hidden email]> wrote:
> > > > Probably the easiest way to get all xtensa toolchain parts correctly it
> > > > by using existing buildroot configuration. E.g. the following configuration
> > > > may be used to build uclinux xtensa toolchain for the dc233c core:
> > > > https://git.buildroot.net/buildroot/tree/configs/qemu_xtensa_lx60_nommu_defconfig
> > > >
> > > OK, thanks for your suggestion. I think I managed to build it.
> > > Now, how/where can I run 'make check' for gcc?
> > > I do not see the GCC build tree.
> >
> > The gcc build tree is usually in the build/host-gcc-final in the buildroot
> > build tree. But that's gcc version selected in the buildroot, you probably
> > want a different version. Usually after the buildroot toolchain is ready I
> > build gcc separately using binutils and sysroot produced by the buildroot.
> > I have a few examples here:
> >
> >   http://wiki.osll.ru/doku.php/etc:users:jcmvbkbc:gcc-xtensa-call0
> >
> > Please note that you'd need to apply gcc part of the xtensa overlay to
> > your gcc source for it to correctly generate code for that configuration.
> >
> > I've run the tests with the current gcc trunk and a lot of execution
> > tests related to TLS (which is expected) and exceptions (which I
> > didn't expect) are failing. I'm looking at it.
> >
>
> I'm not sure if I followed the instructions correctly:
> make qemu_xtensa_lx60_nommu_defconfig
> make all
> which built:
> ./output/host/bin/xtensa-buildroot-uclinux-uclibc-gcc (which is 7.3)
> then I tried to follow the wiki above:
> export TOOLCHAIN=$PWD/output
> PATH=$TOOLCHAIN/host/bin:$PATH ..../gcc/configure [...]
>
> I also built qemu from the branch you mentioned,
> and used the "run it on linux-user QEMU" section in the wiki
>
> I see many execution errors, now realizing that I didn't
> do what you said above: "apply gcc part of the xtensa overlay".
> But what is this? Do you mean the patches in buildroot/packages/gcc/8.2.0 ?
> I tried to apply 0004-gcc-xtensa-fix-NAND-code-in-xtensa_expand_atomic.patch
> but patch says it's already applied (I'm using GCC trunk)
>
> And while writing this reply, I'm just realizing that buildroot builds
> for uclinux-uclibc-gcc, while the wiki uses linux-uclibc :(
> Does the wiki need an update wrt target name?
>

I re-ran the wiki instructions with --target=xtensa-buildroot-uclinux-uclibc
and while gcc/g++ results looks mostly OK, the libstdc++ ones only show:
Running ...f/trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...
ERROR: could not compile testsuite_abi.cc
ERROR: tcl error sourcing.../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp.
Running.../trunk/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp ...
ERROR: could not compile testsuite_abi.cc
etc...

libstdc++.log show many instances of:
.../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
'symbols create_symbols(const char*)':^M
.../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:565: note:
non-delegitimized UNSPEC 3 found in variable location^M
.../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
'void examine_symbol(const char*, const char*)':^M
.../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:355: note:
non-delegitimized UNSPEC 3 found in variable location^M

ERROR: tcl error sourcing
.../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp.
ERROR: could not compile testsuite_abi.cc
    while executing
"error "could not compile $f""
    (procedure "v3-build_support" line 62)
    invoked from within
"v3-build_support"
    (file ".../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp" line 34)
    invoked from within
"source .../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp"
    ("uplevel" body line 1)
    invoked from within
"uplevel #0 source .../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp"
    invoked from within
"catch "uplevel #0 source $test_file_name""

Do you know what the problem is in my setup? Or with GCC trunk?

> > > > Also bFLT executable format is currently not supported for linux-user
> > > > xtensa QEMU. The following branch adds that support:
> > > > https://github.com/OSLL/qemu-xtensa/commits/xtensa-bflt
> > > >
> > > > qemu-xtensa built from this QEMU then may be registered as a binfmt
> > > > handler for bFLT executable images allowing to run gcc tests that want
> > > > to run target binaries.
> > > Do you have the magic commands for this?
> >
> > If you build QEMU from the link above you can use the following command
> > to register binfmt handler for bFLT binaries assuming that you've installed
> > it into $QEMU_PREFIX:
> >
> >   sudo scripts/qemu-binfmt-conf.sh --qemu-path=$QEMU_PREFIX/bin --flat 'xtensa'
> >
> > The --flat switch is not final, it will likely change before it's accepted to
> > the QEMU mainline.
> > --
> > Thanks.
> > -- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Max Filippov
In reply to this post by Christophe Lyon-2
On Wed, Oct 31, 2018 at 8:43 AM Christophe Lyon
<[hidden email]> wrote:
> I also built qemu from the branch you mentioned,
> and used the "run it on linux-user QEMU" section in the wiki
>
> I see many execution errors, now realizing that I didn't
> do what you said above: "apply gcc part of the xtensa overlay".
> But what is this?

It's a set of files that must be replaced in the toolchain source to configure
it for a specific xtensa core.
For gcc the config that you've built changes single file,
include/xtensa-config.h
and the change is the following:

--8<--
diff --git a/include/xtensa-config.h b/include/xtensa-config.h
index d6713d23a6b2..18899f827ea7 100644
--- a/include/xtensa-config.h
+++ b/include/xtensa-config.h
@@ -25,7 +25,7 @@
    macros.  */

 #undef XCHAL_HAVE_BE
-#define XCHAL_HAVE_BE                  1
+#define XCHAL_HAVE_BE                  0

 #undef XCHAL_HAVE_DENSITY
 #define XCHAL_HAVE_DENSITY             1

-->8--

> And while writing this reply, I'm just realizing that buildroot builds
> for uclinux-uclibc-gcc, while the wiki uses linux-uclibc :(
> Does the wiki need an update wrt target name?

Yeah, I used it as an example. Let me update it so that it works
for uclinux without obscure modifications.

I was able to figure out why C++ exceptions didn't work: because
the target code was built as PIC by default and it's not supported
by the bFLT file format. I've posted a fix here:

  https://gcc.gnu.org/ml/gcc-patches/2018-11/msg00286.html

--
Thanks.
-- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Max Filippov
In reply to this post by Christophe Lyon-2
On Fri, Nov 2, 2018 at 3:29 AM Christophe Lyon
<[hidden email]> wrote:

> I re-ran the wiki instructions with --target=xtensa-buildroot-uclinux-uclibc
> and while gcc/g++ results looks mostly OK, the libstdc++ ones only show:
> Running ...f/trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...
> ERROR: could not compile testsuite_abi.cc
> ERROR: tcl error sourcing.../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp.
> Running.../trunk/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp ...
> ERROR: could not compile testsuite_abi.cc
> etc...
>
> libstdc++.log show many instances of:
> .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
> 'symbols create_symbols(const char*)':^M
> .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:565: note:
> non-delegitimized UNSPEC 3 found in variable location^M
> .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
> 'void examine_symbol(const char*, const char*)':^M
> .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:355: note:
> non-delegitimized UNSPEC 3 found in variable location^M

"non-delegitimized UNSPEC 3 found" is a note, not an error.
There should also be an error.

> Do you know what the problem is in my setup? Or with GCC trunk?

I ran make check on gcc trunk from 2018-10-20, libstdc++ testsuite
works for me:

                === libstdc++ Summary ===

# of expected passes            9889
# of unexpected failures        89
# of unexpected successes       4
# of expected failures          79
# of unresolved testcases       17
# of unsupported tests          758

--
Thanks.
-- Max
Reply | Threaded
Open this post in threaded view
|

Re: how to build and test uClinux toolchains

Christophe Lyon-2
On Mon, 5 Nov 2018 at 21:49, Max Filippov <[hidden email]> wrote:

>
> On Fri, Nov 2, 2018 at 3:29 AM Christophe Lyon
> <[hidden email]> wrote:
> > I re-ran the wiki instructions with --target=xtensa-buildroot-uclinux-uclibc
> > and while gcc/g++ results looks mostly OK, the libstdc++ ones only show:
> > Running ...f/trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp ...
> > ERROR: could not compile testsuite_abi.cc
> > ERROR: tcl error sourcing.../trunk/libstdc++-v3/testsuite/libstdc++-abi/abi.exp.
> > Running.../trunk/libstdc++-v3/testsuite/libstdc++-dg/conformance.exp ...
> > ERROR: could not compile testsuite_abi.cc
> > etc...
> >
> > libstdc++.log show many instances of:
> > .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
> > 'symbols create_symbols(const char*)':^M
> > .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:565: note:
> > non-delegitimized UNSPEC 3 found in variable location^M
> > .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc: In function
> > 'void examine_symbol(const char*, const char*)':^M
> > .../trunk/libstdc++-v3/testsuite/util/testsuite_abi.cc:355: note:
> > non-delegitimized UNSPEC 3 found in variable location^M
>
> "non-delegitimized UNSPEC 3 found" is a note, not an error.
> There should also be an error.
>
That's what I thought, but I couldn't find it.

> > Do you know what the problem is in my setup? Or with GCC trunk?
>
> I ran make check on gcc trunk from 2018-10-20, libstdc++ testsuite
> works for me:
>
>                 === libstdc++ Summary ===
>
> # of expected passes            9889
> # of unexpected failures        89
> # of unexpected successes       4
> # of expected failures          79
> # of unresolved testcases       17
> # of unsupported tests          758
>

OK, thanks for the confirmation, I'm now re-building with the patches
from your previous email

> --
> Thanks.
> -- Max