libgo patch committed: Add precise stack scan support

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

libgo patch committed: Add precise stack scan support

Ian Lance Taylor-4
This libgo patch by Cherry Zhang adds support for precise stack
scanning to the Go runtime.  This uses per-function stack maps stored
in the exception tables in the language-specific data area.  The
compiler needs to generate these stack maps; currently this is only
done by a version of LLVM, not by GCC.  Each safepoint in a function
is associated with a (real or dummy) landing pad, and its "type info"
in the exception table is a pointer to the stack map. When a stack is
scanned, the stack map is found by the stack unwinding code.

For precise stack scan we need to unwind the stack. There are three cases:

- If a goroutine is scanning its own stack, it can unwind the stack
and scan the frames.

- If a goroutine is scanning another, stopped, goroutine, it cannot
directly unwind the target stack. We handle this by switching
(runtime.gogo) to the target g, letting it unwind and scan the stack,
and switch back.

- If we are scanning a goroutine that is blocked in a syscall, we send
a signal to the target goroutine's thread, and let the signal handler
unwind and scan the stack. Extra care is needed as this races with
enter/exit syscall.

Currently this is only implemented on GNU/Linux.

Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
to mainline.

Ian

patch.txt (35K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: libgo patch committed: Add precise stack scan support

Rainer Orth-2
Hi Ian,

> This libgo patch by Cherry Zhang adds support for precise stack
> scanning to the Go runtime.  This uses per-function stack maps stored
> in the exception tables in the language-specific data area.  The
> compiler needs to generate these stack maps; currently this is only
> done by a version of LLVM, not by GCC.  Each safepoint in a function
> is associated with a (real or dummy) landing pad, and its "type info"
> in the exception table is a pointer to the stack map. When a stack is
> scanned, the stack map is found by the stack unwinding code.
>
> For precise stack scan we need to unwind the stack. There are three cases:
>
> - If a goroutine is scanning its own stack, it can unwind the stack
> and scan the frames.
>
> - If a goroutine is scanning another, stopped, goroutine, it cannot
> directly unwind the target stack. We handle this by switching
> (runtime.gogo) to the target g, letting it unwind and scan the stack,
> and switch back.
>
> - If we are scanning a goroutine that is blocked in a syscall, we send
> a signal to the target goroutine's thread, and let the signal handler
> unwind and scan the stack. Extra care is needed as this races with
> enter/exit syscall.
>
> Currently this is only implemented on GNU/Linux.
>
> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> to mainline.
this broke Solaris (and other non-Linux) bootstrap:

/vol/gcc/src/hg/trunk/local/libgo/go/runtime/stubs_nonlinux.go:20:1: error: missing return at end of function
   20 | }
      | ^

Fixed by returning 0 for now, the return value is ignored in
go/runtime/proc.go (scang) anyway.

The Solaris equivalents would be thr_self (for gettid) and thr_kill (for
tgkill).  It were probably better to use non-Linux specific names
here...

        Rainer

--
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University



# HG changeset patch
# Parent  2c9abfeb2f017f5c48ccdee7ce72d72bcffb9250
Fix go/runtime/stubs_nonlinux.go compilation

diff --git a/libgo/go/runtime/stubs_nonlinux.go b/libgo/go/runtime/stubs_nonlinux.go
--- a/libgo/go/runtime/stubs_nonlinux.go
+++ b/libgo/go/runtime/stubs_nonlinux.go
@@ -17,4 +17,5 @@ func gettid() _pid_t {
 
 func tgkill(pid _pid_t, tid _pid_t, sig uint32) uint32 {
  throw("tgkill not implemented")
+ return 0;
 }
Reply | Threaded
Open this post in threaded view
|

Re: libgo patch committed: Add precise stack scan support

Ian Lance Taylor-4
On Fri, Dec 7, 2018 at 2:07 AM Rainer Orth <[hidden email]> wrote:

>
> > This libgo patch by Cherry Zhang adds support for precise stack
> > scanning to the Go runtime.  This uses per-function stack maps stored
> > in the exception tables in the language-specific data area.  The
> > compiler needs to generate these stack maps; currently this is only
> > done by a version of LLVM, not by GCC.  Each safepoint in a function
> > is associated with a (real or dummy) landing pad, and its "type info"
> > in the exception table is a pointer to the stack map. When a stack is
> > scanned, the stack map is found by the stack unwinding code.
> >
> > For precise stack scan we need to unwind the stack. There are three cases:
> >
> > - If a goroutine is scanning its own stack, it can unwind the stack
> > and scan the frames.
> >
> > - If a goroutine is scanning another, stopped, goroutine, it cannot
> > directly unwind the target stack. We handle this by switching
> > (runtime.gogo) to the target g, letting it unwind and scan the stack,
> > and switch back.
> >
> > - If we are scanning a goroutine that is blocked in a syscall, we send
> > a signal to the target goroutine's thread, and let the signal handler
> > unwind and scan the stack. Extra care is needed as this races with
> > enter/exit syscall.
> >
> > Currently this is only implemented on GNU/Linux.
> >
> > Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> > to mainline.
>
> this broke Solaris (and other non-Linux) bootstrap:
>
> /vol/gcc/src/hg/trunk/local/libgo/go/runtime/stubs_nonlinux.go:20:1: error: missing return at end of function
>    20 | }
>       | ^
>
> Fixed by returning 0 for now, the return value is ignored in
> go/runtime/proc.go (scang) anyway.

Thanks.  Committed to mainline.

Ian
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

gcc - patches mailing list
Sorry about the breakage. Thanks for the patch.


On Fri, Dec 7, 2018 at 9:23 AM Ian Lance Taylor <[hidden email]> wrote:

> On Fri, Dec 7, 2018 at 2:07 AM Rainer Orth <[hidden email]>
> wrote:
> >
> > > This libgo patch by Cherry Zhang adds support for precise stack
> > > scanning to the Go runtime.  This uses per-function stack maps stored
> > > in the exception tables in the language-specific data area.  The
> > > compiler needs to generate these stack maps; currently this is only
> > > done by a version of LLVM, not by GCC.  Each safepoint in a function
> > > is associated with a (real or dummy) landing pad, and its "type info"
> > > in the exception table is a pointer to the stack map. When a stack is
> > > scanned, the stack map is found by the stack unwinding code.
> > >
> > > For precise stack scan we need to unwind the stack. There are three
> cases:
> > >
> > > - If a goroutine is scanning its own stack, it can unwind the stack
> > > and scan the frames.
> > >
> > > - If a goroutine is scanning another, stopped, goroutine, it cannot
> > > directly unwind the target stack. We handle this by switching
> > > (runtime.gogo) to the target g, letting it unwind and scan the stack,
> > > and switch back.
> > >
> > > - If we are scanning a goroutine that is blocked in a syscall, we send
> > > a signal to the target goroutine's thread, and let the signal handler
> > > unwind and scan the stack. Extra care is needed as this races with
> > > enter/exit syscall.
> > >
> > > Currently this is only implemented on GNU/Linux.
> > >
> > > Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> > > to mainline.
> >
> > this broke Solaris (and other non-Linux) bootstrap:
> >
> > /vol/gcc/src/hg/trunk/local/libgo/go/runtime/stubs_nonlinux.go:20:1:
> error: missing return at end of function
> >    20 | }
> >       | ^
> >
> > Fixed by returning 0 for now, the return value is ignored in
> > go/runtime/proc <https://goto.google.com/runtime/proc>.go (scang)
> anyway.
>
> Thanks.  Committed to mainline.
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "gofrontend-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.
>
Reply | Threaded
Open this post in threaded view
|

Re: libgo patch committed: Add precise stack scan support

Matthias Klose-6
In reply to this post by Ian Lance Taylor-4
On 06.12.18 00:09, Ian Lance Taylor wrote:

> This libgo patch by Cherry Zhang adds support for precise stack
> scanning to the Go runtime.  This uses per-function stack maps stored
> in the exception tables in the language-specific data area.  The
> compiler needs to generate these stack maps; currently this is only
> done by a version of LLVM, not by GCC.  Each safepoint in a function
> is associated with a (real or dummy) landing pad, and its "type info"
> in the exception table is a pointer to the stack map. When a stack is
> scanned, the stack map is found by the stack unwinding code.
>
> For precise stack scan we need to unwind the stack. There are three cases:
>
> - If a goroutine is scanning its own stack, it can unwind the stack
> and scan the frames.
>
> - If a goroutine is scanning another, stopped, goroutine, it cannot
> directly unwind the target stack. We handle this by switching
> (runtime.gogo) to the target g, letting it unwind and scan the stack,
> and switch back.
>
> - If we are scanning a goroutine that is blocked in a syscall, we send
> a signal to the target goroutine's thread, and let the signal handler
> unwind and scan the stack. Extra care is needed as this races with
> enter/exit syscall.
>
> Currently this is only implemented on GNU/Linux.
>
> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> to mainline.

this broke the libgo build on ARM32:

../../../src/libgo/runtime/go-unwind.c: In function 'scanstackwithmap_callback':
../../../src/libgo/runtime/go-unwind.c:754:18: error: '_URC_NORMAL_STOP'
undeclared (first use in this function)
  754 |           return _URC_NORMAL_STOP;
      |                  ^~~~~~~~~~~~~~~~
../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared identifier
is reported only once for each function i
t appears in
../../../src/libgo/runtime/go-unwind.c: In function 'probestackmaps_callback':
../../../src/libgo/runtime/go-unwind.c:802:10: error: '_URC_NORMAL_STOP'
undeclared (first use in this function)
  802 |   return _URC_NORMAL_STOP;
      |          ^~~~~~~~~~~~~~~~
../../../src/libgo/runtime/go-unwind.c:803:1: warning: control reaches end of
non-void function [-Wreturn-type]
  803 | }
      | ^
make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
make[6]: Leaving directory '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'

Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

gcc - patches mailing list
On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]> wrote:

> On 06.12.18 00:09, Ian Lance Taylor wrote:
> > This libgo patch by Cherry Zhang adds support for precise stack
> > scanning to the Go runtime.  This uses per-function stack maps stored
> > in the exception tables in the language-specific data area.  The
> > compiler needs to generate these stack maps; currently this is only
> > done by a version of LLVM, not by GCC.  Each safepoint in a function
> > is associated with a (real or dummy) landing pad, and its "type info"
> > in the exception table is a pointer to the stack map. When a stack is
> > scanned, the stack map is found by the stack unwinding code.
> >
> > For precise stack scan we need to unwind the stack. There are three
> cases:
> >
> > - If a goroutine is scanning its own stack, it can unwind the stack
> > and scan the frames.
> >
> > - If a goroutine is scanning another, stopped, goroutine, it cannot
> > directly unwind the target stack. We handle this by switching
> > (runtime.gogo) to the target g, letting it unwind and scan the stack,
> > and switch back.
> >
> > - If we are scanning a goroutine that is blocked in a syscall, we send
> > a signal to the target goroutine's thread, and let the signal handler
> > unwind and scan the stack. Extra care is needed as this races with
> > enter/exit syscall.
> >
> > Currently this is only implemented on GNU/Linux.
> >
> > Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> > to mainline.
>
> this broke the libgo build on ARM32:
>
> ../../../src/libgo/runtime/go-unwind.c: In function
> 'scanstackwithmap_callback':
> ../../../src/libgo/runtime/go-unwind.c:754:18: error: '_URC_NORMAL_STOP'
> undeclared (first use in this function)
>   754 |           return _URC_NORMAL_STOP;
>       |                  ^~~~~~~~~~~~~~~~
> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
> identifier
> is reported only once for each function i
> t appears in
> ../../../src/libgo/runtime/go-unwind.c: In function
> 'probestackmaps_callback':
> ../../../src/libgo/runtime/go-unwind.c:802:10: error: '_URC_NORMAL_STOP'
> undeclared (first use in this function)
>   802 |   return _URC_NORMAL_STOP;
>       |          ^~~~~~~~~~~~~~~~
> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control reaches end
> of
> non-void function [-Wreturn-type]
>   803 | }
>       | ^
> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
> make[6]: Leaving directory
> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
>
>
Hell Matthias,

Thank you for the report. And sorry about the breakage. Does
https://go-review.googlesource.com/c/gofrontend/+/153417 (or the patch
below) fix ARM32 build? I don't have an ARM32 machine at hand to test.

Thanks,
Cherry

diff --git a/libgo/runtime/go-unwind.c b/libgo/runtime/go-unwind.c
index a1a95585..c44755f9 100644
--- a/libgo/runtime/go-unwind.c
+++ b/libgo/runtime/go-unwind.c
@@ -392,6 +392,12 @@ parse_lsda_header (struct _Unwind_Context *context,
const unsigned char *p,
 #define CONTINUE_UNWINDING return _URC_CONTINUE_UNWIND
 #endif

+#ifdef __ARM_EABI_UNWINDER__
+#define STOP_UNWINDING _URC_FAILURE
+#else
+#define STOP_UNWINDING _URC_NORMAL_STOP
+#endif
+
 #ifdef __USING_SJLJ_EXCEPTIONS__
 #define PERSONALITY_FUNCTION    __gccgo_personality_sj0
 #define __builtin_eh_return_data_regno(x) x
@@ -751,7 +757,7 @@ scanstackwithmap_callback (struct _Unwind_Context
*context, void *arg)
               // TODO: print gp, pc, sp
               runtime_throw ("no stack map");
             }
-          return _URC_NORMAL_STOP;
+          return STOP_UNWINDING;
         }
       case FOUND:
         break;
@@ -799,7 +805,7 @@ probestackmaps_callback (struct _Unwind_Context
*context,

   // Found a stack map. No need to keep unwinding.
   runtime_usestackmaps = true;
-  return _URC_NORMAL_STOP;
+  return STOP_UNWINDING;
 }

 // Try to find a stack map, store the result in global variable
runtime_usestackmaps.
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

Matthias Klose-6
On 10.12.18 16:54, Cherry Zhang wrote:

> On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]> wrote:
>
>> On 06.12.18 00:09, Ian Lance Taylor wrote:
>>> This libgo patch by Cherry Zhang adds support for precise stack
>>> scanning to the Go runtime.  This uses per-function stack maps stored
>>> in the exception tables in the language-specific data area.  The
>>> compiler needs to generate these stack maps; currently this is only
>>> done by a version of LLVM, not by GCC.  Each safepoint in a function
>>> is associated with a (real or dummy) landing pad, and its "type info"
>>> in the exception table is a pointer to the stack map. When a stack is
>>> scanned, the stack map is found by the stack unwinding code.
>>>
>>> For precise stack scan we need to unwind the stack. There are three
>> cases:
>>>
>>> - If a goroutine is scanning its own stack, it can unwind the stack
>>> and scan the frames.
>>>
>>> - If a goroutine is scanning another, stopped, goroutine, it cannot
>>> directly unwind the target stack. We handle this by switching
>>> (runtime.gogo) to the target g, letting it unwind and scan the stack,
>>> and switch back.
>>>
>>> - If we are scanning a goroutine that is blocked in a syscall, we send
>>> a signal to the target goroutine's thread, and let the signal handler
>>> unwind and scan the stack. Extra care is needed as this races with
>>> enter/exit syscall.
>>>
>>> Currently this is only implemented on GNU/Linux.
>>>
>>> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
>>> to mainline.
>>
>> this broke the libgo build on ARM32:
>>
>> ../../../src/libgo/runtime/go-unwind.c: In function
>> 'scanstackwithmap_callback':
>> ../../../src/libgo/runtime/go-unwind.c:754:18: error: '_URC_NORMAL_STOP'
>> undeclared (first use in this function)
>>   754 |           return _URC_NORMAL_STOP;
>>       |                  ^~~~~~~~~~~~~~~~
>> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
>> identifier
>> is reported only once for each function i
>> t appears in
>> ../../../src/libgo/runtime/go-unwind.c: In function
>> 'probestackmaps_callback':
>> ../../../src/libgo/runtime/go-unwind.c:802:10: error: '_URC_NORMAL_STOP'
>> undeclared (first use in this function)
>>   802 |   return _URC_NORMAL_STOP;
>>       |          ^~~~~~~~~~~~~~~~
>> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control reaches end
>> of
>> non-void function [-Wreturn-type]
>>   803 | }
>>       | ^
>> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
>> make[6]: Leaving directory
>> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
>>
>>
> Hell Matthias,
>
> Thank you for the report. And sorry about the breakage. Does
> https://go-review.googlesource.com/c/gofrontend/+/153417 (or the patch
> below) fix ARM32 build? I don't have an ARM32 machine at hand to test.

this fixes the build.

currently running the testsuite, almost every test case core dumps on
arm-linux-gnueabihf

Matthias
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

Ian Lance Taylor-4
On Tue, Dec 11, 2018 at 6:52 AM Matthias Klose <[hidden email]> wrote:

>
> On 10.12.18 16:54, Cherry Zhang wrote:
> > On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]> wrote:
> >
> >> On 06.12.18 00:09, Ian Lance Taylor wrote:
> >>> This libgo patch by Cherry Zhang adds support for precise stack
> >>> scanning to the Go runtime.  This uses per-function stack maps stored
> >>> in the exception tables in the language-specific data area.  The
> >>> compiler needs to generate these stack maps; currently this is only
> >>> done by a version of LLVM, not by GCC.  Each safepoint in a function
> >>> is associated with a (real or dummy) landing pad, and its "type info"
> >>> in the exception table is a pointer to the stack map. When a stack is
> >>> scanned, the stack map is found by the stack unwinding code.
> >>>
> >>> For precise stack scan we need to unwind the stack. There are three
> >> cases:
> >>>
> >>> - If a goroutine is scanning its own stack, it can unwind the stack
> >>> and scan the frames.
> >>>
> >>> - If a goroutine is scanning another, stopped, goroutine, it cannot
> >>> directly unwind the target stack. We handle this by switching
> >>> (runtime.gogo) to the target g, letting it unwind and scan the stack,
> >>> and switch back.
> >>>
> >>> - If we are scanning a goroutine that is blocked in a syscall, we send
> >>> a signal to the target goroutine's thread, and let the signal handler
> >>> unwind and scan the stack. Extra care is needed as this races with
> >>> enter/exit syscall.
> >>>
> >>> Currently this is only implemented on GNU/Linux.
> >>>
> >>> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> >>> to mainline.
> >>
> >> this broke the libgo build on ARM32:
> >>
> >> ../../../src/libgo/runtime/go-unwind.c: In function
> >> 'scanstackwithmap_callback':
> >> ../../../src/libgo/runtime/go-unwind.c:754:18: error: '_URC_NORMAL_STOP'
> >> undeclared (first use in this function)
> >>   754 |           return _URC_NORMAL_STOP;
> >>       |                  ^~~~~~~~~~~~~~~~
> >> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
> >> identifier
> >> is reported only once for each function i
> >> t appears in
> >> ../../../src/libgo/runtime/go-unwind.c: In function
> >> 'probestackmaps_callback':
> >> ../../../src/libgo/runtime/go-unwind.c:802:10: error: '_URC_NORMAL_STOP'
> >> undeclared (first use in this function)
> >>   802 |   return _URC_NORMAL_STOP;
> >>       |          ^~~~~~~~~~~~~~~~
> >> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control reaches end
> >> of
> >> non-void function [-Wreturn-type]
> >>   803 | }
> >>       | ^
> >> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
> >> make[6]: Leaving directory
> >> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
> >>
> >>
> > Hell Matthias,
> >
> > Thank you for the report. And sorry about the breakage. Does
> > https://go-review.googlesource.com/c/gofrontend/+/153417 (or the patch
> > below) fix ARM32 build? I don't have an ARM32 machine at hand to test.
>
> this fixes the build.
>
> currently running the testsuite, almost every test case core dumps on
> arm-linux-gnueabihf

I committed Cherry's patch to trunk, since it looks reasonable to me.
Cherry, Matthias, let me know if you figure out why programs are
failing.

Ian
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

gcc - patches mailing list
On Tue, Dec 11, 2018 at 3:51 PM Ian Lance Taylor <[hidden email]> wrote:

> On Tue, Dec 11, 2018 at 6:52 AM Matthias Klose <[hidden email]> wrote:
> >
> > On 10.12.18 16:54, Cherry Zhang wrote:
> > > On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]>
> wrote:
> > >
> > >> On 06.12.18 00:09, Ian Lance Taylor wrote:
> > >>> This libgo patch by Cherry Zhang adds support for precise stack
> > >>> scanning to the Go runtime.  This uses per-function stack maps stored
> > >>> in the exception tables in the language-specific data area.  The
> > >>> compiler needs to generate these stack maps; currently this is only
> > >>> done by a version of LLVM, not by GCC.  Each safepoint in a function
> > >>> is associated with a (real or dummy) landing pad, and its "type info"
> > >>> in the exception table is a pointer to the stack map. When a stack is
> > >>> scanned, the stack map is found by the stack unwinding code.
> > >>>
> > >>> For precise stack scan we need to unwind the stack. There are three
> > >> cases:
> > >>>
> > >>> - If a goroutine is scanning its own stack, it can unwind the stack
> > >>> and scan the frames.
> > >>>
> > >>> - If a goroutine is scanning another, stopped, goroutine, it cannot
> > >>> directly unwind the target stack. We handle this by switching
> > >>> (runtime.gogo) to the target g, letting it unwind and scan the stack,
> > >>> and switch back.
> > >>>
> > >>> - If we are scanning a goroutine that is blocked in a syscall, we
> send
> > >>> a signal to the target goroutine's thread, and let the signal handler
> > >>> unwind and scan the stack. Extra care is needed as this races with
> > >>> enter/exit syscall.
> > >>>
> > >>> Currently this is only implemented on GNU/Linux.
> > >>>
> > >>> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> > >>> to mainline.
> > >>
> > >> this broke the libgo build on ARM32:
> > >>
> > >> ../../../src/libgo/runtime/go-unwind.c: In function
> > >> 'scanstackwithmap_callback':
> > >> ../../../src/libgo/runtime/go-unwind.c:754:18: error:
> '_URC_NORMAL_STOP'
> > >> undeclared (first use in this function)
> > >>   754 |           return _URC_NORMAL_STOP;
> > >>       |                  ^~~~~~~~~~~~~~~~
> > >> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
> > >> identifier
> > >> is reported only once for each function i
> > >> t appears in
> > >> ../../../src/libgo/runtime/go-unwind.c: In function
> > >> 'probestackmaps_callback':
> > >> ../../../src/libgo/runtime/go-unwind.c:802:10: error:
> '_URC_NORMAL_STOP'
> > >> undeclared (first use in this function)
> > >>   802 |   return _URC_NORMAL_STOP;
> > >>       |          ^~~~~~~~~~~~~~~~
> > >> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control
> reaches end
> > >> of
> > >> non-void function [-Wreturn-type]
> > >>   803 | }
> > >>       | ^
> > >> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
> > >> make[6]: Leaving directory
> > >> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
> > >>
> > >>
> > > Hell Matthias,
> > >
> > > Thank you for the report. And sorry about the breakage. Does
> > > https://go-review.googlesource.com/c/gofrontend/+/153417 (or the patch
> > > below) fix ARM32 build? I don't have an ARM32 machine at hand to test.
> >
> > this fixes the build.
> >
> > currently running the testsuite, almost every test case core dumps on
> > arm-linux-gnueabihf
>
> I committed Cherry's patch to trunk, since it looks reasonable to me.
> Cherry, Matthias, let me know if you figure out why programs are
> failing.
>
>
Thank you.

I don't know for the moment. I'm trying to find an ARM32 machine so I can
test.

Matthias, is it convenient for you to share a stack trace for the failing
programs? That would be very helpful. Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

Matthias Klose-6
On 11.12.18 22:01, Cherry Zhang wrote:

> On Tue, Dec 11, 2018 at 3:51 PM Ian Lance Taylor <[hidden email]> wrote:
>
>> On Tue, Dec 11, 2018 at 6:52 AM Matthias Klose <[hidden email]> wrote:
>>>
>>> On 10.12.18 16:54, Cherry Zhang wrote:
>>>> On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]>
>> wrote:
>>>>
>>>>> On 06.12.18 00:09, Ian Lance Taylor wrote:
>>>>>> This libgo patch by Cherry Zhang adds support for precise stack
>>>>>> scanning to the Go runtime.  This uses per-function stack maps stored
>>>>>> in the exception tables in the language-specific data area.  The
>>>>>> compiler needs to generate these stack maps; currently this is only
>>>>>> done by a version of LLVM, not by GCC.  Each safepoint in a function
>>>>>> is associated with a (real or dummy) landing pad, and its "type info"
>>>>>> in the exception table is a pointer to the stack map. When a stack is
>>>>>> scanned, the stack map is found by the stack unwinding code.
>>>>>>
>>>>>> For precise stack scan we need to unwind the stack. There are three
>>>>> cases:
>>>>>>
>>>>>> - If a goroutine is scanning its own stack, it can unwind the stack
>>>>>> and scan the frames.
>>>>>>
>>>>>> - If a goroutine is scanning another, stopped, goroutine, it cannot
>>>>>> directly unwind the target stack. We handle this by switching
>>>>>> (runtime.gogo) to the target g, letting it unwind and scan the stack,
>>>>>> and switch back.
>>>>>>
>>>>>> - If we are scanning a goroutine that is blocked in a syscall, we
>> send
>>>>>> a signal to the target goroutine's thread, and let the signal handler
>>>>>> unwind and scan the stack. Extra care is needed as this races with
>>>>>> enter/exit syscall.
>>>>>>
>>>>>> Currently this is only implemented on GNU/Linux.
>>>>>>
>>>>>> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
>>>>>> to mainline.
>>>>>
>>>>> this broke the libgo build on ARM32:
>>>>>
>>>>> ../../../src/libgo/runtime/go-unwind.c: In function
>>>>> 'scanstackwithmap_callback':
>>>>> ../../../src/libgo/runtime/go-unwind.c:754:18: error:
>> '_URC_NORMAL_STOP'
>>>>> undeclared (first use in this function)
>>>>>   754 |           return _URC_NORMAL_STOP;
>>>>>       |                  ^~~~~~~~~~~~~~~~
>>>>> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
>>>>> identifier
>>>>> is reported only once for each function i
>>>>> t appears in
>>>>> ../../../src/libgo/runtime/go-unwind.c: In function
>>>>> 'probestackmaps_callback':
>>>>> ../../../src/libgo/runtime/go-unwind.c:802:10: error:
>> '_URC_NORMAL_STOP'
>>>>> undeclared (first use in this function)
>>>>>   802 |   return _URC_NORMAL_STOP;
>>>>>       |          ^~~~~~~~~~~~~~~~
>>>>> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control
>> reaches end
>>>>> of
>>>>> non-void function [-Wreturn-type]
>>>>>   803 | }
>>>>>       | ^
>>>>> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
>>>>> make[6]: Leaving directory
>>>>> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
>>>>>
>>>>>
>>>> Hell Matthias,
>>>>
>>>> Thank you for the report. And sorry about the breakage. Does
>>>> https://go-review.googlesource.com/c/gofrontend/+/153417 (or the patch
>>>> below) fix ARM32 build? I don't have an ARM32 machine at hand to test.
>>>
>>> this fixes the build.
>>>
>>> currently running the testsuite, almost every test case core dumps on
>>> arm-linux-gnueabihf
>>
>> I committed Cherry's patch to trunk, since it looks reasonable to me.
>> Cherry, Matthias, let me know if you figure out why programs are
>> failing.
>>
>>
> Thank you.
>
> I don't know for the moment. I'm trying to find an ARM32 machine so I can
> test.
>
> Matthias, is it convenient for you to share a stack trace for the failing
> programs? That would be very helpful. Thanks!

I'll do a local build this weekend. For now you find the build log at
https://launchpad.net/ubuntu/+source/gcc-snapshot/1:20181210-0ubuntu1/+build/15759748
Reply | Threaded
Open this post in threaded view
|

Re: [gofrontend-dev] Re: libgo patch committed: Add precise stack scan support

gcc - patches mailing list
Thank you, Matthias!

From the log, essentially all the tests aborted. The only place the new
code can cause abort on all programs that I can think of is in the runtime
startup code, probestackmaps, which calls value_size, which aborts due to
an unhandled case. I haven't been able to try out on an ARM machine, but I
managed to cross-compile a Go program and visually inspect the exception
table. The type table's encoding is DW_EH_PE_absptr, which is indeed not
handled, which will cause abort.

I send https://go-review.googlesource.com/c/gofrontend/+/153857 (also as
below). Hopefully this will fix the problem.

Thanks,
Cherry

diff --git a/libgo/runtime/go-unwind.c b/libgo/runtime/go-unwind.c
index c44755f9..f4bbfb60 100644
--- a/libgo/runtime/go-unwind.c
+++ b/libgo/runtime/go-unwind.c
@@ -318,6 +318,8 @@ value_size (uint8_t encoding)
       case DW_EH_PE_sdata8:
       case DW_EH_PE_udata8:
         return 8;
+      case DW_EH_PE_absptr:
+        return sizeof(uintptr);
       default:
         break;
     }



On Tue, Dec 11, 2018 at 7:03 PM Matthias Klose <[hidden email]> wrote:

> On 11.12.18 22:01, Cherry Zhang wrote:
> > On Tue, Dec 11, 2018 at 3:51 PM Ian Lance Taylor <[hidden email]>
> wrote:
> >
> >> On Tue, Dec 11, 2018 at 6:52 AM Matthias Klose <[hidden email]> wrote:
> >>>
> >>> On 10.12.18 16:54, Cherry Zhang wrote:
> >>>> On Mon, Dec 10, 2018 at 1:41 AM Matthias Klose <[hidden email]>
> >> wrote:
> >>>>
> >>>>> On 06.12.18 00:09, Ian Lance Taylor wrote:
> >>>>>> This libgo patch by Cherry Zhang adds support for precise stack
> >>>>>> scanning to the Go runtime.  This uses per-function stack maps
> stored
> >>>>>> in the exception tables in the language-specific data area.  The
> >>>>>> compiler needs to generate these stack maps; currently this is only
> >>>>>> done by a version of LLVM, not by GCC.  Each safepoint in a function
> >>>>>> is associated with a (real or dummy) landing pad, and its "type
> info"
> >>>>>> in the exception table is a pointer to the stack map. When a stack
> is
> >>>>>> scanned, the stack map is found by the stack unwinding code.
> >>>>>>
> >>>>>> For precise stack scan we need to unwind the stack. There are three
> >>>>> cases:
> >>>>>>
> >>>>>> - If a goroutine is scanning its own stack, it can unwind the stack
> >>>>>> and scan the frames.
> >>>>>>
> >>>>>> - If a goroutine is scanning another, stopped, goroutine, it cannot
> >>>>>> directly unwind the target stack. We handle this by switching
> >>>>>> (runtime.gogo) to the target g, letting it unwind and scan the
> stack,
> >>>>>> and switch back.
> >>>>>>
> >>>>>> - If we are scanning a goroutine that is blocked in a syscall, we
> >> send
> >>>>>> a signal to the target goroutine's thread, and let the signal
> handler
> >>>>>> unwind and scan the stack. Extra care is needed as this races with
> >>>>>> enter/exit syscall.
> >>>>>>
> >>>>>> Currently this is only implemented on GNU/Linux.
> >>>>>>
> >>>>>> Bootstrapped and ran Go testsuite on x86_64-pc-linux-gnu.  Committed
> >>>>>> to mainline.
> >>>>>
> >>>>> this broke the libgo build on ARM32:
> >>>>>
> >>>>> ../../../src/libgo/runtime/go-unwind.c: In function
> >>>>> 'scanstackwithmap_callback':
> >>>>> ../../../src/libgo/runtime/go-unwind.c:754:18: error:
> >> '_URC_NORMAL_STOP'
> >>>>> undeclared (first use in this function)
> >>>>>   754 |           return _URC_NORMAL_STOP;
> >>>>>       |                  ^~~~~~~~~~~~~~~~
> >>>>> ../../../src/libgo/runtime/go-unwind.c:754:18: note: each undeclared
> >>>>> identifier
> >>>>> is reported only once for each function i
> >>>>> t appears in
> >>>>> ../../../src/libgo/runtime/go-unwind.c: In function
> >>>>> 'probestackmaps_callback':
> >>>>> ../../../src/libgo/runtime/go-unwind.c:802:10: error:
> >> '_URC_NORMAL_STOP'
> >>>>> undeclared (first use in this function)
> >>>>>   802 |   return _URC_NORMAL_STOP;
> >>>>>       |          ^~~~~~~~~~~~~~~~
> >>>>> ../../../src/libgo/runtime/go-unwind.c:803:1: warning: control
> >> reaches end
> >>>>> of
> >>>>> non-void function [-Wreturn-type]
> >>>>>   803 | }
> >>>>>       | ^
> >>>>> make[6]: *** [Makefile:1474: runtime/go-unwind.lo] Error 1
> >>>>> make[6]: Leaving directory
> >>>>> '/<<PKGBUILDDIR>>/build/arm-linux-gnueabihf/libgo'
> >>>>>
> >>>>>
> >>>> Hell Matthias,
> >>>>
> >>>> Thank you for the report. And sorry about the breakage. Does
> >>>> https://go-review.googlesource.com/c/gofrontend/+/153417 (or the
> patch
> >>>> below) fix ARM32 build? I don't have an ARM32 machine at hand to test.
> >>>
> >>> this fixes the build.
> >>>
> >>> currently running the testsuite, almost every test case core dumps on
> >>> arm-linux-gnueabihf
> >>
> >> I committed Cherry's patch to trunk, since it looks reasonable to me.
> >> Cherry, Matthias, let me know if you figure out why programs are
> >> failing.
> >>
> >>
> > Thank you.
> >
> > I don't know for the moment. I'm trying to find an ARM32 machine so I can
> > test.
> >
> > Matthias, is it convenient for you to share a stack trace for the failing
> > programs? That would be very helpful. Thanks!
>
> I'll do a local build this weekend. For now you find the build log at
>
> https://launchpad.net/ubuntu/+source/gcc-snapshot/1:20181210-0ubuntu1/+build/15759748
>