git conversion of GCC wwwdocs repository

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

git conversion of GCC wwwdocs repository

Joseph Myers
I've done a git conversion of the GCC wwwdocs repository (from CVS, using
cvs-fast-export), plus written a git post-receive hook intended to
correspond to what the CVS hook does.  (For the CVS hook, see loginfo in
the CVSROOT, plus the scripts such as htdocs-checkout referenced from
there.)

To access the test conversion (given SSH access to gcc.gnu.org), do:

git clone git+ssh://gcc.gnu.org/home/gccadmin/wwwdocs-test.git

(This only needs the usual restricted SSH access used to commit to GCC,
not shell access.)

Note that the test repository does *not* have the post-receive hook
enabled; pushing commits to it won't do anything useful; it's simply there
to make it convenient to review the converted history.  In practice, I
expect the post-receive hook may need bugs ironed out once in use in
production, because it's hard to test when not hooked up to the real
website.

I've attached the conversion scripts and proposed post-receive hook.  
do-download checks out cvs-fast-export and ESR's gcc-conversion repository
(for the author map) and rsyncs the wwwdocs CVS repository.  do-convert
builds cvs-fast-export and does the conversion (with a few extra authors
in the author map, where e.g. new since the last update to the author map
or only ever committed to wwwdocs not GCC sources; the names for htdigid
and www are somewhat arbitrary, www commits would actually have been some
human or humans committing as that user by accident); once cvs-fast-export
has been built, this takes about 15 seconds.  All non-master tags and
branches are removed, since they are clearly just accidents from tags /
branches having been applied to the whole CVS repository rather than just
to the GCC sources proper.

post-receive is the proposed hook.  It calls post-receive-email to send
email (could no doubt be adjusted to use AdaCore hooks or any other
email-sending system), then follows logic intended to replicate the old
hooks to update a checkout of wwwdocs and call the preprocess script on
new or modified files.

I'd expect the post-receive script to go in wwwdocs/bin (with the hook
directory having a symlink to there).  There would of course be website
changes needed to update the references to CVS, and the preprocess script
would change to ignore .git when doing global preprocessing on the site.

I don't know whether either the system for updating www.gnu.org's copy of
the site, or Gerald's HTML validation system, would need updating for a
move of wwwdocs to git.

Any comments on the conversion or hook?

--
Joseph S. Myers
[hidden email]

do-download (372 bytes) Download Attachment
do-convert (1K) Download Attachment
post-receive (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Joseph Myers
Would anyone like to make any comments on this conversion from CVS to git?

--
Joseph S. Myers
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Jonathan Wakely-4
On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
>
> Would anyone like to make any comments on this conversion from CVS to git?

It looks pretty good. I note that the author map just uses the
committer's current email address, meaning I have commits using my
@redhat.com address nearly a decade before I started working at Red
Hat. But that's a small price to pay for moving from CVS to Git in my
opinion. And t's arguably correct to have all my commits under one
identity rather than several different ones anyway.
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Joseph Myers
On Tue, 24 Sep 2019, Jonathan Wakely wrote:

> On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
> >
> > Would anyone like to make any comments on this conversion from CVS to git?
>
> It looks pretty good. I note that the author map just uses the
> committer's current email address, meaning I have commits using my
> @redhat.com address nearly a decade before I started working at Red
> Hat. But that's a small price to pay for moving from CVS to Git in my
> opinion. And t's arguably correct to have all my commits under one
> identity rather than several different ones anyway.

If people prefer to use @gcc.gnu.org addresses, that's a small matter of
adjusting the postprocessing of the author map from the gcc-conversion
repository to replace the addresses there with [hidden email] while
keeping the names from that author map.  (The postprocessing is needed
anyway for this conversion because cvs-fast-export uses a more restricted
author map syntax than reposurgeon does.  Because there is no ChangeLog in
wwwdocs, more sophisticated systems for identifying the relevant email
address for each commit from the ChangeLog aren't practical the same way
they are for the main GCC repository.)

--
Joseph S. Myers
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Jonathan Wakely-4
On Tue, 24 Sep 2019 at 23:15, Joseph Myers wrote:

>
> On Tue, 24 Sep 2019, Jonathan Wakely wrote:
>
> > On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
> > >
> > > Would anyone like to make any comments on this conversion from CVS to git?
> >
> > It looks pretty good. I note that the author map just uses the
> > committer's current email address, meaning I have commits using my
> > @redhat.com address nearly a decade before I started working at Red
> > Hat. But that's a small price to pay for moving from CVS to Git in my
> > opinion. And t's arguably correct to have all my commits under one
> > identity rather than several different ones anyway.
>
> If people prefer to use @gcc.gnu.org addresses, that's a small matter of
> adjusting the postprocessing of the author map from the gcc-conversion
> repository to replace the addresses there with [hidden email] while
> keeping the names from that author map.

I don't feel strongly about it.
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Jason Merrill
In reply to this post by Joseph Myers
On Tue, Sep 24, 2019 at 6:16 PM Joseph Myers <[hidden email]> wrote:

> On Tue, 24 Sep 2019, Jonathan Wakely wrote:
> > On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
> > >
> > > Would anyone like to make any comments on this conversion from CVS to git?
> >
> > It looks pretty good. I note that the author map just uses the
> > committer's current email address, meaning I have commits using my
> > @redhat.com address nearly a decade before I started working at Red
> > Hat. But that's a small price to pay for moving from CVS to Git in my
> > opinion. And t's arguably correct to have all my commits under one
> > identity rather than several different ones anyway.
>
> If people prefer to use @gcc.gnu.org addresses, that's a small matter of
> adjusting the postprocessing of the author map from the gcc-conversion
> repository to replace the addresses there with [hidden email] while
> keeping the names from that author map.  (The postprocessing is needed
> anyway for this conversion because cvs-fast-export uses a more restricted
> author map syntax than reposurgeon does.  Because there is no ChangeLog in
> wwwdocs, more sophisticated systems for identifying the relevant email
> address for each commit from the ChangeLog aren't practical the same way
> they are for the main GCC repository.)

That would be my preference.

Jason

Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Thomas Schwinge-8
Hi!

On 2019-09-24T22:51:18-0400, Jason Merrill <[hidden email]> wrote:
> On Tue, Sep 24, 2019 at 6:16 PM Joseph Myers <[hidden email]> wrote:
>> On Tue, 24 Sep 2019, Jonathan Wakely wrote:
>> > On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
>> > > Would anyone like to make any comments on this conversion from CVS to git?

Again, thanks for working on this!  The conversion seems good to me (but
I only had a quick look).


As mentioned at the Cauldron, I shall uninstall CVS from my systems once
this is done.  ;-)


>> > I note that the author map just uses the
>> > committer's current email address, meaning I have commits using my
>> > @redhat.com address nearly a decade before I started working at Red
>> > Hat. But that's a small price to pay for moving from CVS to Git in my
>> > opinion. And t's arguably correct to have all my commits under one
>> > identity rather than several different ones anyway.

(That last sentence/point I don't understand.)

>> If people prefer to use @gcc.gnu.org addresses, that's a small matter of
>> adjusting the postprocessing of the author map from the gcc-conversion
>> repository to replace the addresses there with [hidden email] while
>> keeping the names from that author map.  (The postprocessing is needed
>> anyway for this conversion because cvs-fast-export uses a more restricted
>> author map syntax than reposurgeon does.  Because there is no ChangeLog in
>> wwwdocs, more sophisticated systems for identifying the relevant email
>> address for each commit from the ChangeLog aren't practical the same way
>> they are for the main GCC repository.)
>
> That would be my preference.

It's also my preference to just use '@gcc.gnu.org' addresses, with or
even without user names mapped/prepended.

Or even use an "unknown" Git author for all Git commits, and use each
revision's CVS author for the Git committer.  After all, we can't be sure
that the CVS author is the actual author (and thus Git author) of the
respective CVS revision.

(That'd also have been my pragmatic suggestion for the main GCC
reporitory conversion, but I see that people now have invested effort in
a more sophisticated solution.)


Grüße
 Thomas
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Jonathan Wakely-4
On Wed, 25 Sep 2019 at 11:08, Thomas Schwinge <[hidden email]> wrote:

>
> Hi!
>
> On 2019-09-24T22:51:18-0400, Jason Merrill <[hidden email]> wrote:
> > On Tue, Sep 24, 2019 at 6:16 PM Joseph Myers <[hidden email]> wrote:
> >> On Tue, 24 Sep 2019, Jonathan Wakely wrote:
> >> > On Tue, 24 Sep 2019 at 21:46, Joseph Myers wrote:
> >> > > Would anyone like to make any comments on this conversion from CVS to git?
>
> Again, thanks for working on this!  The conversion seems good to me (but
> I only had a quick look).
>
>
> As mentioned at the Cauldron, I shall uninstall CVS from my systems once
> this is done.  ;-)

Me too! :-)

> >> > I note that the author map just uses the
> >> > committer's current email address, meaning I have commits using my
> >> > @redhat.com address nearly a decade before I started working at Red
> >> > Hat. But that's a small price to pay for moving from CVS to Git in my
> >> > opinion. And t's arguably correct to have all my commits under one
> >> > identity rather than several different ones anyway.
>
> (That last sentence/point I don't understand.)

I mean that there's not much value in having my past commits listed as
coming from various "different" authors:

Jonathan Wakely <xxx at compsoc.man.ac.uk>
Jonathan Wakely <xxx at kayari.org>
Jonathan Wakely <xxx at gmail.com>
Jonathan Wakely <xxx at gcc.gnu.org>
Jonathan Wakely <xxx at redhat.com>

All of those "identities" are the same person, so making all my
commits use the same Author is arguably correct, even if I was using
different addresses at different times.

Using the @gcc.gnu.org address for all my past commits might be the
least worst option, because it's "me" and that address was valid in
2004 and is still valid now, unlike the other options.
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Joseph Myers-2
A version of the conversion using @gcc.gnu.org addresses is now available:

git clone git+ssh://gcc.gnu.org/home/gccadmin/wwwdocs-test2.git

My current versions of the conversion scripts / proposed hook are
attached.  The rest of the description of the first test conversion still
applies.

--
Joseph S. Myers
[hidden email]

do-convert (2K) Download Attachment
do-download (372 bytes) Download Attachment
post-receive (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Segher Boessenkool
In reply to this post by Jonathan Wakely-4
On Wed, Sep 25, 2019 at 12:02:42PM +0100, Jonathan Wakely wrote:

> I mean that there's not much value in having my past commits listed as
> coming from various "different" authors:
>
> Jonathan Wakely <xxx at compsoc.man.ac.uk>
> Jonathan Wakely <xxx at kayari.org>
> Jonathan Wakely <xxx at gmail.com>
> Jonathan Wakely <xxx at gcc.gnu.org>
> Jonathan Wakely <xxx at redhat.com>
>
> All of those "identities" are the same person, so making all my
> commits use the same Author is arguably correct, even if I was using
> different addresses at different times.
>
> Using the @gcc.gnu.org address for all my past commits might be the
> least worst option, because it's "me" and that address was valid in
> 2004 and is still valid now, unlike the other options.

And using a different address is *incorrect*, and it is quite nasty if
you would be associating a commit with the wrong employer.  @gcc.gnu.org
is fine of course.


Segher
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Segher Boessenkool
In reply to this post by Joseph Myers-2
On Wed, Sep 25, 2019 at 04:49:58PM +0000, Joseph Myers wrote:
> A version of the conversion using @gcc.gnu.org addresses is now available:
>
> git clone git+ssh://gcc.gnu.org/home/gccadmin/wwwdocs-test2.git

This looks great, thanks!

Can we test committing to this repo as well?  Are hooks set up already?
Am I too impatient?


Segher
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Joseph Myers
On Wed, 25 Sep 2019, Segher Boessenkool wrote:

> On Wed, Sep 25, 2019 at 04:49:58PM +0000, Joseph Myers wrote:
> > A version of the conversion using @gcc.gnu.org addresses is now available:
> >
> > git clone git+ssh://gcc.gnu.org/home/gccadmin/wwwdocs-test2.git
>
> This looks great, thanks!
>
> Can we test committing to this repo as well?  Are hooks set up already?

The hooks aren't set up.  Setting up the hooks will involve moving various
directories around in /www/gcc on sourceware to replace CVS checkouts with
symlinks into a git checkout, i.e. it's part of the actual move from CVS
to git, since the main thing the hooks do is run the preprocessor to
update the live website.

--
Joseph S. Myers
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Joseph Myers
In reply to this post by Joseph Myers
As far as I can tell from the discussion, people are happy with the
version of the conversion using @gcc.gnu.org addresses.  When shall we (do
a final conversion if there are any commits postdating that one and)
switch over to it as the live repository, set up hooks and update
documentation of editing the website?

--
Joseph S. Myers
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: git conversion of GCC wwwdocs repository

Thomas Schwinge-8
Hi Joseph!

On 2019-09-30T19:56:39+0000, Joseph Myers <[hidden email]> wrote:
> As far as I can tell from the discussion, people are happy with the
> version of the conversion using @gcc.gnu.org addresses.  When shall we (do
> a final conversion if there are any commits postdating that one and)
> switch over to it as the live repository, set up hooks and update
> documentation of editing the website?

Given that nobody has raised any adverse opinions: can do that anytime
now?

Doing 'wwwdocs' before the sources repository conversion, we can use it
as a testbed for hooks setup: commit/push hook obviously, but also
Bugzilla integration, and fast-forward-only requirement for the 'master'
branch, etc.


Grüße
 Thomas

signature.asc (671 bytes) Download Attachment