private components in module files

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

private components in module files

Andrew Benson-3
Hi all,

I've been slowly trying to educate myself about the structure of gfortran
module files, in the hope that I can eventually work on making some
improvements (I'm particularly interested in anything that would reduce the
file size or speed up parsing).

In module.c there's the following code:

   /* Note that components are always saved, even if they are supposed
     to be private.  Component access is checked during searching.  */
  mio_component_list (&sym->components, sym->attr.vtype);
  if (sym->components != NULL)
    sym->component_access
      = MIO_NAME (gfc_access) (sym->component_access, access_types);

Can anyone on this list help me understand why even private components are
saved to the module file? After thinking about this for some time my best guess
is that they might be needed by submodules which read this module file. Is that
correct?

The reason that I want to understand this arises from the following example:

module mod1
  private
  public :: type1
  type :: type1
     private
     integer :: component1
  end type type1
end module mod1

module mod2
  use mod1
  private
  type(type1) :: x
end module mod2

In this "component1" is a private component of "type1". When I compile the
above I find that "component1" is written to mod1.mod, e.g.:

$ gunzip -c mod1.mod |grep component1                                                                                                                                                  
UNKNOWN 0 0 PRIVATE_COMP) ((3 'component1' (INTEGER 4 0 0 0 INTEGER ())

as expected based on the comment from module.c. But I also find that
"component1" appears in mod2.mod, e.g.:

$ gunzip -c mod2.mod |grep component1                                                                                                                                                  
UNKNOWN 0 0 PRIVATE_COMP) ((14 'component1' (INTEGER 4 0 0 0 INTEGER ())

and I haven't been able to understand why there is a need for it to also
appear in "mod2".

In a project with many levels deep of module uses and derived types with large
numbers of components the resulting modules can grow to be very large, so I'm
trying to understand if having these private components from used modules be
present in the file is necessary.

Thanks,
Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Steve Kargl
On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:

>
> I've been slowly trying to educate myself about the structure of gfortran
> module files, in the hope that I can eventually work on making some
> improvements (I'm particularly interested in anything that would reduce the
> file size or speed up parsing).
>
> In module.c there's the following code:
>
>    /* Note that components are always saved, even if they are supposed
>      to be private.  Component access is checked during searching.  */
>   mio_component_list (&sym->components, sym->attr.vtype);
>   if (sym->components != NULL)
>     sym->component_access
>       = MIO_NAME (gfc_access) (sym->component_access, access_types);
>
> Can anyone on this list help me understand why even private components are
> saved to the module file? After thinking about this for some time my best guess
> is that they might be needed by submodules which read this module file. Is that
> correct?
>
> The reason that I want to understand this arises from the following example:
>
> module mod1
>   private
>   public :: type1
>   type :: type1
>      private
>      integer :: component1
>   end type type1
> end module mod1
>
> module mod2
>   use mod1
>   private
>   type(type1) :: x
> end module mod2
>
> In this "component1" is a private component of "type1". When I compile the
> above I find that "component1" is written to mod1.mod, e.g.:
>
> $ gunzip -c mod1.mod |grep component1                                                                                                                                                  
> UNKNOWN 0 0 PRIVATE_COMP) ((3 'component1' (INTEGER 4 0 0 0 INTEGER ())
>
> as expected based on the comment from module.c. But I also find that
> "component1" appears in mod2.mod, e.g.:
>
> $ gunzip -c mod2.mod |grep component1                                                                                                                                                  
> UNKNOWN 0 0 PRIVATE_COMP) ((14 'component1' (INTEGER 4 0 0 0 INTEGER ())
>
> and I haven't been able to understand why there is a need for it to also
> appear in "mod2".
>
> In a project with many levels deep of module uses and derived types with large
> numbers of components the resulting modules can grow to be very large, so I'm
> trying to understand if having these private components from used modules be
> present in the file is necessary.
>

It appears the code you quoted came from revision 206759.

------------------------------------------------------------------------
r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19 lines

AFAICT, submodule appeared after this commit.  I haven't seen a
post from Mikael in sometime, so don't know if can clear up the
need for this code.

--
Steve
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Tobias Burnus-3
In reply to this post by Andrew Benson-3
Hi Andrew,

On 10/23/19 6:07 PM, Andrew Benson wrote:
> In module.c there's the following code:
>
>     /* Note that components are always saved, even if they are supposed
>       to be private.  Component access is checked during searching.  */
>    mio_component_list (&sym->components, sym->attr.vtype);
>    if (sym->components != NULL)
>      sym->component_access
>        = MIO_NAME (gfc_access) (sym->component_access, access_types);

I am not sure whether it is needed to be in other .mod files but the
information might be needed. Assume:

module m
   private
   public :: t2
   type t
     integer, allocatable :: i
   end type t
   type t2
     private
     integer, allocatable :: j
     type(t), allocatable :: k
   end type t2
end module m

module m2
   use m, only: t2
   type t3
     type(t2), allocatable :: x
   end type t3
end module m2

subroutine foo(x)
   use m2, only: t3
   type(t3), allocatable, intent(out) :: x
end subroutine foo

The "intent(out)" means that one needs to deallocatable "x" in "foo".
That implies that all allocatable components of "x" are deallocated.
Hence, "foo" needs to know about "type(t)" and the components "j" and "k"
of type(t2) – even though they are private.

However, it can be argued whether that information has to be in m2.mod or only in m.mod.

Regarding submodules, I don't have any idea. I just observe that using submodules,
one may end up having two identical files – name.mod and name.smod, which also
looks like a waste of space. But I don't know anything about the implementation.

Cheers,

Tobias

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
In reply to this post by Steve Kargl
On Wednesday, October 23, 2019 9:23:15 AM PDT Steve Kargl wrote:

> On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:
> > I've been slowly trying to educate myself about the structure of gfortran
> > module files, in the hope that I can eventually work on making some
> > improvements (I'm particularly interested in anything that would reduce
> > the
> > file size or speed up parsing).
> >
> > In module.c there's the following code:
> >    /* Note that components are always saved, even if they are supposed
> >    
> >      to be private.  Component access is checked during searching.  */
> >  
> >   mio_component_list (&sym->components, sym->attr.vtype);
> >   if (sym->components != NULL)
> >  
> >     sym->component_access
> >    
> >       = MIO_NAME (gfc_access) (sym->component_access, access_types);
> >
> > Can anyone on this list help me understand why even private components are
> > saved to the module file? After thinking about this for some time my best
> > guess is that they might be needed by submodules which read this module
> > file. Is that correct?
> >
> > The reason that I want to understand this arises from the following
> > example:
> >
> > module mod1
> >
> >   private
> >   public :: type1
> >   type :: type1
> >  
> >      private
> >      integer :: component1
> >  
> >   end type type1
> >
> > end module mod1
> >
> > module mod2
> >
> >   use mod1
> >   private
> >   type(type1) :: x
> >
> > end module mod2
> >
> > In this "component1" is a private component of "type1". When I compile the
> > above I find that "component1" is written to mod1.mod, e.g.:
> >
> > $ gunzip -c mod1.mod |grep component1
> > UNKNOWN 0 0 PRIVATE_COMP) ((3 'component1' (INTEGER 4 0 0 0 INTEGER ())
> >
> > as expected based on the comment from module.c. But I also find that
> > "component1" appears in mod2.mod, e.g.:
> >
> > $ gunzip -c mod2.mod |grep component1
> > UNKNOWN 0 0 PRIVATE_COMP) ((14 'component1' (INTEGER 4 0 0 0 INTEGER ())
> >
> > and I haven't been able to understand why there is a need for it to also
> > appear in "mod2".
> >
> > In a project with many levels deep of module uses and derived types with
> > large numbers of components the resulting modules can grow to be very
> > large, so I'm trying to understand if having these private components
> > from used modules be present in the file is necessary.
>
> It appears the code you quoted came from revision 206759.
>
> ------------------------------------------------------------------------
> r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19 lines
>
> AFAICT, submodule appeared after this commit.  I haven't seen a
> post from Mikael in sometime, so don't know if can clear up the
> need for this code.

Thanks - that's a very good point - so it seems that submodules are not the
only reason for having these private components in the module file.

Thanks,
Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
In reply to this post by Tobias Burnus-3
On Wednesday, October 23, 2019 6:25:42 PM PDT Tobias Burnus wrote:

> Hi Andrew,
>
> On 10/23/19 6:07 PM, Andrew Benson wrote:
> > In module.c there's the following code:
> >     /* Note that components are always saved, even if they are supposed
> >    
> >       to be private.  Component access is checked during searching.  */
> >    
> >    mio_component_list (&sym->components, sym->attr.vtype);
> >    if (sym->components != NULL)
> >    
> >      sym->component_access
> >      
> >        = MIO_NAME (gfc_access) (sym->component_access, access_types);
>
> I am not sure whether it is needed to be in other .mod files but the
> information might be needed. Assume:
>
> module m
>    private
>    public :: t2
>    type t
>      integer, allocatable :: i
>    end type t
>    type t2
>      private
>      integer, allocatable :: j
>      type(t), allocatable :: k
>    end type t2
> end module m
>
> module m2
>    use m, only: t2
>    type t3
>      type(t2), allocatable :: x
>    end type t3
> end module m2
>
> subroutine foo(x)
>    use m2, only: t3
>    type(t3), allocatable, intent(out) :: x
> end subroutine foo
>
> The "intent(out)" means that one needs to deallocatable "x" in "foo".
> That implies that all allocatable components of "x" are deallocated.
> Hence, "foo" needs to know about "type(t)" and the components "j" and "k"
> of type(t2) – even though they are private.
>
> However, it can be argued whether that information has to be in m2.mod or
> only in m.mod.
>
> Regarding submodules, I don't have any idea. I just observe that using
> submodules, one may end up having two identical files – name.mod and
> name.smod, which also looks like a waste of space. But I don't know
> anything about the implementation.
>
> Cheers,
>
> Tobias

Thanks for the example - that makes it more clear to me why the symbols have
to be present in m2 also.

So it seems that there isn't a simple way to avoid this.

Cheers,
Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Steve Kargl
In reply to this post by Tobias Burnus-3
On Wed, Oct 23, 2019 at 06:25:42PM +0200, Tobias Burnus wrote:
>
> Regarding submodules, I don't have any idea. I just observe
> that using submodules, one may end up having two identical
> files – name.mod and name.smod, which also looks like a waste
> of space. But I don't know anything about the implementation.
>

A major point of submodules is to avoid compilation cascades.
Consider a project to numerous source file that USE a params.
module.  Prior to submodules, a change to the params module
would require a recompilation of those numerous files.  With
submodules, you compile/create the *.mod once with interface
information.  The *.smod file contains the implementation
detailsi, and can be recompiled multiple time without causing
the cascade.  

--
Steve
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
In reply to this post by Steve Kargl
On Wednesday, October 23, 2019 9:23:15 AM PDT Steve Kargl wrote:

> On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:
> > I've been slowly trying to educate myself about the structure of gfortran
> > module files, in the hope that I can eventually work on making some
> > improvements (I'm particularly interested in anything that would reduce
> > the
> > file size or speed up parsing).
> >
> > In module.c there's the following code:
> >    /* Note that components are always saved, even if they are supposed
> >    
> >      to be private.  Component access is checked during searching.  */
> >  
> >   mio_component_list (&sym->components, sym->attr.vtype);
> >   if (sym->components != NULL)
> >  
> >     sym->component_access
> >    
> >       = MIO_NAME (gfc_access) (sym->component_access, access_types);
> >
> > Can anyone on this list help me understand why even private components are
> > saved to the module file? After thinking about this for some time my best
> > guess is that they might be needed by submodules which read this module
> > file. Is that correct?
> >
> > The reason that I want to understand this arises from the following
> > example:
> >
> > module mod1
> >
> >   private
> >   public :: type1
> >   type :: type1
> >  
> >      private
> >      integer :: component1
> >  
> >   end type type1
> >
> > end module mod1
> >
> > module mod2
> >
> >   use mod1
> >   private
> >   type(type1) :: x
> >
> > end module mod2
> >
> > In this "component1" is a private component of "type1". When I compile the
> > above I find that "component1" is written to mod1.mod, e.g.:
> >
> > $ gunzip -c mod1.mod |grep component1
> > UNKNOWN 0 0 PRIVATE_COMP) ((3 'component1' (INTEGER 4 0 0 0 INTEGER ())
> >
> > as expected based on the comment from module.c. But I also find that
> > "component1" appears in mod2.mod, e.g.:
> >
> > $ gunzip -c mod2.mod |grep component1
> > UNKNOWN 0 0 PRIVATE_COMP) ((14 'component1' (INTEGER 4 0 0 0 INTEGER ())
> >
> > and I haven't been able to understand why there is a need for it to also
> > appear in "mod2".
> >
> > In a project with many levels deep of module uses and derived types with
> > large numbers of components the resulting modules can grow to be very
> > large, so I'm trying to understand if having these private components
> > from used modules be present in the file is necessary.
>
> It appears the code you quoted came from revision 206759.
>
> ------------------------------------------------------------------------
> r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19 lines
>
> AFAICT, submodule appeared after this commit.  I haven't seen a
> post from Mikael in sometime, so don't know if can clear up the
> need for this code.

Looking back through svn history I think this code is from even earlier. Way
back in revision 81764 I see:

/* Note that components are always saved, even if they are supposed
      to be private.  Component access is checked during searching.  */
 
   mio_component_list (&sym->components);


in module.c (lines 2683-2686). This revision log message was "Merge tree-
ssa-20020619-branch into mainline.".

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Steve Kargl
On Wed, Oct 23, 2019 at 09:56:54AM -0700, Andrew Benson wrote:

> On Wednesday, October 23, 2019 9:23:15 AM PDT Steve Kargl wrote:
> > On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:
> > > I've been slowly trying to educate myself about the structure of gfortran
> > > module files, in the hope that I can eventually work on making some
> > > improvements (I'm particularly interested in anything that would reduce
> > > the
> > > file size or speed up parsing).
> > >
> > > In module.c there's the following code:
> > >    /* Note that components are always saved, even if they are supposed
> > >    
> > >      to be private.  Component access is checked during searching.  */
> > >  
> > >   mio_component_list (&sym->components, sym->attr.vtype);
> > >   if (sym->components != NULL)
> > >  
> > >     sym->component_access
> > >    
> > >       = MIO_NAME (gfc_access) (sym->component_access, access_types);
> > >
> > > Can anyone on this list help me understand why even private components are
> > > saved to the module file? After thinking about this for some time my best
> > > guess is that they might be needed by submodules which read this module
> > > file. Is that correct?
> > >

(a bunch removed to keep the emai short)

> > It appears the code you quoted came from revision 206759.
> >
> > ------------------------------------------------------------------------
> > r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19 lines
> >
> > AFAICT, submodule appeared after this commit.  I haven't seen a
> > post from Mikael in sometime, so don't know if can clear up the
> > need for this code.
>
> Looking back through svn history I think this code is from even earlier. Way
> back in revision 81764 I see:
>
> /* Note that components are always saved, even if they are supposed
>       to be private.  Component access is checked during searching.  */
>  
>    mio_component_list (&sym->components);
>
>
> in module.c (lines 2683-2686). This revision log message was "Merge tree-
> ssa-20020619-branch into mainline.".
>

In that case, you would need to ask Steven Bosscher and Paul Brook.
Neither have contributed to gfortran in years if not a decade.

Tobias's explanation seems to be a logical reason.  There is always
the option of removing the code, building gfortran, and running
the testsuite.  Things will/should break if the info is required
for the module.

--
Steve
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
On Wednesday, October 23, 2019 10:23:10 AM PDT Steve Kargl wrote:

> On Wed, Oct 23, 2019 at 09:56:54AM -0700, Andrew Benson wrote:
> > On Wednesday, October 23, 2019 9:23:15 AM PDT Steve Kargl wrote:
> > > On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:
> > > > I've been slowly trying to educate myself about the structure of
> > > > gfortran
> > > > module files, in the hope that I can eventually work on making some
> > > > improvements (I'm particularly interested in anything that would
> > > > reduce
> > > > the
> > > > file size or speed up parsing).
> > > >
> > > > In module.c there's the following code:
> > > >    /* Note that components are always saved, even if they are supposed
> > > >    
> > > >      to be private.  Component access is checked during searching.  */
> > > >  
> > > >   mio_component_list (&sym->components, sym->attr.vtype);
> > > >   if (sym->components != NULL)
> > > >  
> > > >     sym->component_access
> > > >    
> > > >       = MIO_NAME (gfc_access) (sym->component_access, access_types);
> > > >
> > > > Can anyone on this list help me understand why even private components
> > > > are
> > > > saved to the module file? After thinking about this for some time my
> > > > best
> > > > guess is that they might be needed by submodules which read this
> > > > module
> > > > file. Is that correct?
>
> (a bunch removed to keep the emai short)
>
> > > It appears the code you quoted came from revision 206759.
> > >
> > > ------------------------------------------------------------------------
> > > r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19
> > > lines
> > >
> > > AFAICT, submodule appeared after this commit.  I haven't seen a
> > > post from Mikael in sometime, so don't know if can clear up the
> > > need for this code.
> >
> > Looking back through svn history I think this code is from even earlier.
> > Way back in revision 81764 I see:
> >
> > /* Note that components are always saved, even if they are supposed
> >
> >       to be private.  Component access is checked during searching.  */
> >    
> >    mio_component_list (&sym->components);
> >
> > in module.c (lines 2683-2686). This revision log message was "Merge tree-
> > ssa-20020619-branch into mainline.".
>
> In that case, you would need to ask Steven Bosscher and Paul Brook.
> Neither have contributed to gfortran in years if not a decade.
>
> Tobias's explanation seems to be a logical reason.  There is always
> the option of removing the code, building gfortran, and running
> the testsuite.  Things will/should break if the info is required
> for the module.

Agreed - I'll find some time to try this next week and see if anything breaks.

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Tobias Burnus-3
In reply to this post by Steve Kargl
Hi Steve,

On 10/23/19 6:49 PM, Steve Kargl wrote:

>> name.mod and name.smod, which also looks like a waste
>> of space. But I don't know anything about the implementation.
>>
> A major point of submodules is to avoid compilation cascades.
> Consider a project to numerous source file that USE a params.
> module.  Prior to submodules, a change to the params module
> would require a recompilation of those numerous files.  With
> submodules, you compile/create the *.mod once with interface
> information.  The *.smod file contains the implementation
> detailsi, and can be recompiled multiple time without causing
> the cascade.

Still, it is not clear to me why for the out-most module, both a .mod
and a .smod is needed – if the content is completely identical (as it
can happen for some examples at least). – I would understand it if the
content was always different or if only a .mod would be present for that
one – and only .smod for the rest.

Cheers,

Tobias

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Steve Kargl
On Wed, Oct 23, 2019 at 07:26:46PM +0200, Tobias Burnus wrote:
> Hi Steve,
>
> On 10/23/19 6:49 PM, Steve Kargl wrote:
> >> name.mod and name.smod, which also looks like a waste
> >> of space. But I don't know anything about the implementation.
> >>

(correction to poor English inserted into my original reply :)

> > A major point of submodules is to avoid compilation cascades.
> > Consider a project with numerous source files that USE a params
> > module.  Prior to submodules, a change to the params module
> > would require a recompilation of those numerous files.  With
> > submodules, you compile/create the *.mod once with interface
> > information.  The *.smod file contains the implementation
> > details, and can be recompiled multiple time without causing
> > the cascade.
>
> Still, it is not clear to me why for the out-most module, both a .mod
> and a .smod is needed – if the content is completely identical (as it
> can happen for some examples at least). – I would understand it if the
> content was always different or if only a .mod would be present for that
> one – and only .smod for the rest.
>

Probably, need to poke pault about implementation details.  I
suspect it came down to a matter of available time to work on
the submodule feature and what worked at the time.  I suspect
pault would not object to anyone refining the implementation.

--
Steve
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
In reply to this post by Steve Kargl
On Wednesday, October 23, 2019 10:23:10 AM PDT Steve Kargl wrote:

> On Wed, Oct 23, 2019 at 09:56:54AM -0700, Andrew Benson wrote:
> > On Wednesday, October 23, 2019 9:23:15 AM PDT Steve Kargl wrote:
> > > On Wed, Oct 23, 2019 at 09:07:03AM -0700, Andrew Benson wrote:
> > > > I've been slowly trying to educate myself about the structure of
> > > > gfortran
> > > > module files, in the hope that I can eventually work on making some
> > > > improvements (I'm particularly interested in anything that would
> > > > reduce
> > > > the
> > > > file size or speed up parsing).
> > > >
> > > > In module.c there's the following code:
> > > >    /* Note that components are always saved, even if they are supposed
> > > >    
> > > >      to be private.  Component access is checked during searching.  */
> > > >  
> > > >   mio_component_list (&sym->components, sym->attr.vtype);
> > > >   if (sym->components != NULL)
> > > >  
> > > >     sym->component_access
> > > >    
> > > >       = MIO_NAME (gfc_access) (sym->component_access, access_types);
> > > >
> > > > Can anyone on this list help me understand why even private components
> > > > are
> > > > saved to the module file? After thinking about this for some time my
> > > > best
> > > > guess is that they might be needed by submodules which read this
> > > > module
> > > > file. Is that correct?
>
> (a bunch removed to keep the emai short)
>
> > > It appears the code you quoted came from revision 206759.
> > >
> > > ------------------------------------------------------------------------
> > > r206759 | mikael | 2014-01-18 12:05:25 -0800 (Sat, 18 Jan 2014) | 19
> > > lines
> > >
> > > AFAICT, submodule appeared after this commit.  I haven't seen a
> > > post from Mikael in sometime, so don't know if can clear up the
> > > need for this code.
> >
> > Looking back through svn history I think this code is from even earlier.
> > Way back in revision 81764 I see:
> >
> > /* Note that components are always saved, even if they are supposed
> >
> >       to be private.  Component access is checked during searching.  */
> >    
> >    mio_component_list (&sym->components);
> >
> > in module.c (lines 2683-2686). This revision log message was "Merge tree-
> > ssa-20020619-branch into mainline.".
>
> In that case, you would need to ask Steven Bosscher and Paul Brook.
> Neither have contributed to gfortran in years if not a decade.
>
> Tobias's explanation seems to be a logical reason.  There is always
> the option of removing the code, building gfortran, and running
> the testsuite.  Things will/should break if the info is required
> for the module.

Making a simple modification to prevent output of components that should be
private definitely breaks things. But, I think the mio_component_list()
function is also outputting type-bound procedures etc. so I may need to
examine more carefully which components can be excluded from the output.

-Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Salvatore Filippone-3
In reply to this post by Andrew Benson-3
On Wed, Oct 23, 2019 at 09:49:38AM -0700, Steve Kargl wrote:
>
>A major point of submodules is to avoid compilation cascades.
>Consider a project to numerous source file that USE a params.
>module.  Prior to submodules, a change to the params module
>would require a recompilation of those numerous files.  With
>submodules, you compile/create the *.mod once with interface
i>nformation.  The *.smod file contains the implementation
>detailsi, and can be recompiled multiple time without causing
>the cascade.

Along the same lines, to avoid unnecessary recompilations you may want
to split your interface from implementation, and keep only the
interface in the MODULE. However, if you have PRIVATE variables or
type components that need be accessed by the implementation code, the
only way is through a SUBMODULE; if you were to put the implementation
in an external subroutine (as I am forced to do until I bump the
minimum GCC version requirement for my codes to 6) you must declare
components/types/variables as public, or you are forced to put the
implementation in the module itself (hence lots of recompilations).

Salvatore
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Janne Blomqvist-3
In reply to this post by Andrew Benson-3
On Wed, Oct 23, 2019 at 7:07 PM Andrew Benson
<[hidden email]> wrote:
>
> Hi all,
>
> I've been slowly trying to educate myself about the structure of gfortran
> module files, in the hope that I can eventually work on making some
> improvements (I'm particularly interested in anything that would reduce the
> file size or speed up parsing).

AFAICS the big improvements, thought absolutely not low-hanging fruit
(which is why it hasn't been done), is

1) Not including transitive dependencies in .mod files. Implying that
.mod files would need to have the equivalent of USE statements, so
that the module parser knows it might need to parse other modules to
resolve all symbols.

2) Caching parsed module files. This is likely necessary to get (1) to
work decently.

AFAIK the NAG Fortran compiler does something like this.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40958#c13

Also see previous discussion in
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


--
Janne Blomqvist
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
On Thursday, October 24, 2019 11:26:27 AM PDT Janne Blomqvist wrote:

> On Wed, Oct 23, 2019 at 7:07 PM Andrew Benson
>
> <[hidden email]> wrote:
> > Hi all,
> >
> > I've been slowly trying to educate myself about the structure of gfortran
> > module files, in the hope that I can eventually work on making some
> > improvements (I'm particularly interested in anything that would reduce
> > the
> > file size or speed up parsing).
>
> AFAICS the big improvements, thought absolutely not low-hanging fruit
> (which is why it hasn't been done), is
>
> 1) Not including transitive dependencies in .mod files. Implying that
> .mod files would need to have the equivalent of USE statements, so
> that the module parser knows it might need to parse other modules to
> resolve all symbols.
>
> 2) Caching parsed module files. This is likely necessary to get (1) to
> work decently.
>
> AFAIK the NAG Fortran compiler does something like this.
>
> See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40958#c13
>
> Also see previous discussion in
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708


Thanks for the links to the previous discussion on this. For my own usage of
gfortan it's 2) that would probably be the most useful improvement - the size
of the .mod files in itself isn't a problem, but the long compile times which
arise from (re-)reading them is. My guess is that implementing caching for
parsed module files is well beyond my current abilities and understanding of
gfortran, but I'll dig around the code anyway - maybe I'll at least get a
better understanding of how the module parsing works.

-Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
In reply to this post by Janne Blomqvist-3
On Thursday, October 24, 2019 11:26:27 AM PDT Janne Blomqvist wrote:

> On Wed, Oct 23, 2019 at 7:07 PM Andrew Benson
>
> <[hidden email]> wrote:
> > Hi all,
> >
> > I've been slowly trying to educate myself about the structure of gfortran
> > module files, in the hope that I can eventually work on making some
> > improvements (I'm particularly interested in anything that would reduce
> > the
> > file size or speed up parsing).
>
> AFAICS the big improvements, thought absolutely not low-hanging fruit
> (which is why it hasn't been done), is
>
> 1) Not including transitive dependencies in .mod files. Implying that
> .mod files would need to have the equivalent of USE statements, so
> that the module parser knows it might need to parse other modules to
> resolve all symbols.
>
> 2) Caching parsed module files. This is likely necessary to get (1) to
> work decently.
>
> AFAIK the NAG Fortran compiler does something like this.
>
> See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=40958#c13
>
> Also see previous discussion in
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708

I spent an hour or two today working to understand how caching of parsed
module files might be implemented. Following the ideas in:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25708#c5

I was able to implement a linked list of namespaces for modules, and then add
symbols to a symtree within that namespace as the module is parsed and, if the
same module is parsed again, return that cached symbol. (This is a very
crude implementation so far - I'm mostly just trying things to improve my
understanding - I expect I'll end up re-writing this multiple times as I learn
from my mistakes....)

This seems to work in some simple cases, but I have a few questions:

I'm considering a simple case such as this:

module a

  type aa
     integer :: ab
  end type aa
 
end module a

module b

contains

  subroutine b1()
    use a
    type(aa) :: ba
  end subroutine b1

  subroutine b2()
    use a
    type(aa) :: ba
  end subroutine b2

end module b

program test
  use b
  call b1()
end program test


I had initially attempted to return a pointer to the symbol for type(aa) in
the module namespace on the second use of the module. If I do this though then
the second "type(aa) :: ba" triggers an error:

tmp.F90:20:12:

   20 |     type(aa) :: ba
      |            1
Error: Cannot change attributes of USE-associated symbol aa at (1)

This occurs in gfc_match_decl_type_spec() in decl.c at the line:

  if (dt_sym->attr.flavor != FL_DERIVED && dt_sym->attr.flavor != FL_STRUCT
      && !gfc_add_flavor (&dt_sym->attr, FL_DERIVED, sym->name, NULL))
    return MATCH_ERROR;

Therefore I've instead opted to return a copy of the symbol for type(aa) from
the module namespace, which avoids this problem. It's not clear to me if this
is the best solution though, or if there is a way to return just a pointer to
the symbol and avoid this "change attributes" error?


A second question: To add each symbol to the module namespace I need to give
it a unique name. I've tried to construct such a name by concatenating the
"proc_name" of the symbol's namespace with the "true_name" associated with the
symbol. This works in most cases, but fails for symbols such as the "src" and
"dst" symbols that get created - there can be many of these and they all share
the same "proc_name" (the module itself). So, I'm not sure on how best to
generate a unique name. (Each of these "src" and "dst" symbols belong to a
unique namespace within the load_needed() function - I could similarly put
them in unique namespaces within the module namespace. But then I'd need some
way to assign a unique name to each of those namespaces, so this wouldn't
resolve the problem.)

I can post the patch for my modifications if it's useful (although I should
probably clean it up first to make it more comprehensible).

Thanks,
Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus

Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Thomas Koenig-6
Hi Andrew,

first, thanks for taking this on!

> I had initially attempted to return a pointer to the symbol for type(aa) in
> the module namespace on the second use of the module. If I do this though then
> the second "type(aa) :: ba" triggers an error:
>
> tmp.F90:20:12:
>
>     20 |     type(aa) :: ba
>        |            1
> Error: Cannot change attributes of USE-associated symbol aa at (1)
>
> This occurs in gfc_match_decl_type_spec() in decl.c at the line:
>
>    if (dt_sym->attr.flavor != FL_DERIVED && dt_sym->attr.flavor != FL_STRUCT
>        && !gfc_add_flavor (&dt_sym->attr, FL_DERIVED, sym->name, NULL))
>      return MATCH_ERROR;
>
> Therefore I've instead opted to return a copy of the symbol for type(aa) from
> the module namespace, which avoids this problem.

I think this approach can be problematic because we use symbol addresses
to check if we have the same symbol, for example in dependency checking.

> It's not clear to me if this
> is the best solution though, or if there is a way to return just a pointer to
> the symbol and avoid this "change attributes" error?

What I usually do in situations like these is to add a flag to the
gfc_symbol structure, set it when needed and then make an error
condition (or something else) contigent on that flag.

The most recent example of that was the maybe_array flag for the
interface checking; just grep for that in the sources to see
how it is used.

Hope this helps!

Regards

        Thomas
Reply | Threaded
Open this post in threaded view
|

Re: private components in module files

Andrew Benson-3
Hi Thomas,

> What I usually do in situations like these is to add a flag to the
> gfc_symbol structure, set it when needed and then make an error
> condition (or something else) contigent on that flag.
>
> The most recent example of that was the maybe_array flag for the
> interface checking; just grep for that in the sources to see
> how it is used.

Thanks - I'll take a look at the maybe_array code and see if I can implement a
similar solution.

Thanks,
Andrew

--

* Andrew Benson: http://users.obs.carnegiescience.edu/abenson/contact.html

* Galacticus: https://github.com/galacticusorg/galacticus