Namelist IOSTAT behaviour

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Namelist IOSTAT behaviour

Neilen Marais-2
Hi!

What value of IOSTAT should a namelist return in the case that the
namelist is not present whitin the file being read? If I use:

    READ(unit=infileunit, nml=debugging,  iostat=ios)

and the file does not contain a debugging namelist at all, ios is
negative when using intel fortran, but positive when using gfortran.

Is the behaviour defined in the fortran strandard, or can I not make an
assumption in this regard?

Why I'm discerning this in the first place is that intel fortran will
make iostat positive if a namelist contains an unknown variable, and
negative if the namelist is not in the file. This allows me to provide
some diagnostic if a namelist with an unknown variable is used, while
allowing the code to continue if a namelist is simply omitted. If this
behaviour is not supposed to be portable, I'll make a different plan
though.

Thanks
Neilen

Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Steve Kargl
On Thu, Jun 09, 2005 at 02:32:16PM +0200, Neilen Marais wrote:

> Hi!
>
> What value of IOSTAT should a namelist return in the case that the
> namelist is not present whitin the file being read? If I use:
>
>     READ(unit=infileunit, nml=debugging,  iostat=ios)
>
> and the file does not contain a debugging namelist at all, ios is
> negative when using intel fortran, but positive when using gfortran.
>
> Is the behaviour defined in the fortran strandard, or can I not make an
> assumption in this regard?
>
 
In reading 9.4.1.4, I infer that you have found a bug in gfortran.
Please submit a bug report through bugzilla.

--
Steve
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
Neilen,

That is indeed the bug; namelists in modules are partially broken. Code is
generated for the IO operation but it lacks the transfers of the addresses
of the namelist objects. At a guess, I would say that the namelist is not
being found by trans-io. The namelist objects are in scope, though. A "three
piper", as Sherlock Holmes would say.

Please submit a bug report and I will endeavour to fix it as soon as I can.

Thanks

Paul T
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Neilen Marais-2
Hi Paul

On Fri, 2005-06-10 at 17:33 +0100, Paul Thomas wrote:

> Please submit a bug report and I will endeavour to fix it as soon as I can.

Thanks so much. Bug.reported:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=22010

Cheers
Neilen

>
> Thanks
>
> Paul T

Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
> Please submit a bug report and I will endeavour to fix
> it as soon as I can.

Hah! Easier said than done. The namelist is being correctly recorded in the
.mod file and the objects in it are correctly associated.  However, the
namelist structure is not being regenerated by the use statement.

I have found, easily enough, where the use statement is resolved but cannot
see what happens after that. Where do the module variables get added to the
symbol table and where best should namelists be reconstituted?

Paul T
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Steve Kargl
On Sat, Jun 11, 2005 at 05:02:50PM +0100, Paul Thomas wrote:

> > Please submit a bug report and I will endeavour to fix
> > it as soon as I can.
>
> Hah! Easier said than done. The namelist is being correctly recorded in the
> .mod file and the objects in it are correctly associated.  However, the
> namelist structure is not being regenerated by the use statement.
>
> I have found, easily enough, where the use statement is resolved but cannot
> see what happens after that. Where do the module variables get added to the
> symbol table and where best should namelists be reconstituted?
>

You have stumbled into one of the darkest corners of gfortran.
Our implementation of modules is almost completely undocumented
and the code leaves much to desire.  I spent a week or so
looking at a problem with "USE, only : a => z, b => z" where
the "b => z" causes an ICE because the "a => z" removes z from
the symbols in the module.  I also tried to understand the
problem with EQUIVALENCE in modules.  I gave up. ;-(

--
Steve
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
> You have stumbled into one of the darkest corners of gfortran.

Oh....

> the symbols in the module.  I also tried to understand the
> problem with EQUIVALENCE in modules.  I gave up. ;-(

That's encouraging!

At least you can help me with one thing; where does the compiler attempt to
see if a module declaration contains required symbols?
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Steve Kargl
On Sat, Jun 11, 2005 at 06:14:53PM +0100, Paul Thomas wrote:

> > You have stumbled into one of the darkest corners of gfortran.
>
> Oh....
>
> > the symbols in the module.  I also tried to understand the
> > problem with EQUIVALENCE in modules.  I gave up. ;-(
>
>
> At least you can help me with one thing; where does the compiler attempt to
> see if a module declaration contains required symbols?

I only attempted to understand the "USE, ONLY" construct.
There is a struct gfc_use_rename that is used in gfc_match_use()
to build a list of symbols as they are read.   This is the
root of the problem with "USE, only : a => z, b => z".  By the
time "b => z" is parsed, z has already been renamed to a and
so it no longer exists.  I think you want to be around line 504
in module.c.

--
Steve
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
> I only attempted to understand the "USE, ONLY" construct.
> There is a struct gfc_use_rename that is used in gfc_match_use()
> to build a list of symbols as they are read.   This is the
> root of the problem with "USE, only : a => z, b => z".  By the
> time "b => z" is parsed, z has already been renamed to a and
> so it no longer exists.  I think you want to be around line 504
> in module.c.

OK. I was looking around there but obviously did not understand what it is
doing.  I'll have to build the namelist for build_dt, won't I?

Paul T
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Tobias Schlüter
Paul Thomas wrote:

>>I only attempted to understand the "USE, ONLY" construct.
>>There is a struct gfc_use_rename that is used in gfc_match_use()
>>to build a list of symbols as they are read.   This is the
>>root of the problem with "USE, only : a => z, b => z".  By the
>>time "b => z" is parsed, z has already been renamed to a and
>>so it no longer exists.  I think you want to be around line 504
>>in module.c.
>
>
> OK. I was looking around there but obviously did not understand what it is
> doing.  I'll have to build the namelist for build_dt, won't I?

I mostly understood the module code about a year ago.  I'll try to understand
what needs to be done and I'll get back at you tomorrow.

- Tobi
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Tobias Schlüter
In reply to this post by Paul Thomas-4
Paul Thomas wrote:

>>I only attempted to understand the "USE, ONLY" construct.
>>There is a struct gfc_use_rename that is used in gfc_match_use()
>>to build a list of symbols as they are read.   This is the
>>root of the problem with "USE, only : a => z, b => z".  By the
>>time "b => z" is parsed, z has already been renamed to a and
>>so it no longer exists.  I think you want to be around line 504
>>in module.c.
>
>
> OK. I was looking around there but obviously did not understand what it is
> doing.  I'll have to build the namelist for build_dt, won't I?
You have to save the namelist information to the module file alongside the
symbol, which we previously didn't.

The attached patch does this, this is straight from g95.  It works in Neilen's
testcase but I don't know the namelist stuff well enough to write reasonable
testcases.  Paul, can I defer this to you?

WRT how the symbol code works: most of it  is fairly clear once you get a few
basic ideas.  Looking at it again, I have to say that I'm sorry to say that I
don't really understand how symbol renaming works.

The basic ideas are this:
1. the relations between different data structures need tobe saved / restored.
 These relations are encoded in memory via pointers, so these pointers have to
be converted in a reliable fashion.  This is done via the fixup tables, which
basically assign a unique number to every pointer encountered when writing
module files, and which are converted back to pointers upon reading.
2. the low-level I/O routines are symmetric WRT reading and writing, that is,
the mio_* routines contain the code for both reading and writing.  This way it
is easily ensured that reading and writing modules happens consistently
3. the read_ and write_* routines ensure that everything happens in a
reasonable order

- Tobi

2005-06-12  Tobias Schl"uter  <[hidden email]>

        PR fortran/22010
        Port from g95.
        * module.c (mio_namelist): New function.
        (mio_symbol): Call mio_namelist.

Index: module.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/fortran/module.c,v
retrieving revision 1.32
diff -u -p -r1.32 module.c
--- module.c 1 Apr 2005 04:16:52 -0000 1.32
+++ module.c 12 Jun 2005 13:35:36 -0000
@@ -2564,6 +2564,42 @@ mio_expr (gfc_expr ** ep)
 }
 
 
+/* Read and write namelists */
+
+static void
+mio_namelist (gfc_symbol * sym)
+{
+  gfc_namelist *n, *m;
+
+  mio_lparen ();
+
+  if (iomode == IO_OUTPUT)
+    {
+      for (n = sym->namelist; n; n = n->next)
+ mio_symbol_ref (&n->sym);
+    }
+  else
+    {
+
+      m = NULL;
+      while (peek_atom () != ATOM_RPAREN)
+ {
+  n = gfc_get_namelist ();
+  mio_symbol_ref (&n->sym);
+
+  if (sym->namelist == NULL)
+    sym->namelist = n;
+  else
+    m->next = n;
+
+  m = n;
+ }
+    }
+
+  mio_rparen ();
+}
+
+
 /* Save/restore lists of gfc_interface stuctures.  When loading an
    interface, we are really appending to the existing list of
    interfaces.  Checking for duplicate and ambiguous interfaces has to
@@ -2724,6 +2760,7 @@ mio_symbol (gfc_symbol * sym)
     sym->component_access =
       MIO_NAME(gfc_access) (sym->component_access, access_types);
 
+  mio_namelist (sym);
   mio_rparen ();
 }
 
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2

Tobi,

> You have to save the namelist information to the module file
> alongside the symbol, which we previously didn't.

I got that far yesterday and was contemplating the required format.

> The attached patch does this, this is straight from g95.  It
> works in Neilen's testcase but I don't know the namelist stuff
> well enough to write reasonable testcases.  Paul, can I defer
> this to you?

Well that saves me some messing around!  I was going to download the g95
source to see if this had been solved yet. I was intent on retaining the
same format for the .mod file. I'll do a/some test case(s).

Paul
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
Sorry All,

This correspondence unintentionally went off list:

> Paul Thomas wrote:
>>>Thanks!  I already committed the patch on Steve's approval
>>
>>
>> Yes, I know you did.... My concern about our patches interfering with one
>> another was all too real, though!
>
> I hope my patch + the one it was a followup do away with all the sym vs.
> sym->result crap we had to go through before.  With that away there's much
> less interdependencies that can get in eachother's way.
>
>> I applied the namelist patch to module.c. It has at least one bug: If the
>> module and the using scope had a namelist with the same name, an ICE:
>> segmentation fault ensued. sym->namelist_tail was not being set, which I
>> have fixed. On doing this, the namelists are correctly appended to one
>> another.
>>
>> I notice, however, that duplicate symbols are not discarded during the
>> append. This is no big deal, since write.c and list_read.c walk the
>> namelist until they find the first instance of the required symbol.
>
> I'm not sure I understand.  Under which occassions can a symbol be
> multiple times in the same namelist, when we load it from a module?
> Shouldn't the namelist be valid when we read it?

The F2003 standard allows multiple declarations of the same namelist. I do
not have the F95 standard with me.  Whether intentionally or not, gfortran
allows this and so the same namelist can be declared in the module and in
the host. It is not picky about repeat symbols.

>> If the namelist symbol is renamed with, ONLY: myrenamed_nml=>mynml in the
>> using scope, the name from the module is retained and has to be used in
>> the IO. ie. we have
>>
>> read (10, nml=myrenamed_nml)
>>
>> for which the input has to be "&MYNML A=1.0 /". Does anybody have any
>> idea which it should be? The standard doesn't seem to help.  I will
>> check one of the leading commercial brands tonight.
>
> Anyone is only me, because you only mailed me :-)
>
> Note 11.8 answers your question: this is forbidden.  See especially the
> second and third constraint to 5.5.4.

Ah yes; "the namelist-group-name shall not be a name made available by use
association." 5.4 line 8 in F2k

> Back to developing a data structure for representing the Rubik's
> cube efficiently.  A friend of mine has entered into the speedcubing
> craze, and I can't be bothered to learn a solution by heart, so I
> have to compete some other way.

You just have to learn the transformations of the symmetry groups of the
Platonic solids. http://web.usna.navy.mil/~wdj/rubik_nts.htm should get you
started.

Paul T

Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-10
In reply to this post by Neilen Marais-2
Tobi,

Please find enclosed a modified version of the patch for PR22010 and two test cases. The modifications are:

(i) namelist_tail is set, when the module is read, so that concatenation of namelists can occur correctly.
(ii) The renaming of namelists by USE association is now an error. I thought long and hard about this because it does work correctly. The downside is that the namelist name, used in the IO, is that of the namelist in the module, rather than the host name. This, I think, is why the standard does not allow this feature. If you want to turn this into a warning or an error, if the f95 standard is asserted, please do so.

Since the Cygwin cvs build was broken, when I packed for this trip, the enclosed have been bootstrapped and regtested on the 20040515 snapshot. I notice that cutting and pasting in Windows has, once again, lost the tabs. I have attached a file that should have tabs and LF, instead of CR/LF...

Could you, please, bootstrap and regtest on CVS and commit the package? I am not in a position to do so for at least a week.

Thanks

Paul T

2005-06-15 Tobias Schlueter
PR fortran/22010
Port from g95.
* module.c (mio_namelist): New function. Correct to set
namelist_tail and to give error on renaming namelist by use
association.
(mio_symbol): Call mio_namelist.


2005-06-15 Paul Thomas
PR fortran/22010
* gfortran.dg/namelist_use.f90: New.
* gfortran.dg/namelist_use_only.f90: New.
*** module.c.old Mon Jun 13 16:48:16 2005
--- gcc-4.1-20050515/gcc/fortran/module.c Wed Jun 15 17:33:26 2005
*************** mio_expr (gfc_expr ** ep)
*** 2564,2569 ****
--- 2564,2618 ----
}


+ /* Read and write namelists */
+
+ static void
+ mio_namelist (gfc_symbol * sym)
+ {
+ gfc_namelist *n, *m;
+ const char *check_name;
+
+ mio_lparen ();
+
+ if (iomode == IO_OUTPUT)
+ {
+ for (n = sym->namelist; n; n = n->next)
+ mio_symbol_ref (&n->sym);
+ }
+ else
+ {
+ /* This departure from the standard is flagged as an error.
+ It does, in fact, work correctly. TODO: Allow it
+ conditionally? */
+ if (sym->attr.flavor == FL_NAMELIST)
+ {
+ check_name = find_use_name (sym->name);
+ if (check_name && strcmp (check_name, sym->name) != 0)
+ gfc_error("Namelist %s cannot be renamed by USE"
+ " association to %s.",
+ sym->name, check_name);
+ }
+
+ m = NULL;
+ while (peek_atom () != ATOM_RPAREN)
+ {
+ n = gfc_get_namelist ();
+ mio_symbol_ref (&n->sym);
+
+ if (sym->namelist == NULL)
+ sym->namelist = n;
+ else
+ m->next = n;
+
+ m = n;
+ }
+ sym->namelist_tail = m;
+ }
+
+ mio_rparen ();
+ }
+
+
/* Save/restore lists of gfc_interface stuctures. When loading an
interface, we are really appending to the existing list of
interfaces. Checking for duplicate and ambiguous interfaces has to
*************** mio_symbol (gfc_symbol * sym)
*** 2724,2729 ****
--- 2773,2779 ----
sym->component_access =
MIO_NAME(gfc_access) (sym->component_access, access_types);

+ mio_namelist (sym);
mio_rparen ();
}

! { dg-do run }
! This tests the fix for PR22010, where namelists were not being written to
! and read back from modules. It has two namelists: one that is USE
! associated and another that is concatenated by USE and host association.
!
! Contributed by Paul Thomas [hidden email]
!
module global
character*4 :: aa
integer :: ii
real :: rr
namelist /nml1/ aa, ii, rr
namelist /nml2/ aa
end module global
program namelist_use
use global
real :: rrr
namelist /nml2/ ii, rrr ! Concatenate use and host associated variables.
open (10, status="scratch")
write (10,*) "&NML1 aa=lmno ii=1 rr=2.5 /"
write (10,*) "&NML2 aa=pqrs ii=2 rrr=3.5 /"
rewind (10)
read (10,nml=nml1,iostat=i)
if ((i.ne.0).or.(aa.ne."lmno").or.(ii.ne.1).or.(rr.ne.2.5)) call abort ()
read (10,nml=nml2,iostat=i)
if ((i.ne.0).or.(aa.ne."pqrs").or.(ii.ne.2).or.(rrr.ne.3.5)) call abort ()
close (10)
end program namelist_use

! { dg-do run }
! This tests the fix for PR22010, where namelists were not being written to
! and read back from modules. It checks that namelists from modules that are
! selected by an ONLY declaration work correctly, even when the variables in
! the namelist are not host associated. Note that renaming a namelist by USE
! association is not allowed by the standard and this is trapped in module.c.
!
! Contributed by Paul Thomas [hidden email]
!
module global
character*4 :: aa, aaa
integer :: ii, iii
real :: rr, rrr
namelist /nml1/ aa, ii, rr
namelist /nml2/ aaa, iii, rrr
contains
logical function foo()
foo = (aaa.ne."pqrs").or.(iii.ne.2).or.(rrr.ne.3.5)
end function foo
end module global
program namelist_use_only
use global, only : nml1, aa, ii, rr
use global, only : nml2, rrrr=>rrr, foo
open (10, status="scratch")
write (10,*) "&NML1 aa=lmno ii=1 rr=2.5 /"
write (10,*) "&NML2 aaa=pqrs iii=2 rrr=3.5 /"
rewind (10)
read (10,nml=nml1,iostat=i)
if ((i.ne.0).or.(aa.ne."lmno").or.(ii.ne.1).or.(rr.ne.2.5)) call abort ()
read (10,nml=nml2,iostat=i)
if ((i.ne.0).or.(rrrr.ne.3.5).or.foo()) call abort ()
close (10)
end program namelist_use_only

module_patch.txt (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
Steve,

> looking at a problem with "USE, only : a => z, b => z" where
> the "b => z" causes an ICE because the "a => z" removes z from
> the symbols in the module.  

I think that I can fix that but is it legal? It's a bit bizarre to tie two
host symbols to one module variable, isn't it?  Is there an associated PR?

> I also tried to understand the
> problem with EQUIVALENCE in modules.  I gave up. ;-(

This looks as if it is a problem of logic in trans_decl.c, rather than in
module.c.  The error is generated because the equivalenced variable is found
to already have a backend_decl by the time gfc_create_module_variable is
called. I'll have a poke around; the odd thing is that it was a bug that
appeared.

In a similar vein, PR19561 seems to have done the opposite and fixed itself
spontaneously. Is somebody owning up or is this due to Tobi's and my
rummaging around with dereferencing? Either way, I propose to write a test
case so that it is nailed to the floor.

Paul T
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Tobias Schlüter
Paul Thomas wrote:
>>looking at a problem with "USE, only : a => z, b => z" where
>>the "b => z" causes an ICE because the "a => z" removes z from
>>the symbols in the module.  
>
>
> I think that I can fix that but is it legal? It's a bit bizarre to tie two
> host symbols to one module variable, isn't it?  Is there an associated PR?

It's allowed.  Those module renames really do weird things, IIRC one can also
rename generic interfaces.

> In a similar vein, PR19561 seems to have done the opposite and fixed itself
> spontaneously. Is somebody owning up or is this due to Tobi's and my
> rummaging around with dereferencing? Either way, I propose to write a test
> case so that it is nailed to the floor.

The only thing that might have had an effect on this is my bugfix followup to
the f2c patch, also Zdenek did some magic for derived types some time ago, it
might have been his patch.  I'll try to get certainty tomorrow, and extract a
testcase.

- Tobi
Reply | Threaded
Open this post in threaded view
|

RE: Namelist IOSTAT behaviour

Paul Thomas-4
In reply to this post by Neilen Marais-2
Tobi,

> might have been his patch.  I'll try to get certainty tomorrow, and
>  extract a testcase.

If you look at the PR, I already wrote a shortened test case. I was
proposing to submit a version of that tomorrow. If you can sort out the
reason for the spontaneous remission, that would be great. It is definitely
due to the correct level of dereferencing being applied. It was coming back
from the function with the pointer, rather than the value.

Cheers

Paul
Reply | Threaded
Open this post in threaded view
|

Re: Namelist IOSTAT behaviour

Steve Kargl
In reply to this post by Paul Thomas-4
On Thu, Jun 16, 2005 at 01:56:26AM +0100, Paul Thomas wrote:
> Steve,
>
> > looking at a problem with "USE, only : a => z, b => z" where
> > the "b => z" causes an ICE because the "a => z" removes z from
> > the symbols in the module.  
>
> I think that I can fix that but is it legal? It's a bit bizarre to tie two
> host symbols to one module variable, isn't it?  Is there an associated PR?
>

PR 18878.  AFAIK, yes, it's legal.  I found the problem because I have
a module that I use to set kinds.  The typical

   use kinds , only : sp => single, dp => double

one of my programs was acting up and to check that the single
precision part wasn't going unstable, I did

   use kinds , only : sp => double, dp => double

Lo and behold, gfortran barfed at every use of dp.

> > I also tried to understand the
> > problem with EQUIVALENCE in modules.  I gave up. ;-(
>
> This looks as if it is a problem of logic in trans_decl.c, rather than in
> module.c.  The error is generated because the equivalenced variable is found
> to already have a backend_decl by the time gfc_create_module_variable is
> called. I'll have a poke around; the odd thing is that it was a bug that
> appeared.

Hmmm.  I never made to trans-decl.c. I was stuck in module.c land.

> In a similar vein, PR19561 seems to have done the opposite and fixed itself
> spontaneously. Is somebody owning up or is this due to Tobi's and my
> rummaging around with dereferencing? Either way, I propose to write a test
> case so that it is nailed to the floor.

Yes, we should generate a test case.  Mysterious fixes are better
than mysterious regressions.

--
Steve