Problem linking 527.cam4_r SPEC benchmark (undefined externals with -fwhole-program)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem linking 527.cam4_r SPEC benchmark (undefined externals with -fwhole-program)

Steve Ellcey-10
I was wondering if anyone has run into a problem building the 527.cam4_r
benchmark from SPEC 2017 when using '-fwhole-program'?  This test is written
in Fortran and when I compile it with "-Ofast -fwhole-program" I get undefined
externals.  If I leave out the -fwhole-program or use -flto instead then it
compiles and links.

Steve Ellcey
[hidden email]

/extra/sellcey/gcc-tot/install/bin/gfortran       -mabi=lp64         -Ofast -fwhole-program -march=native     -fno-lto -DSPEC_CASE_FLAG     ESMF_BaseMod.fppized.o ESMF_BaseTimeMod.fppized.o ESMF_FractionMod.fppized.o ESMF_CalendarMod.fppized.o ESMF_TimeIntervalMod.fppized.o ESMF_Stubs.fppized.o ESMF_TimeMod.fppized.o ESMF_AlarmMod.fppized.o ESMF_ClockMod.fppized.o ESMF_AlarmClockMod.fppized.o ESMF_Mod.fppized.o cam_logfile.fppized.o debugutilitiesmodule.fppized.o decompmodule.fppized.o shr_kind_mod.fppized.o ghostmodule.fppized.o perf_utils.fppized.o shr_log_mod.fppized.o shr_mpi_mod.fppized.o shr_sys_mod.fppized.o shr_file_mod.fppized.o string_utils.fppized.o namelist_utils.fppized.o perf_mod.fppized.o mod_comm.fppized.o parutilitiesmodule.fppized.o mpishorthand.fppized.o abortutils.fppized.o units.fppized.o spmd_utils.fppized.o fv_control_mod.fppized.o pft_module.fppized.o dynamics_vars.fppized.o FVperf_module.fppized.o GPTLget_memusage.o GPTLprint_memusage.o GPTLutil.o MISR_simulator.fppized.o Meat.fppized.o pmgrid.fppized.o dycore.fppized.o pio_kinds.fppized.o pio_support.fppized.o pio_types.fppized.o pio_msg_mod.fppized.o alloc_mod.fppized.o pio_utils.fppized.o nf_mod.fppized.o pionfput_mod.fppized.o pionfatt_mod.fppized.o pio_spmd_utils.fppized.o calcdisplace_mod.fppized.o box_rearrange.fppized.o rearrange.fppized.o pionfread_mod.fppized.o iompi_mod.fppized.o pionfwrite_mod.fppized.o piodarray.fppized.o ionf_mod.fppized.o calcdecomp.fppized.o pio_mpi_utils.fppized.o piolib_mod.fppized.o pionfget_mod.fppized.o pio.fppized.o shr_timer_mod.fppized.o shr_string_mod.fppized.o shr_const_mod.fppized.o shr_cal_mod.fppized.o time_manager.fppized.o ppgrid.fppized.o physconst.fppized.o constituents.fppized.o commap.fppized.o infnan.fppized.o pspect.fppized.o rgrid.fppized.o spmd_dyn.fppized.o hycoef.fppized.o dyn_grid.fppized.o m_chars.fppized.o m_flow.fppized.o m_stdio.fppized.o m_mpif.fppized.o m_realkinds.fppized.o m_mpif90.fppized.o m_dropdead.fppized.o m_ioutil.fppized.o m_mpout.fppized.o m_die.fppized.o m_MergeSorts.fppized.o phys_grid.fppized.o ioFileMod.fppized.o cam_history_support.fppized.o cam_control_mod.fppized.o m_mall.fppized.o m_String.fppized.o m_rankMerge.fppized.o m_IndexBin_logical.fppized.o m_Permuter.fppized.o m_IndexBin_char.fppized.o m_IndexBin_integer.fppized.o m_SortingTools.fppized.o m_List.fppized.o m_TraceBack.fppized.o m_AttrVect.fppized.o m_GlobalMap.fppized.o m_FcComms.fppized.o m_MCTWorld.fppized.o m_GlobalSegMap.fppized.o m_AttrVectComms.fppized.o m_SparseMatrix.fppized.o m_SparseMatrixDecomp.fppized.o m_SparseMatrixComms.fppized.o m_Navigator.fppized.o m_GlobalToLocal.fppized.o m_SparseMatrixToMaps.fppized.o m_ConvertMaps.fppized.o m_ExchangeMaps.fppized.o m_Router.fppized.o m_Rearranger.fppized.o m_SparseMatrixPlus.fppized.o m_GeneralGrid.fppized.o m_Transfer.fppized.o m_inpak90.fppized.o m_GlobalSegMapComms.fppized.o m_Accumulator.fppized.o m_MatAttrVectMul.fppized.o m_GeneralGridComms.fppized.o mct_mod.fppized.o seq_comm_mct.fppized.o seq_drydep_mod.fppized.o seq_flds_mod.fppized.o seq_io_mod.fppized.o cam_pio_utils.fppized.o sat_hist.fppized.o solar_data.fppized.o shr_orb_mod.fppized.o shr_tInterp_mod.fppized.o shr_stream_mod.fppized.o shr_pcdf_mod.fppized.o shr_mct_mod.fppized.o shr_map_mod.fppized.o shr_ncread_mod.fppized.o shr_dmodel_mod.fppized.o shr_strdata_mod.fppized.o shr_scam_mod.fppized.o wrap_nf.fppized.o scamMod.fppized.o error_messages.fppized.o interpolate_data.fppized.o geopotential.fppized.o ref_pres.fppized.o phys_control.fppized.o physics_types.fppized.o time_utils.fppized.o repro_sum_mod.fppized.o phys_gmean.fppized.o mo_constants.fppized.o m_types.fppized.o mo_flbc.fppized.o chem_surfvals.fppized.o cam_history_buffers.fppized.o filenames.fppized.o cosp_share.fppized.o interp_mod.fppized.o cam_history.fppized.o advect_tend.fppized.o phys_buffer.fppized.o ghg_data.fppized.o radconstants.fppized.o radae.fppized.o quicksort.fppized.o radlw.fppized.o pkg_cldoptics.fppized.o phys_debug_util.fppized.o conv_water.fppized.o param_cldoptics.fppized.o mo_util.fppized.o rad_solar_var.fppized.o cmparray_mod.fppized.o radsw.fppized.o phys_prop.fppized.o rad_constituents.fppized.o wv_saturation.fppized.o modal_aer_opt.fppized.o aer_rad_props.fppized.o camsrfexch_types.fppized.o radiation_data.fppized.o cosp_constants.fppized.o cosp_utils.fppized.o radar_simulator_types.o cosp_types.fppized.o modis_simulator.fppized.o cosp_modis_simulator.fppized.o cosp_lidar.fppized.o cosp_misr_simulator.fppized.o llnl_stats.fppized.o lmd_ipsl_stats.fppized.o cosp_stats.fppized.o atmos_lib.o mrgrnk.o array_lib.o format_input.o math_lib.o optics_lib.o cosp_radar.fppized.o cosp_isccp_simulator.fppized.o cosp_simulator.fppized.o cosp.fppized.o cospsimulator_intr.fppized.o chemistry.fppized.o radheat.fppized.o radiation.fppized.o horizontal_interpolate.fppized.o polar_avg.fppized.o tracer_data.fppized.o prescribed_ghg.fppized.o prescribed_aero.fppized.o trb_mtn_stress.fppized.o phys_debug.fppized.o hb_diff.fppized.o upper_bc.fppized.o molec_diff.fppized.o diffusion_solver.fppized.o eddy_diff.fppized.o vertical_diffusion.fppized.o boundarydata.fppized.o cam3_ozone_data.fppized.o aoa_tracers.fppized.o iondrag.fppized.o cam3_aero_data.fppized.o tracers_suite.fppized.o tracers.fppized.o tropopause.fppized.o prescribed_volcaero.fppized.o prescribed_ozone.fppized.o aircraft_emit.fppized.o modal_aero_deposition.fppized.o aerodep_flx.fppized.o advnce.fppized.o drydep_mod.fppized.o wetdep.fppized.o dust_sediment_mod.fppized.o dust_intr.fppized.o progseasalts_intr.fppized.o scyc.fppized.o aerosol_intr.fppized.o history_scam.fppized.o xpavg_mod.fppized.o ncdio_atm.fppized.o metdata.fppized.o co2_data_flux.fppized.o co2_cycle.fppized.o buffer.fppized.o comsrf.fppized.o restart_physics.fppized.o pfixer.fppized.o ctem.fppized.o diag_module.fppized.o dyn_comp.fppized.o dyn_internal_state.fppized.o restart_dynamics.fppized.o cam_restart.fppized.o readinitial.fppized.o history_defaults.fppized.o error_function.fppized.o uw_conv.fppized.o uwshcu.fppized.o cldwat.fppized.o zm_conv.fppized.o hk_conv.fppized.o convect_shallow.fppized.o cloud_fraction.fppized.o cldwat2m_macro.fppized.o ndrop.fppized.o cldwat2m_micro.fppized.o microp_aero.fppized.o microp_driver.fppized.o pkg_cld_sediment.fppized.o stratiform.fppized.o inidat.fppized.o startup_initialconds.fppized.o inital.fppized.o check_energy.fppized.o dp_coupling.fppized.o fv_prints.fppized.o stepon.fppized.o ionosphere.fppized.o cloud_diagnostics.fppized.o cloud_rad_props.fppized.o tidal_diag.fppized.o constituent_burden.fppized.o cam_diagnostics.fppized.o flux_avg.fppized.o macrop_driver.fppized.o gw_drag.fppized.o zm_conv_intr.fppized.o convect_deep.fppized.o rayleigh_friction.fppized.o majorsp_diffusion.fppized.o sslt_rebin.fppized.o physpkg.fppized.o cam_comp.fppized.o seq_infodata_mod.fppized.o seq_cdata_mod.fppized.o runtime_opts.fppized.o cam_cpl_indices.fppized.o seq_timemgr_mod.fppized.o atm_comp_mct.fppized.o benergy.fppized.o binary_io.fppized.o bnddyi.fppized.o seq_flds_indices.fppized.o seq_diag_mct.fppized.o seq_rearr_mod.fppized.o map_ocnocn_mct.fppized.o mrg_x2s_mct.fppized.o ice_comp_mct.fppized.o mrg_x2o_mct.fppized.o seq_avdata_mod.fppized.o seq_hist_mod.fppized.o mrg_x2a_mct.fppized.o map_iceocn_mct.fppized.o seq_map_mod.fppized.o map_atmlnd_mct.fppized.o map_atmice_mct.fppized.o map_snoglc_mct.fppized.o map_atmocn_mct.fppized.o seq_domain_mct.fppized.o seq_rest_mod.fppized.o mrg_x2l_mct.fppized.o seq_frac_mct.fppized.o shr_flux_mod.fppized.o seq_flux_mct.fppized.o map_lndlnd_mct.fppized.o map_iceice_mct.fppized.o map_glcglc_mct.fppized.o lnd_comp_mct.fppized.o ocn_types.fppized.o ocn_comp.fppized.o ocn_comp_mct.fppized.o map_rofrof_mct.fppized.o map_atmatm_mct.fppized.o map_rofocn_mct.fppized.o shr_mem_mod.fppized.o map_snosno_mct.fppized.o mrg_x2g_mct.fppized.o mrg_x2i_mct.fppized.o glc_comp_mct.fppized.o ccsm_comp_mod.fppized.o ccsm_driver.fppized.o tp_core.fppized.o sw_core.fppized.o cd_core.fppized.o cldsav.fppized.o collective.o comm.o comspe.fppized.o cpslec.fppized.o d2a3dijk.fppized.o d2a3dikj.fppized.o dadadj.fppized.o datetime.fppized.o diag_dynvar_ic.fppized.o mean_module.fppized.o dryairm.fppized.o dsd.o mapz_module.fppized.o epvd.fppized.o esinti.fppized.o f_wrappers.o fft99.fppized.o fill_module.fppized.o fort.fppized.o gases.o gauaw_mod.fppized.o geopk.fppized.o get_zeits.o gffgch.fppized.o gptl.o gptl_papi.o group.o handles.o hirsbtpar.o hirsbt.o icarus.fppized.o initcom.fppized.o initindx.fppized.o intp_util.fppized.o iop_surf.fppized.o lidar_simulator.fppized.o list.o load_hydrometeor_classes.o m_AccumulatorComms.fppized.o m_AttrVectReduce.fppized.o m_StrTemplate.fppized.o m_FileResolv.fppized.o m_Filename.fppized.o m_Merge.fppized.o m_SpatialIntegralV.fppized.o m_SpatialIntegral.fppized.o m_zeit.fppized.o marsaglia.fppized.o mcshallow.fppized.o pio_quicksort.fppized.o mct_rearrange.fppized.o miesubs.fppized.o mo_msis_ubc.fppized.o mo_regrider.fppized.o mo_solar_parms.fppized.o mpi.o msise00.fppized.o p_d_adjust.fppized.o pack.o par_vecsum.fppized.o par_xsum.fppized.o pf_to_mr.fppized.o pio_msg_callbacks.fppized.o pio_msg_getput_callbacks.fppized.o pio_nf_utils.fppized.o pkez.fppized.o prec_scops.fppized.o puminterfaces.fppized.o qneg3.fppized.o qneg4.fppized.o radar_simulator.o recv.o redistributemodule.fppized.o req.o scops.fppized.o send.o sgexx.fppized.o shr_jlcp.o shr_msg_mod.fppized.o shr_vmath_fwrap.o shr_vmath_mod.fppized.o srchutil.fppized.o srfxfer.fppized.o sulchem.fppized.o te_map.fppized.o threadutil.o time.o topology.o tphysac.fppized.o tphysbc.fppized.o tphysidl.fppized.o trac2d.fppized.o trunc.fppized.o tsinti.fppized.o uv3s_update.fppized.o virtem.fppized.o vrtmap.fppized.o wrap_mpi.fppized.o wrf_error_fatal.fppized.o wrf_message.fppized.o zeff.o zenith.fppized.o netcdf/attr.o netcdf/dim.o netcdf/error.o netcdf/fort-attio.o netcdf/fort-control.o netcdf/fort-dim.o netcdf/fort-genatt.o netcdf/fort-geninq.o netcdf/fort-genvar.o netcdf/fort-lib.o netcdf/fort-misc.o netcdf/fort-v2compat.o netcdf/fort-var1io.o netcdf/fort-varaio.o netcdf/fort-vario.o netcdf/fort-varmio.o netcdf/fort-varsio.o netcdf/libvers.o netcdf/nc.o netcdf/ncx.o netcdf/posixio.o netcdf/putget.o netcdf/string.o netcdf/v1hpg.o netcdf/v2i.o netcdf/var.o netcdf/typeSizes.o netcdf/netcdf.o spec_qsort/spec_qsort.o                      -o cam4_r  
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.o: in function `main':
ccsm_driver.fppized.f90:(.text.startup+0x1c): undefined reference to `__ccsm_comp_mod_MOD_ccsm_pre_init'
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.f90:(.text.startup+0x2c): undefined reference to `__esmf_stubs_MOD_esmf_initialize'
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.f90:(.text.startup+0x30): undefined reference to `__ccsm_comp_mod_MOD_ccsm_init'
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.f90:(.text.startup+0x34): undefined reference to `__ccsm_comp_mod_MOD_ccsm_run'
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.f90:(.text.startup+0x38): undefined reference to `__ccsm_comp_mod_MOD_ccsm_final'
/extra/sellcey/gcc-tot/install/lib/gcc/aarch64-linux-gnu/10.0.0/../../../../aarch64-linux-gnu/bin/ld: ccsm_driver.fppized.f90:(.text.startup+0x40): undefined reference to `__esmf_stubs_MOD_esmf_finalize'
collect2: error: ld returned 1 exit status
specmake: *** [/extra/sellcey/spec2017/benchspec/Makefile.defaults:337: cam4_r] Error 1
Reply | Threaded
Open this post in threaded view
|

Re: Problem linking 527.cam4_r SPEC benchmark (undefined externals with -fwhole-program)

Thomas Koenig-6
Hi Steve,

> I was wondering if anyone has run into a problem building the 527.cam4_r
> benchmark from SPEC 2017 when using '-fwhole-program'?

I think that this is what -fwhole-program is supposed to do:

`-fwhole-program'
      Assume that the current compilation unit represents the whole
      program being compiled.  All public functions and variables with
      the exception of `main' and those merged by attribute
      `externally_visible' become static functions and in effect are
      optimized more aggressively by interprocedural optimizers.

Your command line looks like the program consists of more than one
translation unit, so it does seem to be expected behavior if you get
missing external functions.

Regards

        Thomas
Reply | Threaded
Open this post in threaded view
|

Re: Problem linking 527.cam4_r SPEC benchmark (undefined externals with -fwhole-program)

Steve Ellcey-10
On Tue, 2019-08-06 at 22:43 +0200, Thomas Koenig wrote:

>
> Hi Steve,
>
> > I was wondering if anyone has run into a problem building the
> > 527.cam4_r
> > benchmark from SPEC 2017 when using '-fwhole-program'?
>
> I think that this is what -fwhole-program is supposed to do:
>
> `-fwhole-program'
>       Assume that the current compilation unit represents the whole
>       program being compiled.  All public functions and variables
> with
>       the exception of `main' and those merged by attribute
>       `externally_visible' become static functions and in effect are
>       optimized more aggressively by interprocedural optimizers.
>
> Your command line looks like the program consists of more than one
> translation unit, so it does seem to be expected behavior if you get
> missing external functions.
>
> Regards
>
> Thomas

I guess I am confused about what -fwhole-program means.  I was thinking
it was sort of a super-lto.  I.e.  I could use -flto to combine
multiple objects into one compilation and use -fwhole-program to say
these objects are the entire program.  I guess that is not the case.

Reading the documentation (lto.texi) I guess what I want is WHOPR mode
but I am not sure how to invoke that.  The document talks about LTO
mode and WHOPR or partitioned mode but I am not sure what flags I need
to use to get WHOPR mode.

Steve Ellcey
[hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Problem linking 527.cam4_r SPEC benchmark (undefined externals with -fwhole-program)

Richard Biener-2
On Tue, Aug 6, 2019 at 11:06 PM Steve Ellcey <[hidden email]> wrote:

>
> On Tue, 2019-08-06 at 22:43 +0200, Thomas Koenig wrote:
> >
> > Hi Steve,
> >
> > > I was wondering if anyone has run into a problem building the
> > > 527.cam4_r
> > > benchmark from SPEC 2017 when using '-fwhole-program'?
> >
> > I think that this is what -fwhole-program is supposed to do:
> >
> > `-fwhole-program'
> >       Assume that the current compilation unit represents the whole
> >       program being compiled.  All public functions and variables
> > with
> >       the exception of `main' and those merged by attribute
> >       `externally_visible' become static functions and in effect are
> >       optimized more aggressively by interprocedural optimizers.
> >
> > Your command line looks like the program consists of more than one
> > translation unit, so it does seem to be expected behavior if you get
> > missing external functions.
> >
> > Regards
> >
> >       Thomas
>
> I guess I am confused about what -fwhole-program means.  I was thinking
> it was sort of a super-lto.  I.e.  I could use -flto to combine
> multiple objects into one compilation and use -fwhole-program to say
> these objects are the entire program.  I guess that is not the case.
>
> Reading the documentation (lto.texi) I guess what I want is WHOPR mode
> but I am not sure how to invoke that.  The document talks about LTO
> mode and WHOPR or partitioned mode but I am not sure what flags I need
> to use to get WHOPR mode.

-flto "auto-detects" -fwhole-program if you use a linker plugin.  If you do not
use a linker-plugin it might help to do -flto -fwhole-program (note it will
only apply at link time).  -fwhole-program on its own without using LTO
assumes the whole program is in a single TU.

So to summarize when you are using a linker plugin -flto is already "super" ;)

And WHOPR "mode" exactly is what -flto does.  The docs are probably
confusing here.  There's non-partitioned mode (-flto-partition=none) but
there's hardly a difference in optimization.

Richard.

> Steve Ellcey
> [hidden email]