People who attended the meeting:

From NEC-CCRL, René Redler, Hubert Ritzdorf; from SGI Deutschland, Reiner Vogelsang; from NEC, Thomas Schoenemeyer; from CERFACS, Sophie Valcke, Damien Declat.

The following points were discussed:

1. Info on OASIS3 (S. Valcke)
2. Info on ESMF-PRISM interaction (S. Valcke)
3. Poster for VECPAR (D. Declat)
4. Review of current developments and establishment of priorities for OASIS4 final version
5. Conclusions and next meeting

1. Info on OASIS3 (S. Valcke)

A new version of OASIS3, tagged "oasis3_prism_2-2" was officially released on June 18th. It is currently available on CERFACS CVS server and will be available on PRISM CVS server pretty soon. Another version, including appropriate treatment of vector interpolation will most probably be released later this summer.

2. Info on ESMF-PRISM interaction (S. Valcke, R. Redler)

Sophie and René will participate to the PRISM-ESMF meeting planned during the 2nd week of Setptember at GFDL. 

Giang Nong contacted Sophie with questions related to running OASIS3 toymodels on their SGI at GFDL. OASIS3 and associated toymodels use the full PRISM Standard Compiling and Running Environment (PRISM SCE and SRE). We are happy of course that they test and/or use OASIS3 + PRISM SRE and SCE. But we would like also to base the ESMF-PRISM collaboration on a comparison running a coupled model based on MOM and a toy atmosphere model both with OASIS4 (which does not use yet the full PRISM SCE and SRE) and with ESMF infrastructure. René agreed to work on this.
Contact after the meeting confirmed that this is still ESMF intention too; however, even if a first version of OASIS4 tagged "OASIS4_0_beta1" is available on PRISM CVS server bedano, the proto_ex bug (see below) first has to be solved before going on with this collaboration.

3. Poster for VECPAR (D. Declat)

The final version of the poster that will be presented at VECPAR'04 6th International meeting in Valencia on June 28-30 by Damien was discussed. The final version can be found here.

4. Review of current developments and establishment of priorities for OASIS4 final version

The current status of OASIS4 developments was reviewed and the following list of priorities was established. The developments marked with "High priority" should be done as soon as possible. Then a period of heavy testing should follow. Then we should proceed with the developments  marked with "Medium priority", hopefully to be included in the PRISM final version of OASIS4. The developments  marked with ""Medium-Low priority" and "Low priority" will be done after the end of PRISM. (In what follows, T and PS stand respectively for Transformer and PSMILe.)

4.0 Review of current developments

• Interpolation:
• 3D nearest-neighbour (nneighbour3D) and trilinear (trilinear) currently supported
• Can be used only if vertical units in meters
  
• PS: local parallel neighbour search only (corresponding to para_search=approximate)
• PS: optimised neighborhood search depending on grid_type
• PS: no value calculated for masked target points
• nneighbour3D: PS: currently, number of neighbours is fixed
• nneighbour3D: PS: source nearest neighbours chosen among non masked points only (corresponding to used_masked=false)
• trilinear: PS: if some of the 8 trilinear neighbours are masked, 8 non masked nearest neighbours are found
  
• T detects masked source neighbours and does not use them in weights calculation

• Support of REAL, DOUBLE, INTEGER fields : OK for PS, T, I/O
• Support of bundles, subgrids, vectors, and bundle of vectors: PS API: OK (see notes on var_nodims(2) in "High priority" below); PS I/O: bundle name need to be initialised; PS: not OK; T: not OK
• Support of different type of exchange_date: only period for now
• Support of source_time_operation: linear taverage + accumul OK
• Support of statistics (masked_points, notmasked_points, all_points) on source and target side: OK
• Support of scattering on source side below prism_put: OK
• Support of gathering on target side below prism_get: OK
• Support of add_scalar and mult_scalar on source and target side: OK
• Support of target_time_operation (time_nneighbour | time_linear) for I/O input data only (as in 13/01/2004 minutes): OK
• Calendars supported: only proleptic Gregorian for now
• Use of date_bounds by PS: OK; PS verifies that the datebounds cover the whole running period
• PSMILe I/O:
• Format supported:  mpp_netcdf OK; mpp_ieee32 OK for O; mpp_native OK for O
• GME grid (one grid per block, only one variable name for many blocks): OK
• File name identified by base + extension (input and output): OK
• io_mode: iosingle, distributed: OK 
• packing, scaling, adding: OK; only for NetCDF
• Support of debug_mode (below prism_put and prism_get): OK
  

4.1 High priority developments

• Debug proto_ex example. Why is transi_out_id sent by ocean PS to T is always 2? Determination of global IDs by Driver seems OK in all cases. (RR, HR, DD)

• Treatment of lag and associated automatic writing and reading of coupling restart (cold and normal restarts) (RR)
• lag is defined in terms of "prism_put period", i.e. the difference between the associated date_bounds. E.g. for a lag of 1, the time to add to the prism_put date and date_bounds arguments would be 1 X the difference between the associated date_bounds. There is still the problem that for models with varying timestep, the "lagged" period calculated that way may not exactly cover the period that would have been covered by the next prism_put, as the timestep may vary. (We could add an additional argument giving explicitely the period covered by the next prism_put, but even then this information may not be known by the model.)
• below the last prism_put, fields with lag should be written to files (one file per process with field name, component name, application name, process number and run final date) 
• at restart, at the end of prism_enddef, fields with lag will be read by PSMILe and send to Transformer. 

• Interpolation:
• 3D nearest-neighbour (nneighbour3D) and trilinear (trilinear)
• T: attribution of  PRISM_undef value to masked target points or to target points for which no value can be calculated (DD)
  
• PS: nneighbour3D: use of user's choice for nbr_neighbours value (HR)
• T: nneighbour3D: use of user's choice for gaussian_variance (DD)
• PS: nneighbour3D: use of user's choice for used_masked value (used_masked=false or true) (HR)
• PS: trilinear: use of user's choice for if_masked (see below) 
• For 2D grid (i.e for grids having a vertical dimension of one), nneighbour3D and trilinear should automatically result in  nneighbour2D and bilinear respectively (PS: HR; T: DD)
  
• SMIOC: add if_masked element; modify Driver accordingly (SV)
• novalue: if some of the 8 trilinear neighbours are masked, no neighbours are transfered from PS to T, and  PRISM_undef value is given to that target point
• tneighbour: if some of the 8 trilinear neighbours are masked, the non-masked points among those 8 points are used and transfered from PS to T; if the 8 trilinear neighbours are masked, PRISM_undef value is given to that target point
• nneighbour: if some of the 8 trilinear neighbours are masked, the non-masked points among those 8 points are used; if the 8 trilinear neighbours are masked, the non masked nearest neighbour is used 
• 2D1D:
• 2D1D can be used for all grids which vertical dimension can be expressed as z(k), i.e. for i.e. for source grid types PRISM_reglonlatvrt, PRISM_irrlonlat_regvrt, PRISM_unstructlatlon_regvrt; the mask may vary with depth. The interest of 2D1D is to allow a linear interpolation in the vertical. Change definition in ADSCCPMIODSMIOC.tex (SV).
• Can be used for grids with vertical units into which linear interpolation makes sense.
  
• 2D1D supported in PSMILe following user's choice in SMIOC  (inter2D,interp1D) (HR?)
• 2D: 2Dnneighbour, bilinear (PS: HR; T: DD)
• 1D: 1Dnneighbour, linear (PS: HR; T: DD)
• No need of particular PS-T transfer routines
• T: Modify the weights calculation to really apply a 2D1D interpolation following user's choice in SMIOC

• License issues: OASIS4 should be delivered with a LGPL license. However, mpp_io which is included in OASIS4 uses a GPL license. Sophie will contact Balaji to see if the mpp_io version modified by Reiner (support of GMD-like grids) could be released with LGPL. If not, two versions of OASIS4 will have to be released: one LGPL without I/O, and one GPL with I/O. (SV)

• Transformer:
• Parallelisation (TS, DD)
• too much allocation/deallocation because of use of local structures? (DD)

• Workarounds:
• Definition of communicators if applications may contain many components running sequentially:
• Check if ranks of components per application are needed (RR) (Since then, René confirmed that they are needed, 21/06)
• If so, transfer ranks of components per application from Driver (DD)
• Use ranks to define local component communicators (RR)
• PS: Correct transfer of SMIOC information if application contains more than one component: currently, component information is overwritten. (RR, DD) (See proposition from René in mail dated June 22)

• Grid corner localisation: 
• update WEB site (RR)
• check if shape of corner_xxx_array are coherent with what is expected (RR)
• ordering: Fortran ordering if applicable; counterclockwise looking in direction of z at each level for irrlonlat_regvrt, irrlonlat_sigmavrt, reglonlat_sigmavrt, unstructlonlat_regvrt, unstructlonlat_sigmavrt; volume_type needed (see below in low priorities) for irrlonlatvrt and unstuctlonlatvrt
  

grid_type							nbr_c
reglonlatvrt	x(i)	y(j)	z(k)	c_1_a(i,lx)	c_2_a(j,ly)	c_3_a(k,lz)	lx*ly*lz	lx=ly=lz=2
irrlonlat_regvrt	x(i,j)	y(i,j)	z(k)	c_1_a(i,j,lh)	c_2_a(i,j,lh)	c_3_a(k,lz)	lh*lz	lz=2
irrlonlatvrt	x(i,j,k)	y(i,j,k)	z(i,j,k)	c_1_a(i,j,k,lc)	c_2_a(i,j,k,lc)	c_3_a(i,j,k,lc)	lc
irrlonlat_sigmavrt	x(i,j)	y(i,j)	z(i,j,k)	c_1_a(i,j,lh)	c_2_a(i,j,lh)	c_3_a(i,j,k,lc)	2*lh
reglonlat_sigmavrt	x(i)	y(i)	z(i,j,k)	c_1_a(i,lx)	c_2_a(j,ly)	c_3_a(i,j,k,lc)	lc=2*lx*ly
unstructlonlat_regvrt	x(nhor)	y(nhor)	z(k)	c_1_a(nhor,lh)	c_2_a(nhor,lh)	c_3_a(k,lz)	lh*lz	lz=2
unstructlonlat_sigmavrt	x(nhor)	y(nhor)	z(nhor,k)	c_1_a(nhor,lh)	c_2_a(nhor,lh)	c_3_a(nhor,k,lc)	lc=2*lh
unstuctlonlatvrt	x(ntot)	y(ntot)	z(ntot)	c_1_a(ntot,lc)	c_2_a(ntot,lc)	c_3_a(ntot,lc)	lc

• Updates on PSMILe API WEB site (RR)
• remove flow charts from WEB
• remove prism_def_restvar 
• prism_def_var: keep var_nodims(2)for bundle only, dimension var_nodims(1) otherwise
  

• Check if prism_get_localcomm called with PRISM_appl_id works fine (RR)

• Redefine prism_def_partition: extent of blocks need to be added as arguments (SV, RV); update WEB(RR)

• I/O:Check fill_value: input grid point values that equal fill_value are turned to PRISM_undef in the PSMILe, and output grid point values that equal PRISM_undef are turned to fill_value in the output file (RV)

• Driver:
• Check that coupling fields are of same type on source and target sides (SV, DD)
• Check error bound checking (see René's mail June 16th)
  

• XML files:
• Sasa: DOUBLE read as INTEGER: use sscanf (DD) 
• Move sasa routines out of psmile_smioc.F90, put CPP flag around sasa calls in prism_init_comp, in order to avoid linking the application with libxml in non standalone case (SV)
• Add reading of interp2D nneighbour2D bilinear and interp1D nneighbour1D linear SMIOC info (SV, DD)
  
• SMIOC: change transient units to character strings; modify Driver accordingly (SV)
• grid_name  (and not grid_family_name) should be coherent with PRISM_def_grid argument; change documentation and comments in code (SV)
• Add bunvec for vector of bundles in transient_type which should be mandatory; modify Driver accordingly (SV)
  

4.2 Medium priority developments

• Interpolation:
• Documentation on algorithms implemented in PSMILe (HR)
  
• 3D nearest-neighbour (nneighbour3D) and trilinear (trilinear): PS: exact parallel neighbour search and use of user's choice for para_search value (para_search=precise or approximate) (HR)
• 2D conservativ2D (PS-T transfer of additional grid information (e.g. areas) in array(epio_size, nbr_info) will be needed - all these info will therefore be transfered with same precision) (HR, DD)
• 2D bicubic
• 1D1D1D:
• Can be used for grids with vertical units into which linear interpolation makes sense.
• 1D1D1D supported in PSMILe following user's choice in SMIOC  (inter1D,interp1D,interp1D) 
• 1D: 1Dnneighbour, linear, none
• No need of particular PS-T transfer routine
  

• Workarounds to be solved with MP:
• Translate comments in German (RR)
• One workaround for gridless data
  

• XML files
• Write wrapping routines for transfer or direct reading in stand-alone case of SCC info, e.g. psmile_scc_init (transfer only application info) (DD)
• AD: add element nbr_procs and arguments (lauching arguments) (SV)
• SCC: transfer and use redirect attribute (DD, RR)
• SMIOC: 
• use of pole_covered (RR)
• remove useless elements in interp3D, interp2D, interp1D (SV)
• Check if packing, scaling, adding is for NetCDF only (RV); if so, change SMIOC structure and doc (SV)
  

• PSMILe function to get SCC and SMIOC info in model code (prism_get_persist, prism_query_var) (SV)

• PSMILe calendar tool (SV)

• Support of bundles, subgrids, vectors, and bundle of vectors for PS internal and DEL, PS I/O (bundle name need to be initialised OK) and T (definition of new PS-T transfer routines?, transfer of info on pole covered for vectors)

• Support of non-gridded data

• Support of other type of exchange_date (once, precise, lastdayinmonth, list) 

• Support of calendars other than proleptic Gregorian
  

• I/O:
• Add option in SMIOC and use it in PS so that grid information is not written to the file with the data (RV, SV)
• io_mode: parallel (use of parNetCFD)
  

• Transformer:
• Support of coupling fields with different types on source and target sides

• Driver
• Spawn of applications on different host not possible presently as only one host name is given as argument of PSMILe_Spawn_Child (DD)
• Check that interpolation method chosen is appropriate for the grid (e.g. check that vertical units are in meters if  nneighbour3D or trilinear is chosen (DD, SV)
• Check that the different component SMIOC info is coherent (e.g. global parameters, transient units, sg_exch_date for a transient_out match the sg_exch_date for the corresponding transient_in, coupling frequency cannot be higher that the frequency of the prism_get, uniquess of application names...)

• Coherence checks between information given through PSMILe API and SMIOC info: consistency checks only (e.g. grid info) or stop the run if incoherence is observed (i.e. coupling data)? 

• Check error code handling for all calls

• Check prism_set_scalefactors (RR)
• Implement prism_put_restart (RR)

4.3 Medium-Low priority developments

• Support of stand-alone models (no Driver)

• Implement prism_put_inquire, prism_iput, prism_iget, prism_wait 

• Additional Driver functions (prism_timestamp, prism_restartsaved, prism_message) 

• PS and T support of  GME (one grid per block, only one variable name for many blocks)
  

4.4 Low priority developments

• Interpolation
• Interpolator-only mode: OASIS4 will not be usable in interpolator-only mode. Use of pseudo-models will be mandatory. Pseudo-models could be provided with OASIS4. Pseudo-models developed by Christian Mohn, presently National University of Ireland Galway, will be investigated (RR).
• Storage of neighbours weights and address in file to avoid recalculation each time. Given OASIS4 design, this would not be trivial to implement; we leave this question open until the need is expressed. 
• 3D conservativ3D, user3d
• 2D user2d
• 1D cubic, conservativ1D, user1d
  
• 1D vertical interpolation in log of vertical units (unit_logarithm)
• reconstruction of vertical position  of grid points  based on the units and auxiliary fields (see CF convention)
  

• Support of source_time_operation: tmin and tmax

• Support of reduction, coupling and I/O after reduction

• Support of algebraic_combinations (to be defined further)
  

• Support of unstructured grid (PS I/O already OK if local partition can be expressed with only an offset in the global index space. For unstructured grids, this is the case if the local partition covers one segment in a 1D global space), expression of connectivity.

• Volume type: for  irrlonlatvrt and unstuctlonlatvrt grids, a description of the way the corners are connected is needed. An additional PSMILe call defining the connectivity between the corners could be used, but the prefered solution for now would be to explicitely defined different volume types in a User's Guide and refer to those explicitely in the SMIOC.

• Saving of coupling restarts more frequently than run chuncks

• Treatment of "abort restart"

• Support of grid evolving with time
• Automatic verification of what has changed and recalculation only for what has changed?
• Problem if the grid changes between prism_put and coreesponding prism_get with lag
  
• Problem of automatic writing and reading of coupling restarts
  

• XML files
• SMIOC: 
• compute_space not needed for now neither in SMIOC, Driver, PSMIle (corresponding ig_compute_id transfered but not defined).
• conditional_computation not needed for now neither in SMIOC, Driver, PSMILe
• SCC keyword for saving more frequently than run chuncks
  

• PSMILe C API

• prism_def_var: arguments redundant with SMIOC information (SV)

• PSMILe I/O:
• Format supported:  mpp_ieee32 OK for I; mpp_native for I; mpp_ascii; GRIB; other
• Output file size chosen by user; add a keyword in SMIOC (RV, SV)