PSMILe interface proposal

-in green: IPSL last comments
-in red: WP3A new comments or WP3a old comments that still need to be discussed,
-in orange: modifications that wp3a/wp4a agreed on.
-in black and small characters: WP3a old comments on issues that are solved
-in magenta and small character: text that wp3a/wp4a agreed that should be removed

• PRISM_init
• PRISM_init_comp
• PRISM_get_localcomm
• PRISM_initialized (?)
• PRISM_enddef
• PRISM_decommission (?)
• PRISM_terminate
• PRISM_terminated (?)

Initializing the PSMILe:

Name :

PSMILe_init(model_name, SMIOC_file_name, [MPI_id], [domain])

Description :

This function will initialize the PSMILe. This will trigger the following actions : 1) read the SMIOC, 2) set-up all data structure, verify that the SMIOC matches with the model name.

A flow diagram for the PSMILe_init function

>Rene> Agreed with your (IPSL) second paragraph. That's how we think of implementing it. Nevertheless we would recommend to implement the two calls. Concerning your third paragraph: We have to perform the initialisation for every process. This is absolutely necessay to set up the communication patterns. How you have to initialise the I/O is up to you. Once we have established the communication we can distribute all information that is required.

Arguments :

model_name (IN) : provides the name of the model.


>WP3a: OK for PRISM_init_comp; the model_name will be in the PMIOD, in the SMIOC and in the SCC.

SMIOC_file_name (IN) : Provides the name of the SMIOC file to be read. If the name start with a "$" it will refer to an environment variable to be obtained from the shell.

>WP3a: Not OK; the SMIOC name should be determined automatically based on the model_name. A check of coherence to make sure that the model is accessing the right SMIOC will be possible by verifying the model_name contained in the SMIOC. We discussed this issue with WP4b people who strongly discourage to hard-code any file name in the codes. For multi-simulations (e.g. many simulations which differ only by the value of one SMIOC parameter), the PRISM System (WP4b) will automatically manage the modification in the SMIOC. Furthermore, using an environment variable is a possible cause of bug if it is by error not defined in the run script.

>IPSL : From our experience with these issues we would regret to close this option by not providing the possibility for the component developer to impose a name for the SMIOC file. The typical usage we would do is to put under 'model_name' a name which is generated automatically by CVS with information on the model version. This would allow us to know from each file created by this model which version was used. On the other hand we would not wish the name in 'SMIOC_file_name ' to change that often. Another solution would be to have two names for each model : A generic one which would be 'NCAR-GCM' and a detailed one which could be in this case 'CCM3-3.1'. In this scenario the SMIOC file name would be generated from the generic name and not the detailed one.

> Sophie 26/07/02: I agree that the SMIOC file name should be generated from a generic model name, and I do not understand why we could not use the argument model_name to built that SMIOC name automatically.

MPI_id (IN ?, OPTIONAL) : Allows to get the MPI id for this part of the model (?. is it needed ?).
>WP3a: Instead of using an optional argument, we propose to use another routine PRISM_get_localcomm (see below).

domain (IN,OPTIONAL) : Described the domain over which this instance of the model runs. The method to describe this domain still remains to be determined : vector of indices, bounds of intervals or mask.
>WP3a: We do not understand the utility of this argument. The domain will be described later by the grid definition.
>IPSL : OK, it makes more sense to describe the halo of the grid together with the grid. Especially since this description will depend on the structure of the grid.
 

>WP3A: I guess that depending on the discussions for the points marked ??? above, we will or not maintain the following  suggestions for PRISM_init, PRISM_init_comp.

Name :
PRISM_init(ierror)
Responsible: NEC-CCRLE/SGI

Description:
This routine

if PRISM_Init has been called before, returns with a warning

if MPI_Init has not been called, calls MPI_Init

opens coupling output files (one per process implied in coupling for debugging; for production, only root process writes to stderr and stdout).

determines if the calling process is part of a standalone executable

if MPI2 is used, participates to the launching the other executables

Name :
PRISM_init_comp(model_name, comp_id, ierror)
Responsible: NEC-CCRLE/SGI

Description:

 This function

• calls PRISM_Init if it it has not been called before
• if PRISM_init_comp has been called before with argument 'model_name', returns with a warning.
• If MPI1 is used, defines local communicator  for the component (coloring) following the method provided by Reiner Vogelsang (SGI) implemented in OASIS 2.5.
• reads all the information on the component variables in the component SMIOC; for coupling variables, receives SCC information from Driver if the calling process is part of a non  standalone executable, or  reads SCC information directly in SCC if the the calling process is part of a standalone executable. (Note: For a parallel standalone component, only process with rank 0 in the model local communicator should perform the SMIOC or SCC access and distributes information to the other processes.)

>Sophie 24/07/02: Stephanie prefers PRISM_init_model as a name, as "_comp" could be understood as "compile" or "compare".

>WP3A: As there were no comment from IPSL, we maintain the following suggestions for PRISM_initialized and PRISM_get_localcomm.
Name :
PRISM_initialized(ierror)
Responsible: NEC-CCRLE/SGI
Description:
This routine checks if PRISM_init has been called before.

Name :
PRISM_get_localcomm(local_comm, ierror)
Responsible: NEC-CCRLE/SGI

Description:
This routine returns the local communicator defined by PRISM_init_comp. Only MPI1 parallel codes need to call this function.
>Sophie 24/07: In my toymodels, the models do not know a priori if they run with MPI1 or MPI2. Therefore, I suggest that this PRISM_get_localcomm could also be used by MPI2 code and, in that case, MPI_COMM_WORLD would be returned as local_comm

>WP3a: In this category, we also need:

Name:
PRISM_setup(ierror):
PRISM_enddef(ierror):
Responsible: NEC-CCLRE

Description:
This function terminates the initialisation phase:

• builds all communicators.
• builds the interpolation matrices based on complete grid information of all participating components.
• sets-up all data structures.

>IPSL : As we commented this morning such a function (which should rather be called PRISM_enddef as it is not very logical to call a routine, which is supposed to terminate something 'setup' !) would be needed if we try to avoid having to carry a component ID. Else we can always take the needed actions for the communicators with the first call to PRISM_put or PRISM_get. If we select to introduce such a command then we also need to have a function PRISM_redef to return into the definition phase.

>WP3a: A comp_id is needed even if there is a PRISM_setup -or PRISM_enddef. This PRISM_setup/enddef is a collective action that needs to be called once per process for all processes implied in coupling and not once per component. Therefore it cannot be called below the first PRISM_put or PRISM_get (see the document http://www.cerfacs.fr/~valcke/PRISM/MTCI/notes_PSMILe_V1.html ).

>Sophie: I am OK for PRISM_enddef as a name.

Name:
PRISM_decommission (gridormask_ids, no_gridormask_ids, ierror):
Responsible: NEC-CCLRE

Description:

In case a grid or a mask has changed, this call will free all grids or masks which Ids are indicated in the list gridormask_ids. For further communication these grids have to be redefined by calls to the appropriate define functions and the PRISM_enddef.

Arguments:

gridormask_ids: array of grid or mask Ids.

no_gridormask_ids: the number of elements in gridormask_ids

>IPSL : It seems a little awkward to have these decommission functions. Could this not be covered much more efficiently by offering the possibility of defining grids which point to a time evolving variable for the coordinates of the grid points ? I am convinced that if the grids change in an earth system model then they will change all the time and not every month or twice a year. Thus if we have to call a _decommission and _defgrif at each time step it is going to be hell ! We do not have here a classical hydrodynamic problem where you change regime and thus redefine your grid. The earth system is in constant change of regime and this is the reason why adaptive grids have never taken off in our field of research, a personal view I have to say !
>WP3a: ??? This PRISM_decommission is analog to your PRISM_redef suggested above.

>Sophie: didn't we say at one point that PRISM_decommission could be omitted: it is obvious that if a PRISM_def_grid is called for a certain grid_id, this grid has to be decommissionned first, isn't it?.

Closing the PSMILE

Name :

PSMILe_close()

Name :

PRISM_terminate(ierror)

Responsible: NEC-CCRLE

Description :

This function will close the PSMILe. This should trigger the closure of all communications, files and save in a restart files all variables for which the processing is not finished.

A flow diagram for the PSMILe_close function

Arguments :

>WP3a: OK; We had PRISM_Finalize(ierror). "Finalize" evokes message passing termination, "close"  evokes I/O termination. What about something neutral, like "finish" or "terminate". We propose:

Name :

PRISM_finish() or PRISM_terminate()

Responsible: NEC-CCRLE

IPSL : Yes good idea. We clearly prefer the ''terminate'' name over ''finalize''

>WP3A: Do we finally need that next one???

Name :
PRISM_terminated()
Responsible: NEC-CCRLE/SGI

Description:

This function checks if PRISM_terminate has been called before.

>wp3a: Additional notes

We have now identified three functions that contain global operations over the whole coupled system:

PRISM_init
PRISM_terminate
PRISM_enddef

PRISM_enddef terminates the initialisation phase. All communicators are built. The interpolation matrices based on complete grid information of all participating components are built. Since this phase requires some global operations all processes have to participate. PRISM_enddef has to have to ability to wait for information provided by remote applications and the coupler. Multiple calls to PRISM_enddef before any PRISM_decommission (see below) will not be allowed.

The wrapper example would then look like

program wrapper

  call PRISM_init ( ierror )
!
!   At this point access to SCC information through a PRISM_get_persist is allowed
!
  call init_a ! which calls PRISM_initcomp( 'a', compid, ierror )
  call init_b ! which calls PRISM_initcomp( 'b', compid, ierror )

  call PRISM_enddef

  do i=1,endloop
    call main_a
    call main_b
  enddo

  call final_a
  call final_b

  call PRISM_terminate

end program wrapper

The grid, mask, areas definition  and field definition routines will be called in the init_a and init_b parts. The grid Ids and other Ids will have to be global variables defined in common blocks or f90 modules shared with other parts of the component code (main_a and main_b).

If a grid or a mask changes within one component during the run, the component will have to call a PRISM_decommision and then define a new grid or a new mask. An additional call to PRISM_enddef will then work on the new grid or mask and announce this new grid or mask to the coupler (in case of multi executables).