Back to OASIS4 home

Toy gridless :
atm : 1 output field, on gridless grid (set of points) [1:1000]
ocn : 1 input field, on gridless grid
(set of points) [1:1000]

All the tags used in psmile_find_intersect and psmile_get_inetrsect are defined in psmile.inc. On this site they are here.

In the search of intersections, dest is the remote proc with which the comp on the local component has an intersection. It is the process in communicator comm_psmile.
lastag = PSMILe_Enddef_Tag =100 (Tag used to exchange data within applications.)

Here paction%n_answer=paction%n_answer2recv=0

Source :
 psmile_find_intersect : ninter, nmyint , nnull:           1           0           1
  Sending tag          100  to destination            1                                              --------------------> tells target that there is an intersection
 [1] source: psmile_find_intersect: eof ierror =  0
 [1] source: psmile_get_intersect: ninter   1, nmyint   0, nnull   1
 Laure paction%n_fin2recv :           1
  Posting Irecv request(4)            2 with tag          104
  Posting Irecv request(2)            3 with tag          100                     ------------> ninter = 1 so knows that must recieve an intersection from the target
 [1] source: psmile_get_intersect: calling waitany: n, ninter   0   1;
  n_answer, n_answer2recv   0   0; grid2receive F; nloc_recv, nreq-num_req_types+1   0   0; n_selected   0
     list of requests            1           0
     list of requests            2           3
     list of requests            3           0
     list of requests            4           2
     list of requests            5           0
 [1] source: n, ninter, sender, index           0           1           1           2                 ------------------> index : index of the request (index in the array lrequest)
psmile_enddef_action is called (except for index=3 which corresponds to the search) : performs the action required after a request/message from another process was received
 [1] source: psmile_enddef_action: index   2, sender   1
!---------------------------------------------------------------------------
!        index = 2: Message from a partner with an intersection was found (cf. routine psmile_find_intersect)
!        (1) send acknowledge (using tag reqtag)
!        (2) allocate space
!        (3) receive grid function
!---------------------------------------------------------------------------
 In psmile_enddef_action, index=2, npart :           0
and then nothing is exchange.
npart=0 because of what is coded in psmile_find_intersect.
I modified psmile_find_intersect and this is what I get :
 [1] source: psmile_recv_req_subgrid: comp_id =  1; method id =   1; sender =   1; tag = 102
 new_search =   T
 ndim_3d =             3
 [1] source: psmile_recv_req_mask: sender =   1, npart =  1, receive_mask T
 Ipart in recv_req_mask :            1
 search%search_data%range(1,1, ipart),search%search_data%range(2,1, ipart)
            1         1000
 search%search_data%range(1,2, ipart),search%search_data%range(2,2, ipart)
            1       -65535
 search%search_data%range(1,3, ipart),search%search_data%range(2,3, ipart)
            1       -65535
 len_mask :   -131071000
 psmile_error_common :  ierror   nerrp =             3            3
 [0] PSMILe: Error in component: source; File /space/coquart/oasis4_dev_buildbot/lib/psmile_oa4/src/psmile_recv_req_mask.F90: line   154
 Error in MPI routine MPI_Irecv of mask: Error message :Invalid count argument is -131071000
 Error occured in Receive from process   1 using tag  102
 [1] source: psmile_abort called from application source
I modified the calculation of the shape in different routines (psmile_def_grid and psmile_enddef_com) and now we have :
 extents intersect
 local extent :  0.100000E+01  0.100000E+04 -0.655350E+05 -0.655350E+05 -0.655350E+05 -0.655350E+05
 remote extent:  0.100000E+01  0.100000E+04 -0.655350E+05 -0.655350E+05 -0.655350E+05 -0.655350E+05
 -----------------------------------------------------
local extent :  0.100000E+01  0.100000E+04 -0.655350E+05 -0.655350E+05 -0.655350E+05 -0.655350E+05
remote extent:  0.100000E+01  0.100000E+04 -0.655350E+05 -0.655350E+05 -0.655350E+05 -0.655350E+05
intersection:  0.100000E+01  0.100000E+04 -0.655350E+05 -0.655350E+05 -0.655350E+05 -0.655350E+05
 inters :            1         1000       -65535       -65535       -65535
       -65535
 inters :            1         1000       -65535       -65535       -65535
       -65535
 Grids(grid_id)%grid_shape(1:2,:) :            1         1000       -65535
       -65535       -65535       -65535
 Number of input fields :            0
 Number of output fields :            1
 psmile_find_intersect: psmile_rank->dest, tag:            0            1
          100
 psmile_find_intersect: datatype, npart:      -280177            0
source
target
psmile_find_intersect psmile_find_intersect
sends msgint to the process (dest) with which has an intersection
call psmile_bsend (msgint, ip, MPI_INTEGER, dest, lastag, comm_psmile, ierror)
lastag = 100
sends msgint to the process (dest) with which has an intersection
call psmile_bsend (msgint, ip, MPI_INTEGER, dest, lastag, comm_psmile, ierror)
lastag = 100


psmile_get_intersect psmile_get_intersect

set up request for a grid transfer (answer to tag "lastag")    
if (paction%n_answer < paction%n_answer2recv) then
         call MPI_Irecv (paction%msgreq, nd_msgint, MPI_INTEGER,      &
                         MPI_ANY_SOURCE, reqtag, comm_psmile, paction%lrequest(1), ierror)
' Posting Irecv request(1) ', paction%lrequest(1), 'with tag ', reqtag (=101)
set up request for receive of an extra search request
     
if (paction%n_fin2recv > 0) then
         call MPI_Irecv (paction%msg_extra, msg_extra_size, MPI_INTEGER, &
                         MPI_ANY_SOURCE, exttag, comm_psmile,  paction%lrequest(4), ierror)
' Posting Irecv request(4) ', paction%lrequest(4), 'with tag ', exttag (=104)
set up request for receive of an extra search request
     
if (paction%n_fin2recv > 0) then
         call MPI_Irecv (paction%msg_extra, msg_extra_size, MPI_INTEGER, &
                         MPI_ANY_SOURCE, exttag, comm_psmile,  paction%lrequest(4), ierror)
' Posting Irecv request(4) ', paction%lrequest(4), 'with tag ', exttag (=104)
psmile_get_intersect psmile_get_intersect
set up request for receive of an intersection   
 do while ( paction%n   < paction%ninter    .or.    paction%n_answer   < paction%n_answer2recv .or. &
                 paction%nloc_recv  < paction%n_answer2recv .or.   paction%n_selected > 0   .or.     paction%grid2receive )
            call MPI_Irecv (paction%msgint, nd_msgint, MPI_INTEGER,      &
              maxval(paction%intersect_ranks), paction%lastag, comm_psmile, paction%lrequest(2),  ierror)
' Posting Irecv request(2) ', paction%lrequest(2), 'with tag ', paction%lastag (=100)
set up request for receive of an intersection   
 do while ( paction%n   < paction%ninter    .or.    paction%n_answer   < paction%n_answer2recv .or. &
                 paction%nloc_recv  < paction%n_answer2recv .or.   paction%n_selected > 0   .or.     paction%grid2receive )
            call MPI_Irecv (paction%msgint, nd_msgint, MPI_INTEGER,      &
              maxval(paction%intersect_ranks), paction%lastag, comm_psmile, paction%lrequest(2),  ierror)
' Posting Irecv request(2) ', paction%lrequest(2), 'with tag ', paction%lastag (=100)

 psmile_find_intersect : ninter, nmyint , nnull:            1            0
            1
  Sending tag           100  to destination             1
 [1] source: psmile_find_intersect: eof ierror =  0
 [1] source: psmile_get_intersect: ninter   1, nmyint   0, nnull   1
  Posting Irecv request(4)           167 with tag           104
  Posting Irecv request(2)           168 with tag           100
 [1] source: psmile_get_intersect: calling waitany: n, ninter   0   1;
  n_answer, n_answer2recv   0   0; grid2receive F; nloc_recv, nreq-num_req_types+1   0   0; n_selected   0
     list of requests             1            0
     list of requests             2          168
     list of requests             3            0
     list of requests             4          167
     list of requests             5            0

  [1] source: psmile_enddef_action: index   2, sender   1                                --------------------------------------------------> index=2
!---------------------------------------------------------------------------
!        index = 2: Message from a partner with an intersection was found (cf. routine psmile_find_intersect)
!        (1) send acknowledge (using tag reqtag)
!        (2) allocate space
!        (3) receive grid function
!---------------------------------------------------------------------------
psmile_enddef_action : index = 2 ; Message from a partner with an intersection was found
(see psmile_find_intersect of target)
psmile_enddef_action : index = 2 ; Message from a partner with an intersection was found
(see psmile_find_intersect of target)
send message back as acknowledge to sending process
           
call psmile_bsend (paction%msgint, nd_msgint, MPI_INTEGER, sender, reqtag, comm_psmile, ierror)
set up receive the subgrid sent by process sender
call psmile_recv_req_subgrid (msg_intersections, sender, grdtag, search, paction%recv_req, new_search, ierror)
nothing to do because not a source
will send the coordinates when asked by the source (request 1)
psmile_recv_req_subgrid
call psmile_recv_req_mask for gridless
call psmile_recv_req_corners_dble for conservative (+ mask)
call psmile_recv_req_coords_dble for point method (+ mask)

psmile_recv_req_mask
         call MPI_Irecv (search%search_mask(ipart)%vector, len_mask, &
                         MPI_LOGICAL, sender, tag, comm_psmile, recv_req (ndim_3d+1, ipart), ierror)
or if no mask to receive
         call MPI_Irecv (dummy_mask, 0, MPI_LOGICAL, &
                         sender, tag, comm_psmile, recv_req (ndim_3d+1, 1), ierror)
index=ndim_3d+1=3

psmile_recv_req_corners_dble for conservative
psmile_recv_req_coords_dble for point method
psmile_get_intersect
         do i = 1, paction%nreq
            print *, '    list of requests ', i, paction%lrequest(i)
         enddo



psmile_enddef_action : index = 1 ; Request for sending grid data found. Send subgrid to
partner which performs search of donor cells

call psmile_send_req_subgrid (msg_intersections, sender, grdtag, ierror)

psmile_send_req_subgrid

call psmile_send_req_mask for gridless
call psmile_send_req_corners_dble for conservative (+ mask)
call psmile_send_req_coords_dble for point method (+ mask)

psmile_enddef_action : index = 1

set up request for receive of locations
         call MPI_Irecv (paction%loc_messages(1,ind), msgloc_size, MPI_INTEGER, &
               sender, loctag+ind, comm_psmile, paction%lrequest (num_req_types-1+ind), ierror)

 [1] source: n, ninter, sender, index            0            1            1
            2
 [1] source: psmile_enddef_action: index   2, sender   1
 [1] source: psmile_recv_req_subgrid: comp_id =  1; method id =   1; sender =   1; tag = 102      -----------------------> for the gridless, only the mask is sent          tag=grdtag=102
 new_search =   T
 ndim_3d =             3
 [1] source: psmile_recv_req_mask: sender =   1, npart =  1, receive_mask F
 Gridless request for the mask :          168                                                                  
------------------------------> new request in psmile_get_intersect with index=3
 [1] source: psmile_recv_req_mask: eof ierror =   0
 [1] source: psmile_recv_req_subgrid: comp_id =  1; eof sender =  1, ierror =   0
 [1] source: psmile_enddef_action: eof ierror =  0
 [1] source: psmile_get_intersect: calling waitany: n, ninter   1   1;
  n_answer, n_answer2recv   0   0; grid2receive T; nloc_recv, nreq-num_req_types+1   0   0; n_selected   0
     list of requests             1            0
     list of requests             2            0
     list of requests             3          168
     list of requests             4          167
     list of requests             5            0
 [1] source: n, ninter, sender, index            1            1            1                                        -------------------------------------> index=3 : search is performed
            3
 [1] source: psmile_search_donor_cells: comp_id =  1; sender =   1
 [1] source: psmile_find_corr_field: comp_id =  1; sender =   1
 psmile_find_corr_field: comp_id, grid_id             1            1
 search for Taskout id             1  and Taskin Id             1
  for dest component            2
 Taskout%In_channel(  1): origin_type 3800, remote_transi_id   1, global_transi_id   1, remote_comp_id   2
 [1] source: psmile_find_corr_field: comp_id =  1; eof var_id =  1, ierror =   0
 [1] source: psmile_search_donor_gridless: comp_id =  1
 Laure inter :            1         1000       -65535       -65535       -65535
        -65535
 Laure interl:            1         1000       -65536       -65536       -65536
        -65536
 [1] source: psmile_is_mask_defined:
There is a problem because :
 Laure mask_shape :            1         1000       -65535       -65535
       -65535       -65535
 Laure inter:            1         1000       -65536       -65536       -65536
       -65536


Target :
 psmile_find_intersect: psmile_rank->dest, tag:            1            0                ---------------------------> tells the source that there is an intersection
          100
 psmile_find_intersect: datatype, npart:            0            1
 psmile_find_intersect : ninter, nmyint , nnull:            1            0
            0
 psmile_find_intersect: ipart, inter            1            1         1000
       -65535       -65535       -65535       -65535
 psmile_find_intersect: ipart, ids            1            1            1
  Sending tag           100  to destination             0
 testing comp:             2  of             2  (global comp_id             2 )
 [2] target: psmile_find_intersect: eof ierror =  0
 [2] target: psmile_get_intersect: ninter   1, nmyint   0, nnull   0
  Posting Irecv request(1)           167 with tag           101
  Posting Irecv request(4)           168 with tag           104
  Posting Irecv request(2)           169 with tag           100
 [2] target: psmile_get_intersect: calling waitany: n, ninter   0   1;
  n_answer, n_answer2recv   0   1; grid2receive F; nloc_recv, nreq-num_req_types+1   0   1; n_selected   0
     list of requests             1          167
     list of requests             2          169
     list of requests             3            0
     list of requests             4          168
     list of requests             5            0
     list of requests             6            0
 [2] target: n, ninter, sender, index            0            1            0
            2
 [2] target: psmile_enddef_action: index   2, sender   0                                   -----------------------> nothing is done for index 2
 [2] target: psmile_enddef_action: eof ierror =  0
request 2 was fullfill and nothing was done
 [2] target: psmile_get_intersect: calling waitany: n, ninter   1   1;
  n_answer, n_answer2recv   0   1; grid2receive F; nloc_recv, nreq-num_req_types+1   0   1; n_selected   0
     list of requests             1          167
     list of requests             2            0
     list of requests             3            0
     list of requests             4          168
     list of requests             5            0
     list of requests             6            0
 [2] target: n, ninter, sender, index            1            1            0         ----------------------> treat request 1 (sending the grid and/or mask)
            1
 [2] target: psmile_enddef_action: index   1, sender   0
 [2] target: psmile_send_req_subgrid: grid_id =  1; method id =   1; dest =   0
 [2] target: psmile_send_req_mask: grid_id =  1; dest =   0
 [2] target: psmile_send_req_mask: eof grid_id =  1; ierror =   0
 [2] target: psmile_send_req_subgrid: grid_id =  1; eof dest =  0, ierror =   0
  Posting Irecv request(            6 )           167 with tag           111        ------------------------------------> new request in psmile_get_intersect with index=6
  and size            15
 [2] target: psmile_enddef_action: eof ierror =  0
 [2] target: psmile_get_intersect: calling waitany: n, ninter   1   1;
  n_answer, n_answer2recv   1   1; grid2receive F; nloc_recv, nreq-num_req_types+1   0   1; n_selected   0
     list of requests             1            0
     list of requests             2            0
     list of requests             3            0
     list of requests             4          168
     list of requests             5            0
     list of requests             6          167