Hello. I have a unique distributed memory (MPP) application that I'm working on, and I ran into an issue that I am hoping I can get some help with. I am working with a commercial finite element product that allows for the development of user-defined material models. I have access to an objects package from the software company, most of the files of which are already compiled. I have access to a few files with source code, and this is where I add my Fortran code for my user-defined material model. My code has been added, and I'm using the Intel Fortran compiler for Linux to build the finite element program executable to be used in MPP processing. I'm using the Platform MPI environment. A feature of my user-defined material model is a two-dimensional saved array that accumulates data upon each call to my material model subroutine. From time to time I operate on the array, reset its value to zero, and begin accumulated data again during subsequent subroutine calls. I've already implemented this successfully in an SMP processing configuration. I had to protect this accumulation array using critical regions, else the array was getting sloshed around in the shared memory. I have a similar issue with this array during the MPP processing approach. My first work-around was to for all relevant parts of my finite element model that use my user-defined material model to a single processor (the rest of the model was permitted to be distributed amongst the various processors). This approach worked, but the cost involved with forced distribution rendered the simulation time longer than my SMP approach. I am looking for some guidance/thoughts regarding a different MPP approach that protects my accumulation array and allows all processors to see it, but that doesn't require me to force processor distribution of some of my finite element model. Some internet research has turned me onto some MPI barrier/synchronization techniques, but I'm not sure. Does anyone have experience with this or perhaps can point me to some guidance?
Thank you!