Parallel programming is to speed up the calculation and is performed by the writing of a program so that it could be run on multiple processors simultaneously. Processors can have its own private memory (distributed memory) or shared memory available for each processor. In distributed memory tasks can only operate on local data and if data from others processors is required system have to provide additional communications. Of course, this implies extra time needed to finish a task. In shared memory systems processors have to ability access to single memory space used by all processors. This is very fast but can cause additional problems, such as race conditions or the difficulty of hardware implementation.
OpenMP (Open Multi-Processing) is a way to program on shared memory devices. This standard can be realized for example by splitting up the loop among different threads, but data are in one memory space.
OpenMPI is realization MPI (Message Passing Interface) that is wat to program on distributed memory devices. Parallelism can be implemented by splitting up the loop and send to each available processor, computing and receiving finished data.
I conducted a test of this two ways of parallel computing. My test case is adding two very large arrays. I assumed size array as 200 000 000. Tests have been performed on the supercomputer called Zeus (81 in the Top 500 in 2011).
Test results are not surprising. OpenMP is faster than OpenMPI. OpenMPI can be better options in computing where we provide and expect as result the small portion of data, but computation are complicated and time-consuming. I checked too as the number of processors in the OpenMPI affect to time computing. Conclusions are simple: the more processors, the calculations get faster.
The algorithm used in this tests is available on my GitHub.