Collective communication differs from point to point communication because it uses commands that can broadcast, scatter, gather and reduce to multiple processors at once.
Broadcasting is used to broadcast the same information to multiple processors.
In the example code, processor 0 broadcasts an array of size size which is the total number of processors available.
The variable used for the data must be initialized which is why processor 0 creates the data to be shared and the rest
create a buffer for the data to go into. Once all processors call the Bcast command with the data and root as zero, the data
is broadcasted to all the processors.
If you want to scatter the information to each processor then you use the scatter command to send an element of an
array to each processor. The example code provided shows how the scatter commands scatters the data from an array
of size processors available to each processor. Each processor must have the data variable set even if it’s empty
so that the scatter command knows where to send the data to. Similar to broadcast it has data and root=0 as parameters.
The gather command is used to gather information from the processors available. This is usually done after scatter as it works, in the same way, to get back the information that was scattered. This is only done on the root which for the code example is zero. The gathered information is then printed out by rank since the other processors have not gathered the information.
Reducing allows for operations to be done on the information being gathered as its been gathered this provides various operations but standard reducing is the summation of all numbers. The example code is similar to the gather example but instead of outputting an array of the data gathered it results in the summation of all the numbers that were initially scattered.