groupByKey vs reduceByKey in Spark

While both of these functions will produce the same result

1. reduceByKey example works much better on a large dataset because Spark knows it can combine output with a common key on each partition before shuffling the data.

reduceByKey diagram Explain that same machine with the same key are combined before the data is shuffled. Then the function is called again to reduce all the values from each partition to produce one final result.

2. groupByKey all the key-value pairs are shuffled around. This is a lot of unnecessary data to being transferred over the network

The groupByKey diagram Explain that the same key data is shuffled to each worker node then combined the data.