Apache Spark Quick Notes :: Part - 3


Bhaskar S 10/24/2015


Hands-on with Spark Core

We will continue our session from where we left off in Part-2.

To wrap up this part, let us summarize all the transformation functions we used thus far in this part.

The following is the summary of the RDD transformation functions we used in this part:

Tranformation Function Description
reduceByKey Works on a collection of (KEY, VALUE) pairs and applies the specified lambda function to VALUEs with the same KEY
sortByKey Works on a collection of (KEY, VALUE) pairs and sorts them by the KEY
aggregateByKey Works on a collection of (KEY, VALUE) pairs and aggregates the VALUEs for each KEY, using the given initial value, the merge function and combine function
combineByKey Works on a collection of (KEY, VALUE) pairs and is a generic function that combines the VALUEs for each KEY, using the given initial function, the merge function and combine function

References

Apache Spark Quick Notes :: Part - 1

Apache Spark Quick Notes :: Part - 2