Updated December 1, 2017
In this article we will cover the obscure idea of MapReduce. MapReduce is actually a very simple concept once explained in generic terms, so let’s get started! MapReduce is all about parallel data.
Say you want to build 3 new cycles.
The first step in building these cycles is figuring out the components you will need.
- 6 Axels
- 6 wheels
- 2 handlebars
- 3 sets of cranks
- 6 Pedels
How did I find out that information? Well, simply I thought about these three cycles and broke down the needed components in my head.
But what if I have millions of bicycles, tricycles, and unicycles to build….how will I know how many components I need?
Map: Takes a set of data and converts it into another set of data. In this new set of data the individual elements(wheels, handlebars, etc…) are broken down into tuples (key/values).
Reduce: Reduce takes the converted data outputs from Map and inputs them into a reduction function. Reduce creates smaller sets of data to categorize the necessary cycle components needed to build the cycles.
MapReduce is a function(a complicated combination of instructions turned into a single line of code) applied to a node (individual part of a larger data structure), that aggregates a result for an accurate components count.
Enter the amount of each cycle, the components we need, and the number components needed for each cycle. Then you would run the function and wait for the result.
MapReduce has increased data mining speeds immensely. It is said that you can pull 1,000,000 gigs of data in less than an hour when using MapReduce!
That is a very simple breakdown of MapReduce. Get all the knowledge you need to become a data analyst with a full course from Edureka. Click here, enroll in the Data Masters course and start you journey to becoming a data analyst!
Big Data Masters Program makes you proficient in tools and systems used by Big Data experts. It includes training on Hadoop and Spark stack, Cassandra, Talend and Apache Kafka messaging system. The curriculum has been determined by extensive research on 5000+ job descriptions across the globe.