Clash Royale CLAN TAG#URR8PPP

Examples of MapReduce where the output function depends on all input files

I am doing research on MapReduce (specifically on communication between machines jointly executing a MapReduce job) and I am looking for examples where all input files are required to be processed in order to accurately evaluate the output function.

Consider the word counting example: We have a book and some computing machines and we are interested in counting the number of occurrences of each distinct word that appears in the book. We split the book into chapters and each Mapper (machine) will count the occurrences of each word in its assigned chapter. Also based on some prearranged Reduce function allocation, we will partition the words of the book (e.g. based on initial letter) and each machine will have to count the occurrences of all words in its assigned partition. To that end, the Mappers will also map the words into their partition and send them to the appropriate Reducer. The Reducers will do the final counting and output the result.

Now, the above is a very trivial example of the MapReduce framework. Another example is sorting of large data sets of keys e.g. TeraSort. In the word counting example we have to go over all chapters to compute the correct result, we cannot skip some chapter. What are other real-world examples of MapReduce that require us to process all input files? It will be appreciated if you can cite the names of the algorithms and/or provide links.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.