MapReduce - MATLAB & Simulink (original) (raw)
Main Content
Programming technique for analyzing data sets that do not fit in memory
mapreduce is a programming technique which is suitable for analyzing large data sets that otherwise cannot fit in your computer’s memory. Using a datastore to process the data in small chunks, the technique is composed of a Map phase, which formats the data or performs a precursory calculation, and a Reduce phase, which aggregates all of the results from the Map phase. For more information, see Getting Started with MapReduce.
For information about using other products withmapreduce, see Speed Up and Deploy MapReduce Using Other Products.
Functions
| mapreduce | Programming technique for analyzing data sets that do not fit in memory |
|---|---|
| datastore | Create datastore for large collections of data |
| add | Add single key-value pair to KeyValueStore |
|---|---|
| addmulti | Add multiple key-value pairs to KeyValueStore |
| hasnext | Determine if ValueIterator has one or more values available |
| getnext | Get next value from ValueIterator |
| mapreducer | Define execution environment for mapreduce or tall arrays |
|---|---|
| gcmr | Get current mapreducer configuration |
Objects
Topics
- Getting Started with MapReduce
Learn about the MapReduce programming technique and run an example calculation. - Write a Map Function
Create a map function for use in amapreducealgorithm. - Write a Reduce Function
Create a reduce function for use in amapreducealgorithm. - Speed Up and Deploy MapReduce Using Other Products
Capabilities of other products to speed up and sharemapreducealgorithms.
Troubleshooting
This example shows how to debug mapreduce algorithms in MATLAB®. Debugging enables you to follow the movement of data between the different phases of mapreduce execution and inspect the state of all intermediate variables.