matlab.mapreduce.DeployHadoopMapReducer - Configure a MapReduce application for deployment against Hadoop - MATLAB (original) (raw)
Namespace: matlab.mapreduce
Configure a MapReduce application for deployment against Hadoop
Description
A DeployHadoopMapReducer
object represents executing MapReduce on a Hadoop® cluster with MATLAB® Runtime.
Construction
`config` = matlab.mapreduce.DeployHadoopMapReducer
creates a matlab.mapreduce.DeployHadoopMapReducer
object that specifies the default properties for Hadoop execution.
Use the resulting object as input to the mapreducer function to specify the configuration properties for Hadoop execution. For deploying a standalone application, pass thematlab.mapreduce.DeployHadoopMapReducer
object as input tomapreduce.
`config` = matlab.mapreduce.DeployHadoopMapReducer([Name,Value](#namevaluepairarguments))
creates a matlab.mapreduce.DeployHadoopMapReducer
object with properties specified by one or more name-value pair arguments.
Name-Value Arguments
Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN
, where Name
is the argument name and Value
is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.
Before R2021a, use commas to separate each name and value, and enclose Name
in quotes.
Path to Hadoop installation, specified as the comma-separated pair consisting of theHadoopInstallFolder
and a character vector or a string scalar.
The default value of Hadoop install folder is specified by the environment variables in the order of precedence of MATLAB_HADOOP_INSTALL
, HADOOP_PREFIX
, and HADOOP_HOME
.
Path to Hadoop application configuration files, specified as the comma-separated pair consisting of theHadoopConfigurationFile
and a character vector or a string scalar.
MATLAB Runtime installation folder for Hadoop cluster, specified as the comma-separated pair consisting of the MCRRoot
and a character vector or a string scalar.
MCRRoot specifies the MATLAB Runtime installation folder used by Hadoop when executingmapreduce
tasks in Hadoop.
Example: 'MCRRoot','/hd-shared/hadoop-2.2.0/MCR/v84'
A containers.Map
object of name-value pairs that specify Hadoop configuration properties for a specific job or application. Name-value pairs must be specified as character vectors.
The properties specified in the containers.Map
object are passed as a [GENERIC_OPTION] consisting of name-value pairs signaled by a -D
flag to thehadoop
shell command.
Example:
setenv('HADOOP_PREFIX', '/usr/lib/hadoop') % replace with your Hadoop install location name = {'mapreduce.map.maxattempts','mapreduce.job.reduces'}; value = {'2','1'}; prop = containers.Map(name,value); obj = matlab.mapreduce.DeployHadoopMapReducer('HadoopProperties', prop)
Examples
Create and use a matlab.mapreduce.DeployHadoopMapReducer
object to deploy into a standalone application, and deploy against Hadoop.
config = matlab.mapreduce.DeployHadoopMapReducer('MCRRoot',... '/hd-shared/hadoop-2.2.0/MCR/v84'); mr = mapreducer(config);