matlab.mapreduce.DeployHadoopMapReducer - Configure a MapReduce application for deployment against Hadoop - MATLAB (original) (raw)

Namespace: matlab.mapreduce

Configure a MapReduce application for deployment against Hadoop

Description

A DeployHadoopMapReducer object represents executing MapReduce on a Hadoop® cluster with MATLAB® Runtime.

Construction

`config` = matlab.mapreduce.DeployHadoopMapReducer creates a matlab.mapreduce.DeployHadoopMapReducer object that specifies the default properties for Hadoop execution.

Use the resulting object as input to the mapreducer function to specify the configuration properties for Hadoop execution. For deploying a standalone application, pass thematlab.mapreduce.DeployHadoopMapReducer object as input tomapreduce.

`config` = matlab.mapreduce.DeployHadoopMapReducer([Name,Value](#namevaluepairarguments)) creates a matlab.mapreduce.DeployHadoopMapReducer object with properties specified by one or more name-value pair arguments.

Name-Value Arguments

expand all

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Path to Hadoop installation, specified as the comma-separated pair consisting of theHadoopInstallFolder and a character vector or a string scalar.

The default value of Hadoop install folder is specified by the environment variables in the order of precedence of MATLAB_HADOOP_INSTALL, HADOOP_PREFIX, and HADOOP_HOME.

Path to Hadoop application configuration files, specified as the comma-separated pair consisting of theHadoopConfigurationFile and a character vector or a string scalar.

MATLAB Runtime installation folder for Hadoop cluster, specified as the comma-separated pair consisting of the MCRRoot and a character vector or a string scalar.

MCRRoot specifies the MATLAB Runtime installation folder used by Hadoop when executingmapreduce tasks in Hadoop.

Example: 'MCRRoot','/hd-shared/hadoop-2.2.0/MCR/v84'

A containers.Map object of name-value pairs that specify Hadoop configuration properties for a specific job or application. Name-value pairs must be specified as character vectors.

The properties specified in the containers.Map object are passed as a [GENERIC_OPTION] consisting of name-value pairs signaled by a -D flag to thehadoop shell command.

Example:

setenv('HADOOP_PREFIX', '/usr/lib/hadoop') % replace with your Hadoop install location name = {'mapreduce.map.maxattempts','mapreduce.job.reduces'}; value = {'2','1'}; prop = containers.Map(name,value); obj = matlab.mapreduce.DeployHadoopMapReducer('HadoopProperties', prop)

Examples

collapse all

Create and use a matlab.mapreduce.DeployHadoopMapReducer object to deploy into a standalone application, and deploy against Hadoop.

config = matlab.mapreduce.DeployHadoopMapReducer('MCRRoot',... '/hd-shared/hadoop-2.2.0/MCR/v84'); mr = mapreducer(config);