hadoop单机映射_如何在命令行中设置Hadoop的映射器和缩减器的数量？

hadoop单机映射

How to set the number of mappers and reducers of Hadoop in command line?

如何在命令行中设置Hadoop的映射器和缩减器的数量？

Number of mappers and reducers can be set like (5 mappers, 2 reducers):

可以设置映射器和缩减器的数量，例如（5个映射器，2个缩减器）：

-D mapred.map.tasks=5 -D mapred.reduce.tasks=2

in the command line.

在命令行中。

In the code, one can configure JobConf variables.

在代码中，可以配置JobConf变量。

job.setNumMapTasks(5); // 5 mappers
job.setNumReduceTasks(2); // 2 reducers

Note that on Hadoop 2 (YARN), the mapred.map.tasks and mapred.reduce.tasks are deprecated and are replaced by other variables:

请注意，在Hadoop 2（YARN）上，已弃用 mapred.map.tasks和mapred.reduce.tasks并由其他变量替换：

mapred.map.tasks     -->mapreduce.job.maps
mapred.reduce.tasks  -->	mapreduce.job.reduces

Using map reduce.job.maps on command line does not work. Is there a particular syntax to use?

在命令行上使用地图reduce.job.maps不起作用。 有没有要使用的特定语法？

You can add the options to the command like

您可以将选项添加到命令中，例如

bin/hadoop jar -Dmapreduce.job.maps=5 yourapp.jar ...

翻译自: https://www.systutorials.com/how-to-set-the-number-of-mappers-and-reducers-of-hadoop-in-command-line/

hadoop单机映射