hadoop3.3.0 编译环境搭建

最新推荐文章于 2023-11-29 10:43:35 发布

原创

最新推荐文章于 2023-11-29 10:43:35 发布 · 2.3k 阅读

标签

#hadoop #big data #hdfs

收录于

本文介绍如何在CentOS 7.9环境下搭建Hadoop 3.3.0的源码阅读环境，包括Java 1.8、Maven 3.8.4的安装及依赖配置，并详细说明了编译过程中所需的软件包和步骤。

1 概述

本文主要简单描述下搭建hadoop3.3.0源码阅读环境的搭建过程。

在搭建时可以先阅读下源码中的 building.txt文件，主要各个依赖的版本参见如下：

Requirements:

* Unix System
* JDK 1.8
* Maven 3.3 or later
* Protocol Buffers 3.7.1 (if compiling native code)
* CMake 3.1 or newer (if compiling native code)
* Zlib devel (if compiling native code)
* Cyrus SASL devel (if compiling native code)
* One of the compilers that support thread_local storage: GCC 4.8.1 or later, Visual Studio,
  Clang (community version), Clang (version for iOS 9 and later) (if compiling native code)
* openssl devel (if compiling native hadoop-pipes and to get the best HDFS encryption performance)
* Linux FUSE (Filesystem in Userspace) version 2.6 or above (if compiling fuse_dfs)
* Doxygen ( if compiling libhdfspp and generating the documents )
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)
* python (for releasedocs)
* bats (for shell code testing)
* Node.js / bow