HDP 环境下Phoenix 的安装

最新推荐文章于 2023-06-01 10:06:04 发布

原创最新推荐文章于 2023-06-01 10:06:04 发布 · 3.1k 阅读

3 ·

本内容遵循CC 4.0 BY-SA版权协议

标签

#hbase #phoenix

Phoenix 专栏收录该内容

7 篇文章

订阅专栏

本文详细介绍了在ambari2.4.2+hdp2.5.3+hbase1.1.2+Kerberos环境下Phoenix的本地源安装过程及常见问题解决方法。

一、环境

   ambari2.4.2 + hdp.2.5.3 + hbase1.1.2 + Kerberos

二、本地源安装

由于开发环境对外没有开放网络连接，因此只能通过本地源方式进行安装，通过hdp界面进行安装
操作图如下所示：

附：本地源配置可见如下链接
https://blog.csdn.net/u013850277/article/details/78897913

附：如果非hdp界面安装可以参考如下文章
https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7/bk_installing_manually_book/content/rpm-chap-phoenix.html

三、安装完成后需要重启Hbase

   在shell 命令中输入如下命令，如下所示便表示安装成功

./sqlline.py hdp04:2181   其中hdp04为zookeeper对应的server主机，多个主机通过逗号分割
[root@hdp04 bin]# ./sqlline.py hdp04:2181
Setting property: [incremental, false]
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:hdp04:2181 none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:hdp04:2181
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/phoenix/phoenix-4.7.0.2.5.3.0-37-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.5.3.0-37/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
18/03/27 17:40:35 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
18/03/27 17:40:37 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
Connected to: Phoenix (version 4.7)
Driver: PhoenixEmbeddedDriver (version 4.7)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
Building list of tables and columns for tab-completion (set fastconnect to true to skip)...
84/84 (100%) Done
Done
sqlline version 1.1.8

Phoenix 默认的客户端超时时间比较短，笔者修改成5min，运行一切正常。在hdp界面修改如下：这个属性在Hbase组件中，官方建议不好修改超过3min，但事实对上千万数据进行处理经常超过3min，可能是机器不给力吧。
这里写图片描述

四、遇到的坑

1、通过hdp安装报如下错

stderr:   /var/lib/ambari-agent/data/errors-6120.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py", line 88, in <module>
    PhoenixQueryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py", line 34, in install
    self.install_packages(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 567, in install_packages
    retry_count=agent_stack_retry_count)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 54, in action_install
    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 51, in install_package
    self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 86, in checked_call_with_retries
    return self._call_with_retries(cmd, is_checked=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 98, in _call_with_retries
    code, out = func(cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install phoenix_2_5_3_0_37' returned 1. Error: Cannot retrieve repository metadata (repomd.xml) for repository: base. Please verify its path and try again

stderr:   /var/lib/ambari-agent/data/errors-6072.txt

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py", line 88, in <module>
    PhoenixQueryServer().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py", line 34, in install
    self.install_packages(env)
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 567, in install_packages
    retry_count=agent_stack_retry_count)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 54, in action_install
    self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 51, in install_package
    self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 86, in checked_call_with_retries
    return self._call_with_retries(cmd, is_checked=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 98, in _call_with_retries
    code, out = func(cmd, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
    tries=tries, try_sleep=try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 293, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install phoenix_2_5_3_0_37' returned 1. Error: Cannot find a valid baseurl for repo: base
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'"

解决：
1、首先通过日志可以得出应该是本地源的原因导致错误产生
2、复制日志中如下命令到shell中执行如下所示：

root@hdp05 yum.repos.d]# yum -d 0 -e 0 -y install phoenix_2_5_3_0_37
Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os&infra=stock error was
14: PYCURL ERROR 6 - "Couldn't resolve host 'mirrorlist.centos.org'"
Error: Cannot find a valid baseurl for repo: base

解决如下：

[root@hdp05 yum.repos.d]# ll
total 40
-rw-r--r--  1 root root  285 Jul 10  2017 ambari.repo
-rw-r--r--. 1 root root 1991 Mar 28  2017 CentOS-Base.repo
-rw-r--r--. 1 root root  647 Mar 28  2017 CentOS-Debuginfo.repo
-rw-r--r--. 1 root root  289 Mar 28  2017 CentOS-fasttrack.repo
-rw-r--r--. 1 root root  630 Mar 28  2017 CentOS-Media.repo
-rw-r--r--. 1 root root 7989 Mar 28  2017 CentOS-Vault.repo
-rw-r--r--  1 root root   95 Jul 10  2017 HDP.repo
-rw-r--r--  1 root root  136 Jun 30  2017 HDP-UTILS.repo
-rw-r--r--  1 root root 2413 Jul 10  2017 tmp_hostcheck.result

[root@hdp05 yum.repos.d]# service httpd status
httpd (pid  27066) is running...

[root@hdp05 yum.repos.d]# mv CentOS-Base.repo CentOS-Base.repo_bak
[root@hdp05 yum.repos.d]# yum -d 0 -e 0 -y install phoenix_2_5_3_0_37
        --------- 至此安装成功-----------------------------
[root@hdp05 yum.repos.d]#