加强网站人才建设,代码编程基础知识,服装定制加盟,wordpress头像禁用之前记录的#xff1a;Flink 1.9 CDH 6.3 集成 有些下载链接可能被官方关闭了#xff0c;这里介绍1.12版本集成#xff0c;并把安装包下载地址换为百度网盘链接
下载安装包
链接: https://pan.baidu.com/s/112fiaaMAMMXMsyiTDh3qjg
提取码: ar5f 安装包内容
FLINK-1.12…之前记录的Flink 1.9 CDH 6.3 集成 有些下载链接可能被官方关闭了这里介绍1.12版本集成并把安装包下载地址换为百度网盘链接
下载安装包
链接: https://pan.baidu.com/s/112fiaaMAMMXMsyiTDh3qjg
提取码: ar5f 安装包内容
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha
FLINK_ON_YARN-1.12.1.jar
manifest.json安装
第一步
将FLINK_ON_YARN-1.12.1.jar放到 /opt/cloudera/csd目录下
第二步
在我们安装httpd服务 /var/www/html目录下创建flink目录将下面三个文件放在flink目录下
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel
FLINK-1.12.1-BIN-SCALA_2.12-el7.parcel.sha
manifest.json第三步 修改httpd服务
参考https://datamining.blog.csdn.net/article/details/121630806 新增 .parcel配置
[rootlocalhost bigdata]# vim /etc/httpd/conf/httpd.conf#在IfModule mime_module模块添加 parcel选项IfModule mime_module## TypesConfig points to the file containing the list of mappings from# filename extension to MIME-type.#TypesConfig /etc/mime.types## AddType allows you to add to or override the MIME configuration# file specified in TypesConfig for specific file types.##AddType application/x-gzip .tgz## AddEncoding allows you to have certain browsers uncompress# information on the fly. Note: Not all browsers support this.##AddEncoding x-compress .Z#AddEncoding x-gzip .gz .tgz## If the AddEncoding directives above are commented-out, then you# probably should define those extensions to indicate media types:#AddType application/x-compress .ZAddType application/x-gzip .gz .tgz .parcel #此处添加.parcel#重启http服务
[rootlocalhost bigdata]#systemctl restart httpd3.配置CDH Flink Parcel 点击配置进入新增flink链接 确认后会在parcel中出现flink如下图 PS:这里截图是之前版本的
点击下载 点击分配 点击激活 激活成功 重启服务
systemctl restart cloudera-scm-server添加服务 添加Flink 选择需要安装节点点击继续 注意这里安装一个节点就行yarn模式都是在yarn中运行 启动成功 重启Cloudera Management Service(不重启的话CDH无法监控到)
问题解决
NoClassDefFoundError: org/apache
启动flink时失败日志报错招不到Yarn的jar包 下载jar包
https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-3-uber/3.1.1.7.2.9.0-173-9.0上传到目录
/opt/cloudera/parcels/FLINK/lib/flink/libjava.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;
16:12:34.454 [main] ERROR org.apache.flink.yarn.cli.FlinkYarnSessionCli - Error while running the Flink session.
java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;at org.apache.flink.yarn.cli.FlinkYarnSessionCli.init(FlinkYarnSessionCli.java:230) ~[flink-dist_2.12-1.12.1.jar:1.12.1]at org.apache.flink.yarn.cli.FlinkYarnSessionCli.init(FlinkYarnSessionCli.java:156) ~[flink-dist_2.12-1.12.1.jar:1.12.1]at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851) [flink-dist_2.12-1.12.1.jar:1.12.1]------------------------------------------------------------The program finished with the following exception:java.lang.NoSuchMethodError: org.apache.commons.cli.Option.builder(Ljava/lang/String;)Lorg/apache/commons/cli/Option$Builder;at org.apache.flink.yarn.cli.FlinkYarnSessionCli.init(FlinkYarnSessionCli.java:230)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.init(FlinkYarnSessionCli.java:156)at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:851)
[rootr-wb-14 ~]# 下载jar包放到目录
https://mvnrepository.com/artifact/commons-cli/commons-cli/1.3.1/opt/cloudera/parcels/FLINK/lib/flink/libkerberos login configuration is invalid 原因是因为配置了kerberos上面安装包默认的配置这个要删除掉 相关资料文档
Flink1.12.3CDH6.3.1集成.docx 链接: https://pan.baidu.com/s/1QaaihgJM7QDO7yRzi4xbCg 提取码: 97n2*https://blog.csdn.net/weixin_43215250/article/details/105725330
