这里主要测试的华为云服务器,但是我想所有的云服务器应该是大同小异的,其次是外网访问阶段,kafka部署后无法访问,以及在容器中更改了server.properties,重启容器还是不正确,通过查找发现服务器可以做以下修改之后就能成功使用kafka tools和代码访问kafka了 这里使用的kafka和zookeeper
docker pull wurstmeister/zookeeper docker pull wurstmeister/kafka1.进入hosts文件
vi /etc/hosts //添加公网ip 公网ip2.删除已经部署好的kafka,也可以不做删除,但重新给端口,命名就可以了
docker run -d --name kafka -p 9093:9093 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=(一定要是公网ip,否则远程是无法操作kafka的,kaka tools也是无法使用的):2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://(一定要是公网ip,否则远程是无法操作kafka的,kaka tools也是无法使用的):9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -t wurstmeister/kafka这里着重说明以下,我通过查询以下的网站,均试过了,发现只有这样改才能做到kafka的外部访问,如果访问不了大家也可以查看以下的博客试一试 https://blog.csdn.net/xiaoyu_bd/article/details/82496257 https://blog.csdn.net/chenyulancn/article/details/79499401 这个时候其实已经可以访问kafka了,但是有些人还是无法访问,究其原因是因为在云服务器的出入站规则中没有增加2181端口和个人给的kafka端口,所以无法访问的,以下是我的出入站规则 3.现在就能通过代码测试是否可以访问了,我这里创建了一个test的topic;然后还可以通过kafka tools查看是否有kafka了.
package com.project.spark.util /** * @Author ： * @Date ： Create in 2019-07-17 14:40 * @Description ：<描述> */ object Producer extends App { import java.util.Properties import org.apache.kafka.clients.producer.{KafkaProducer, ProducerRecord} val topic = "par" val brokers = "公网ip:9092" val props = new Properties() props.put("bootstrap.servers", brokers) props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer") props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer") val producer = new KafkaProducer[String, String](props) val t = System.currentTimeMillis() var msg = "启动成功" val record = new ProducerRecord[String, String](topic, "key", msg) producer.send(record) producer.close() }关于有些小伙伴在这些都做成功之后,在控制台begin topic的时候发现有以下错误
[2019-07-18 05:45:26,746] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2019-07-18 05:45:26,849] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2019-07-18 05:45:26,950] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2019-07-18 05:45:27,152] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2019-07-18 05:45:27,654] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) [2019-07-18 05:45:28,558] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)这个原因是因为你开启的时候ip没有写公网ip,所以会造成无法连接,下面是我测试的几条命令
kafka-console-consumer.sh --bootstrap-server 公网ip:给的端口 --topic par --from-beginning kafka-console-consumer.sh --bootstrap-server 公网ip:给的端口 --topic par --from-beginning