Deploy cluster on docker, with 3 clusters #22228

paulm17 · 2024-03-08T16:33:14Z

paulm17
Mar 8, 2024

I'm able to deploy 1 cluster with docker on a mac. The problem is that vlan is broken with docker compose. No sign of it being fixed.

So I'm thinking, set zookeeper to 3181 right? Wrong.

docker run --net=pulsar \
    --name initialize-pulsar-cluster-metadata \
    apachepulsar/pulsar-all:latest bash -c "bin/pulsar initialize-cluster-metadata \
--cluster cluster-a \
--zookeeper zookeeper:3181 \
--configuration-store zookeeper:3181 \
--web-service-url http://broker:8080 \
--broker-service-url pulsar://broker:6650"

This doesn't seem to work. It only works for 2181...

Answered by paulm17

Mar 12, 2024

# docker setup
server 1 - west
server 2 - cent
server 3 - east

# run on broker west:
bin/pulsar-admin clusters create --broker-url pulsar://broker-east:6652 --url http://broker-east:8082 east
bin/pulsar-admin clusters create --broker-url pulsar://broker-cent:6651 --url http://broker-cent:8081 cent

# run on broker cent:
bin/pulsar-admin clusters create --broker-url pulsar://broker-west:6650 --url http://broker-west:8080 west
bin/pulsar-admin clusters create --broker-url pulsar://broker-east:6652 --url http://broker-east:8082 east

# run on broker east:
bin/pulsar-admin clusters create --broker-url pulsar://broker-cent:6651 --url http://broker-cent:8081 cent
bin/pulsar-admin clusters crea…

View full answer

paulm17 · 2024-03-09T11:06:14Z

paulm17
Mar 9, 2024
Author

Using the docker run, changing the port to 3181:

paul@Pauls-MacBook-Pro cluster % docker run -d -p 3181:3181 --net=pulsar \
    -e metadataStoreUrl=zk:zookeeper:3181 \
    -e cluster-name=cluster-a -e managedLedgerDefaultEnsembleSize=1 \
    -e managedLedgerDefaultWriteQuorum=1 \
    -e managedLedgerDefaultAckQuorum=1 \
    -v $(pwd)/data/zookeeper:/pulsar/data/zookeeper \
    --name zookeeper --hostname zookeeper \
    apachepulsar/pulsar-all:latest \
    bash -c "bin/apply-config-from-env.py conf/zookeeper.conf && bin/generate-zookeeper-config.sh conf/zookeeper.conf && exec bin/pulsar zookeeper"

More info when creating the metadata:

paul@Pauls-MacBook-Pro cluster % docker run --net=pulsar \
    --name initialize-pulsar-cluster-metadata \
    apachepulsar/pulsar-all:latest bash -c "bin/pulsar initialize-cluster-metadata \
--cluster cluster-a \
--zookeeper zookeeper:3181 \
--configuration-store zookeeper:3181 \
--web-service-url http://broker:8080 \
--broker-service-url pulsar://broker:6650"
2024-03-09T11:04:13,003+0000 [main] INFO  org.apache.pulsar.PulsarClusterMetadataSetup - Setting up cluster cluster-a with metadata-store=zk:zookeeper:3181 configuration-metadata-store=zookeeper:3181
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.9.1-40487256d9b9f274484798758699e49c26d91cda, built on 2023-10-02 15:06 UTC
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=c5f02b4b280d
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=17.0.9
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Eclipse Adoptium
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=/usr/lib/jvm/temurin-17-jdk-arm64
2024-03-09T11:04:13,033+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/pulsar/conf:::/pulsar/lib/net.jodah-failsafe-2.4.4.jar:/pulsar/lib/io.prometheus-simpleclient_log4j2-0.16.0.jar:/pulsar/lib/org.apache.bookkeeper.stats-prometheus-metrics-provider-4.16.3.jar:/pulsar/lib/io.grpc-grpc-servlet-jakarta-1.55.3.jar:/pulsar/lib/org.jetbrains.kotlin-kotlin-stdlib-jdk8-1.8.20.jar:/pulsar/lib/io.grpc-grpc-grpclb-1.55.3.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-server-4.16.3.jar:/pulsar/lib/org.eclipse.jetty.websocket-javax-websocket-client-impl-9.4.53.v20231009.jar:/pulsar/lib/io.opencensus-opencensus-api-0.28.0.jar:/pulsar/lib/org.apache.bookkeeper.http-vertx-http-server-4.16.3.jar:/pulsar/lib/com.google.protobuf-protobuf-java-3.19.6.jar:/pulsar/lib/org.eclipse.jetty-jetty-http-9.4.53.v20231009.jar:/pulsar/lib/org.glassfish.jersey.media-jersey-media-json-jackson-2.34.jar:/pulsar/lib/org.apache.avro-avro-protobuf-1.11.3.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-auth-sasl-3.2.0.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-proto-4.16.3.jar:/pulsar/lib/com.yahoo.datasketches-sketches-core-0.8.3.jar:/pulsar/lib/javax.validation-validation-api-1.1.0.Final.jar:/pulsar/lib/io.prometheus-simpleclient_tracer_otel-0.16.0.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final-osx-aarch_64.jar:/pulsar/lib/org.eclipse.jetty-jetty-server-9.4.53.v20231009.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final.jar:/pulsar/lib/javax.annotation-javax.annotation-api-1.3.2.jar:/pulsar/lib/org.glassfish.hk2-osgi-resource-locator-1.0.3.jar:/pulsar/lib/io.vertx-vertx-auth-common-4.3.8.jar:/pulsar/lib/org.apache.logging.log4j-log4j-api-2.18.0.jar:/pulsar/lib/io.netty-netty-codec-http2-4.1.104.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-testclient-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-proto-3.2.0.jar:/pulsar/lib/com.google.errorprone-error_prone_annotations-2.5.1.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-core-2.14.2.jar:/pulsar/lib/io.grpc-grpc-protobuf-lite-1.55.3.jar:/pulsar/lib/com.google.guava-listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-common-2.34.jar:/pulsar/lib/io.netty-netty-codec-http-4.1.104.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-package-core-3.2.0.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-common-9.4.53.v20231009.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-service-api-4.16.3.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-databind-2.14.2.jar:/pulsar/lib/org.eclipse.jetty-jetty-util-9.4.53.v20231009.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-common-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-transaction-common-3.2.0.jar:/pulsar/lib/org.apache.curator-curator-client-5.1.0.jar:/pulsar/lib/com.google.auth-google-auth-library-credentials-1.4.0.jar:/pulsar/lib/io.netty-netty-codec-haproxy-4.1.104.Final.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-api-4.16.3.jar:/pulsar/lib/io.netty-netty-handler-4.1.104.Final.jar:/pulsar/lib/io.kubernetes-client-java-api-18.0.0.jar:/pulsar/lib/io.grpc-grpc-services-1.55.3.jar:/pulsar/lib/org.apache.logging.log4j-log4j-slf4j-impl-2.18.0.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-server-9.4.53.v20231009.jar:/pulsar/lib/org.checkerframework-checker-qual-3.33.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-instance-3.2.0.jar:/pulsar/lib/com.google.re2j-re2j-1.6.jar:/pulsar/lib/org.asynchttpclient-async-http-client-2.12.1.jar:/pulsar/lib/io.netty-netty-codec-4.1.104.Final.jar:/pulsar/lib/com.fasterxml.jackson.module-jackson-module-jaxb-annotations-2.14.2.jar:/pulsar/lib/io.prometheus-simpleclient_common-0.16.0.jar:/pulsar/lib/io.grpc-grpc-rls-1.55.3.jar:/pulsar/lib/io.vertx-vertx-grpc-4.3.5.jar:/pulsar/lib/org.apache.pulsar-pulsar-metadata-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-messagecrypto-bc-3.2.0.jar:/pulsar/lib/org.rocksdb-rocksdbjni-7.9.2.jar:/pulsar/lib/org.jctools-jctools-core-2.1.2.jar:/pulsar/lib/io.dropwizard.metrics-metrics-graphite-4.1.12.1.jar:/pulsar/lib/io.prometheus-simpleclient_servlet_common-0.16.0.jar:/pulsar/lib/io.prometheus-simpleclient_caffeine-0.16.0.jar:/pulsar/lib/org.glassfish.hk2-hk2-utils-2.6.1.jar:/pulsar/lib/org.glassfish.jersey.containers-jersey-container-servlet-2.34.jar:/pulsar/lib/commons-configuration-commons-configuration-1.10.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-worker-3.2.0.jar:/pulsar/lib/org.glassfish.hk2-hk2-api-2.6.1.jar:/pulsar/lib/com.yahoo.datasketches-memory-0.8.3.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-3.9.1.jar:/pulsar/lib/io.etcd-jetcd-core-0.7.5.jar:/pulsar/lib/org.apache.logging.log4j-log4j-core-2.18.0.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-client-2.34.jar:/pulsar/lib/jakarta.activation-jakarta.activation-api-1.2.2.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-admin-api-3.2.0.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-common-4.16.3.jar:/pulsar/lib/com.beust-jcommander-1.82.jar:/pulsar/lib/io.prometheus-simpleclient-0.16.0.jar:/pulsar/lib/com.auth0-java-jwt-4.3.0.jar:/pulsar/lib/org.apache.yetus-audience-annotations-0.12.0.jar:/pulsar/lib/io.netty-netty-resolver-dns-native-macos-4.1.104.Final-osx-aarch_64.jar:/pulsar/lib/io.gsonfire-gson-fire-1.8.5.jar:/pulsar/lib/org.eclipse.jetty-jetty-alpn-server-9.4.53.v20231009.jar:/pulsar/lib/com.fasterxml.jackson.datatype-jackson-datatype-jsr310-2.14.2.jar:/pulsar/lib/commons-logging-commons-logging-1.1.1.jar:/pulsar/lib/io.prometheus-simpleclient_tracer_common-0.16.0.jar:/pulsar/lib/org.reactivestreams-reactive-streams-1.0.3.jar:/pulsar/lib/io.netty-netty-transport-4.1.104.Final.jar:/pulsar/lib/org.glassfish.jersey.inject-jersey-hk2-2.34.jar:/pulsar/lib/io.prometheus-simpleclient_servlet-0.16.0.jar:/pulsar/lib/io.prometheus-simpleclient_tracer_otel_agent-0.16.0.jar:/pulsar/lib/org.apache.httpcomponents-httpcore-4.4.15.jar:/pulsar/lib/org.apache.httpcomponents-httpclient-4.5.13.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-java-client-4.16.3.jar:/pulsar/lib/org.eclipse.jetty-jetty-servlets-9.4.53.v20231009.jar:/pulsar/lib/jakarta.ws.rs-jakarta.ws.rs-api-2.1.6.jar:/pulsar/lib/com.squareup.okhttp3-okhttp-4.9.3.jar:/pulsar/lib/org.glassfish.jersey.media-jersey-media-multipart-2.34.jar:/pulsar/lib/io.netty-netty-handler-proxy-4.1.104.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-io-core-3.2.0.jar:/pulsar/lib/org.jetbrains.kotlin-kotlin-stdlib-1.8.20.jar:/pulsar/lib/org.apache.pulsar-pulsar-config-validation-3.2.0.jar:/pulsar/lib/org.apache.bookkeeper.http-http-server-4.16.3.jar:/pulsar/lib/jakarta.validation-jakarta.validation-api-2.0.2.jar:/pulsar/lib/org.apache.bookkeeper-cpu-affinity-4.16.3.jar:/pulsar/lib/org.eclipse.jetty-jetty-alpn-conscrypt-server-9.4.53.v20231009.jar:/pulsar/lib/io.etcd-jetcd-common-0.7.5.jar:/pulsar/lib/io.netty-netty-transport-native-unix-common-4.1.104.Final-linux-x86_64.jar:/pulsar/lib/javax.websocket-javax.websocket-client-api-1.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-common-3.2.0.jar:/pulsar/lib/com.google.api.grpc-proto-google-common-protos-2.9.0.jar:/pulsar/lib/com.google.code.findbugs-jsr305-3.0.2.jar:/pulsar/lib/io.grpc-grpc-auth-1.55.3.jar:/pulsar/lib/io.reactivex.rxjava3-rxjava-3.0.1.jar:/pulsar/lib/com.google.protobuf-protobuf-java-util-3.19.6.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-original-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-auth-sasl-3.2.0.jar:/pulsar/lib/io.grpc-grpc-xds-1.55.3.jar:/pulsar/lib/javax.servlet-javax.servlet-api-3.1.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-package-filesystem-storage-3.2.0.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-slogger-api-4.16.3.jar:/pulsar/lib/io.netty.incubator-netty-incubator-transport-native-io_uring-0.0.24.Final-linux-x86_64.jar:/pulsar/lib/io.etcd-jetcd-grpc-0.7.5.jar:/pulsar/lib/org.roaringbitmap-RoaringBitmap-0.9.44.jar:/pulsar/lib/io.netty-netty-transport-classes-epoll-4.1.104.Final.jar:/pulsar/lib/com.typesafe.netty-netty-reactive-streams-2.0.6.jar:/pulsar/lib/org.apache.bookkeeper.stats-bookkeeper-stats-api-4.16.3.jar:/pulsar/lib/io.vertx-vertx-bridge-common-4.3.8.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-tools-api-3.2.0.jar:/pulsar/lib/jakarta.xml.bind-jakarta.xml.bind-api-2.3.3.jar:/pulsar/lib/com.google.code.gson-gson-2.8.9.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-java-client-base-4.16.3.jar:/pulsar/lib/io.prometheus-simpleclient_httpserver-0.16.0.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-common-allocator-4.16.3.jar:/pulsar/lib/org.conscrypt-conscrypt-openjdk-uber-2.5.2.jar:/pulsar/lib/javax.ws.rs-javax.ws.rs-api-2.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-transaction-coordinator-3.2.0.jar:/pulsar/lib/io.netty-netty-resolver-dns-native-macos-4.1.104.Final-osx-x86_64.jar:/pulsar/lib/com.fasterxml.jackson.module-jackson-module-jsonSchema-2.14.2.jar:/pulsar/lib/com.google.j2objc-j2objc-annotations-1.3.jar:/pulsar/lib/net.jodah-typetools-0.5.0.jar:/pulsar/lib/io.grpc-grpc-context-1.55.3.jar:/pulsar/lib/io.grpc-grpc-alts-1.55.3.jar:/pulsar/lib/javax.xml.bind-jaxb-api-2.3.1.jar:/pulsar/lib/com.fasterxml.jackson.module-jackson-module-parameter-names-2.14.2.jar:/pulsar/lib/com.carrotsearch-hppc-0.9.1.jar:/pulsar/lib/org.bouncycastle-bcutil-jdk18on-1.75.jar:/pulsar/lib/com.google.guava-failureaccess-1.0.1.jar:/pulsar/lib/com.github.seancfoley-ipaddress-5.3.3.jar:/pulsar/lib/org.glassfish.jersey.ext-jersey-entity-filtering-2.34.jar:/pulsar/lib/io.grpc-grpc-netty-shaded-1.55.3.jar:/pulsar/lib/org.hdrhistogram-HdrHistogram-2.1.9.jar:/pulsar/lib/jline-jline-2.14.6.jar:/pulsar/lib/io.dropwizard.metrics-metrics-core-4.1.12.1.jar:/pulsar/lib/io.netty-netty-resolver-dns-4.1.104.Final.jar:/pulsar/lib/io.grpc-grpc-api-1.55.3.jar:/pulsar/lib/org.jetbrains-annotations-13.0.jar:/pulsar/lib/org.apache.commons-commons-compress-1.21.jar:/pulsar/lib/io.etcd-jetcd-api-0.7.5.jar:/pulsar/lib/com.google.http-client-google-http-client-1.41.0.jar:/pulsar/lib/com.github.ben-manes.caffeine-caffeine-2.9.1.jar:/pulsar/lib/io.swagger-swagger-models-1.6.2.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-tools-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-docs-tools-3.2.0.jar:/pulsar/lib/io.netty.incubator-netty-incubator-transport-classes-io_uring-0.0.24.Final.jar:/pulsar/lib/org.bouncycastle-bcprov-jdk18on-1.75.jar:/pulsar/lib/org.apache.pulsar-testmocks-3.2.0.jar:/pulsar/lib/commons-cli-commons-cli-1.5.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-servlet-9.4.53.v20231009.jar:/pulsar/lib/io.vertx-vertx-web-common-4.3.8.jar:/pulsar/lib/io.prometheus-simpleclient_hotspot-0.16.0.jar:/pulsar/lib/commons-lang-commons-lang-2.6.jar:/pulsar/lib/com.google.auto.value-auto-value-annotations-1.9.jar:/pulsar/lib/io.prometheus.jmx-collector-0.16.1.jar:/pulsar/lib/io.grpc-grpc-servlet-1.55.3.jar:/pulsar/lib/org.eclipse.jetty-jetty-proxy-9.4.53.v20231009.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-jute-3.9.1.jar:/pulsar/lib/io.grpc-grpc-netty-1.55.3.jar:/pulsar/lib/org.bitbucket.b_c-jose4j-0.9.3.jar:/pulsar/lib/io.netty-netty-codec-dns-4.1.104.Final.jar:/pulsar/lib/org.jvnet.mimepull-mimepull-1.9.13.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-api-3.2.0.jar:/pulsar/lib/io.swagger-swagger-core-1.6.2.jar:/pulsar/lib/org.glassfish.hk2.external-jakarta.inject-2.6.1.jar:/pulsar/lib/com.fasterxml.jackson.jaxrs-jackson-jaxrs-base-2.14.2.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final-osx-x86_64.jar:/pulsar/lib/jakarta.annotation-jakarta.annotation-api-1.3.5.jar:/pulsar/lib/org.eclipse.jetty-jetty-util-ajax-9.4.53.v20231009.jar:/pulsar/lib/org.javassist-javassist-3.25.0-GA.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-utils-3.2.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-websocket-3.2.0.jar:/pulsar/lib/com.google.http-client-google-http-client-gson-1.41.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-cli-utils-3.2.0.jar:/pulsar/lib/com.google.auth-google-auth-library-oauth2-http-1.4.0.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-slogger-slf4j-4.16.3.jar:/pulsar/lib/org.apache.curator-curator-recipes-5.1.0.jar:/pulsar/lib/commons-codec-commons-codec-1.15.jar:/pulsar/lib/io.vertx-vertx-web-4.3.8.jar:/pulsar/lib/org.slf4j-jcl-over-slf4j-1.7.32.jar:/pulsar/lib/io.grpc-grpc-all-1.55.3.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-servlet-9.4.53.v20231009.jar:/pulsar/lib/io.airlift-aircompressor-0.20.jar:/pulsar/lib/org.glassfish.jersey.core-jersey-server-2.34.jar:/pulsar/lib/com.auth0-jwks-rsa-0.22.0.jar:/pulsar/lib/com.squareup.okhttp3-logging-interceptor-4.9.3.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-tools-framework-4.16.3.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-runtime-3.2.0.jar:/pulsar/lib/com.github.oshi-oshi-core-java11-6.4.0.jar:/pulsar/lib/com.squareup.okio-okio-3.4.0.jar:/pulsar/lib/net.java.dev.jna-jna-jpms-5.12.1.jar:/pulsar/lib/io.kubernetes-client-java-proto-18.0.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-io-9.4.53.v20231009.jar:/pulsar/lib/io.netty-netty-transport-native-epoll-4.1.104.Final-linux-x86_64.jar:/pulsar/lib/org.asynchttpclient-async-http-client-netty-utils-2.12.1.jar:/pulsar/lib/org.glassfish.jersey.containers-jersey-container-servlet-core-2.34.jar:/pulsar/lib/com.fasterxml.jackson.core-jackson-annotations-2.14.2.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-client-9.4.53.v20231009.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-local-runner-original-3.2.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-security-9.4.53.v20231009.jar:/pulsar/lib/org.bouncycastle-bcpkix-jdk18on-1.75.jar:/pulsar/lib/org.glassfish.hk2.external-aopalliance-repackaged-2.6.1.jar:/pulsar/lib/com.google.guava-guava-32.1.2-jre.jar:/pulsar/lib/io.opencensus-opencensus-proto-0.2.0.jar:/pulsar/lib/org.xerial.snappy-snappy-java-1.1.10.5.jar:/pulsar/lib/org.apache.logging.log4j-log4j-web-2.18.0.jar:/pulsar/lib/org.apache.pulsar-bouncy-castle-bc-3.2.0-pkg.jar:/pulsar/lib/io.netty-netty-codec-socks-4.1.104.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-package-bookkeeper-storage-3.2.0.jar:/pulsar/lib/io.opencensus-opencensus-contrib-http-util-0.28.0.jar:/pulsar/lib/org.eclipse.jetty-jetty-client-9.4.53.v20231009.jar:/pulsar/lib/net.jcip-jcip-annotations-1.0.jar:/pulsar/lib/io.grpc-grpc-core-1.55.3.jar:/pulsar/lib/com.fasterxml.jackson.dataformat-jackson-dataformat-yaml-2.14.2.jar:/pulsar/lib/io.netty-netty-common-4.1.104.Final.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-auth-oidc-3.2.0.jar:/pulsar/lib/io.netty-netty-buffer-4.1.104.Final.jar:/pulsar/lib/io.netty.incubator-netty-incubator-transport-native-io_uring-0.0.24.Final-linux-aarch_64.jar:/pulsar/lib/org.eclipse.jetty.websocket-websocket-api-9.4.53.v20231009.jar:/pulsar/lib/io.vertx-vertx-core-4.3.8.jar:/pulsar/lib/org.apache.bookkeeper.stats-codahale-metrics-provider-4.16.3.jar:/pulsar/lib/io.grpc-grpc-stub-1.55.3.jar:/pulsar/lib/io.netty-netty-transport-native-unix-common-4.1.104.Final.jar:/pulsar/lib/org.apache.thrift-libthrift-0.14.2.jar:/pulsar/lib/org.apache.bookkeeper-bookkeeper-common-4.16.3.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-core-4.16.3-tests.jar:/pulsar/lib/org.apache.curator-curator-framework-5.1.0.jar:/pulsar/lib/io.jsonwebtoken-jjwt-impl-0.11.1.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-core-4.16.3.jar:/pulsar/lib/io.kubernetes-client-java-18.0.0.jar:/pulsar/lib/org.slf4j-slf4j-api-1.7.32.jar:/pulsar/lib/org.apache.pulsar-pulsar-client-admin-original-3.2.0.jar:/pulsar/lib/io.prometheus-simpleclient_jetty-0.16.0.jar:/pulsar/lib/com.squareup.okio-okio-jvm-3.4.0.jar:/pulsar/lib/org.apache.pulsar-managed-ledger-3.2.0.jar:/pulsar/lib/org.jetbrains.kotlin-kotlin-stdlib-common-1.8.20.jar:/pulsar/lib/com.sun.activation-javax.activation-1.2.0.jar:/pulsar/lib/org.apache.bookkeeper-statelib-4.16.3.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final-linux-x86_64.jar:/pulsar/lib/io.grpc-grpc-protobuf-1.55.3.jar:/pulsar/lib/org.apache.commons-commons-lang3-3.11.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-proto-4.16.3.jar:/pulsar/lib/org.jetbrains.kotlin-kotlin-stdlib-jdk7-1.8.20.jar:/pulsar/lib/org.yaml-snakeyaml-2.0.jar:/pulsar/lib/commons-io-commons-io-2.8.0.jar:/pulsar/lib/org.apache.bookkeeper-circe-checksum-4.16.3.jar:/pulsar/lib/org.jline-jline-3.21.0.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-server-4.16.3.jar:/pulsar/lib/io.dropwizard.metrics-metrics-jvm-4.1.12.1.jar:/pulsar/lib/io.netty-netty-resolver-dns-classes-macos-4.1.104.Final.jar:/pulsar/lib/org.apache.commons-commons-text-1.10.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-proxy-3.2.0.jar:/pulsar/lib/io.netty-netty-tcnative-classes-2.0.61.Final.jar:/pulsar/lib/com.github.zafarkhaja-java-semver-0.9.0.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-secrets-3.2.0.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-service-impl-4.16.3.jar:/pulsar/lib/io.dropwizard.metrics-metrics-jmx-4.1.12.1.jar:/pulsar/lib/org.apache.avro-avro-1.11.3.jar:/pulsar/lib/org.eclipse.jetty-jetty-continuation-9.4.53.v20231009.jar:/pulsar/lib/io.netty-netty-resolver-4.1.104.Final.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final-windows-x86_64.jar:/pulsar/lib/io.swagger-swagger-annotations-1.6.2.jar:/pulsar/lib/org.apache.commons-commons-collections4-4.4.jar:/pulsar/lib/org.apache.distributedlog-distributedlog-protocol-4.16.3.jar:/pulsar/lib/com.fasterxml.jackson.datatype-jackson-datatype-jdk8-2.14.2.jar:/pulsar/lib/io.perfmark-perfmark-api-0.26.0.jar:/pulsar/lib/net.java.dev.jna-jna-platform-jpms-5.12.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-broker-3.2.0.jar:/pulsar/lib/org.glassfish.hk2-hk2-locator-2.6.1.jar:/pulsar/lib/org.apache.bookkeeper-native-io-4.16.3.jar:/pulsar/lib/org.apache.bookkeeper-stream-storage-common-4.16.3.jar:/pulsar/lib/org.apache.zookeeper-zookeeper-prometheus-metrics-3.9.1.jar:/pulsar/lib/io.jsonwebtoken-jjwt-api-0.11.1.jar:/pulsar/lib/com.fasterxml.jackson.jaxrs-jackson-jaxrs-json-provider-2.14.2.jar:/pulsar/lib/io.netty-netty-tcnative-boringssl-static-2.0.61.Final-linux-aarch_64.jar:/pulsar/lib/org.bouncycastle-bcprov-ext-jdk18on-1.75.jar:/pulsar/lib/io.jsonwebtoken-jjwt-jackson-0.11.1.jar:/pulsar/lib/org.apache.pulsar-pulsar-functions-api-3.2.0.jar:
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=aarch64
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.6.16-linuxkit
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=pulsar
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/pulsar
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/pulsar
2024-03-09T11:04:13,034+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=1948MB
2024-03-09T11:04:13,035+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=2048MB
2024-03-09T11:04:13,035+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=2048MB
2024-03-09T11:04:13,037+0000 [main] INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=zookeeper:3181 sessionTimeout=30000 watcher=org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase@5e8f9e2d
2024-03-09T11:04:13,039+0000 [main] INFO  org.apache.zookeeper.common.X509Util - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
2024-03-09T11:04:13,042+0000 [main] INFO  org.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 10485760 Bytes
2024-03-09T11:04:13,046+0000 [main] INFO  org.apache.zookeeper.ClientCnxn - zookeeper.request.timeout value is 0. feature enabled=false
2024-03-09T11:04:13,050+0000 [main-SendThread(zookeeper:3181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper/192.168.16.2:3181.
2024-03-09T11:04:13,050+0000 [main-SendThread(zookeeper:3181)] INFO  org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
2024-03-09T11:04:13,053+0000 [main-SendThread(zookeeper:3181)] WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server zookeeper/192.168.16.2:3181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException.
java.net.ConnectException: Connection refused
	at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
	at sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946) ~[?:?]
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344) ~[org.apache.zookeeper-zookeeper-3.9.1.jar:3.9.1]
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1274) ~[org.apache.zookeeper-zookeeper-3.9.1.jar:3.9.1]
2024-03-09T11:04:14,171+0000 [main-SendThread(zookeeper:3181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server zookeeper/192.168.16.2:3181.
2024-03-09T11:04:14,172+0000 [main-SendThread(zookeeper:3181)] INFO  org.apache.zookeeper.ClientCnxn - SASL config status: Will not attempt to authenticate using SASL (unknown error)
2024-03-09T11:04:14,174+0000 [main-SendThread(zookeeper:3181)] WARN  org.apache.zookeeper.ClientCnxn - Session 0x0 for server zookeeper/192.168.16.2:3181, Closing socket connection. Attempting reconnect except it is a SessionExpiredException.
java.net.ConnectException: Connection refused
	at sun.nio.ch.Net.pollConnect(Native Method) ~[?:?]
	at sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946) ~[?:?]
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:344) ~[org.apache.zookeeper-zookeeper-3.9.1.jar:3.9.1]
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1274) ~[org.apache.zookeeper-zookeeper-3.9.1.jar:3.9.1]

0 replies

paulm17 · 2024-03-12T16:14:15Z

paulm17
Mar 12, 2024
Author

Using the experience that I got with running pulsar with 2 clients via parallels on a mac. I was able to understand where to change the ports in the default configuration files.

Everything seems to run. Will update once the cluster is verified to be working.

My docker compose:

version: '3.1'

services:
  zk-west:
    image: apachepulsar/pulsar:latest
    container_name: zk-west
    restart: on-failure
    volumes:
      - ./data/west/zookeeper:/pulsar/data/zookeeper
    environment:
      - metadataStoreUrl=zk:zk-west:2181
      - clusterName=west
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
    command: >
      bash -c "sed -i 's/clusterName=standalone/clusterName=west/' conf/standalone.conf && \
        sed -i 's/clientPort=2181/clientPort=2181/' conf/zookeeper.conf && \
        bin/apply-config-from-env.py conf/zookeeper.conf && \
        bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
        exec bin/pulsar zookeeper"
    healthcheck:
      test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
      interval: 10s
      timeout: 5s
      retries: 30
    ports:
      - 2181:2181

  # Init cluster metadata
  pulsar-init-west:
    container_name: pulsar-init-west
    hostname: pulsar-init-west
    image: apachepulsar/pulsar:latest
    command: >
      bin/pulsar initialize-cluster-metadata \
        --cluster west \
        --zookeeper zk-west:2181 \
        --configuration-store zk-west:2181 \
        --web-service-url http://broker-west:8080 \
        --broker-service-url pulsar://broker-west:6650
    depends_on:
      zk-west:
        condition: service_healthy
  
  # Start bookie
  bookie-west:
    image: apachepulsar/pulsar:latest
    container_name: bookie-west
    restart: on-failure
    environment:
      - clusterName=west
      - zkServers=zk-west:2181
      - metadataServiceUri=metadata-store:zk:zk-west:2181
      # otherwise every time we run docker compose uo or down we fail to start due to Cookie
      # See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
      - advertisedAddress=bookie-west
      - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-west:
        condition: service_healthy
      pulsar-init-west:
        condition: service_completed_successfully
    # Map the local directory to the container to avoid bookie startup failure due to insufficient container disks.
    volumes:
      - ./data/west/bookkeeper:/pulsar/data/bookkeeper
    command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"

  # Start broker
  broker-west:
    image: apachepulsar/pulsar:latest
    container_name: broker-west
    hostname: broker-west
    restart: on-failure
    environment:
      - metadataStoreUrl=zk:zk-west:2181
      - zookeeperServers=zk-west:2181
      - clusterName=west
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - advertisedAddress=broker-west
      - advertisedListeners=external:pulsar://broker-west:6650
      - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-west:
        condition: service_healthy
      bookie-west:
        condition: service_started
    ports:
      - "6650:6650"
      - "8080:8080"
    command: bash -c "
      sed -i 's/brokerServicePort=6650/brokerServicePort=6650/' conf/broker.conf && \
      sed -i 's#webServiceUrl=http://localhost:8080/#webServiceUrl=http://localhost:8080/#' conf/client.conf && \
      sed -i 's/webServicePort=8080/webServicePort=8080/' conf/broker.conf && \
      bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"

  zk-cent:
    image: apachepulsar/pulsar:latest
    container_name: zk-cent
    restart: on-failure
    volumes:
      - ./data/central/zookeeper:/pulsar/data/zookeeper
    environment:
      - metadataStoreUrl=zk:zk-cent:2182
      - clusterName=cent
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
    command: >
      bash -c "sed -i 's/clusterName=standalone/clusterName=cent/' conf/standalone.conf && \
        sed -i 's/clientPort=2181/clientPort=2182/' conf/zookeeper.conf && \
        bin/apply-config-from-env.py conf/zookeeper.conf && \
        bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
        exec bin/pulsar zookeeper"
    healthcheck:
      test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
      interval: 10s
      timeout: 5s
      retries: 30
    ports:
      - 2182:2182

  # Init cluster metadata
  pulsar-init-central:
    container_name: pulsar-init-central
    hostname: pulsar-init-central
    image: apachepulsar/pulsar:latest
    command: >
      bin/pulsar initialize-cluster-metadata \
        --cluster cent \
        --zookeeper zk-cent:2182 \
        --configuration-store zk-cent:2182 \
        --web-service-url http://broker-cent:8081 \
        --broker-service-url pulsar://broker-cent:6651
    depends_on:
      zk-cent:
        condition: service_healthy
  
  # Start bookie
  bookie-cent:
    image: apachepulsar/pulsar:latest
    container_name: bookie-cent
    restart: on-failure
    environment:
      - clusterName=cent
      - zkServers=zk-cent:2182
      - metadataServiceUri=metadata-store:zk:zk-cent:2182
      # otherwise every time we run docker compose uo or down we fail to start due to Cookie
      # See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
      - advertisedAddress=bookie-cent
      - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-cent:
        condition: service_healthy
      pulsar-init-central:
        condition: service_completed_successfully
    # Map the local directory to the container to avoid bookie startup failure due to insufficient container disks.
    volumes:
      - ./data/central/bookkeeper:/pulsar/data/bookkeeper
    command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"

  # Start broker
  broker-cent:
    image: apachepulsar/pulsar:latest
    container_name: broker-cent
    hostname: broker-cent
    restart: on-failure
    environment:
      - metadataStoreUrl=zk:zk-cent:2182
      - zookeeperServers=zk-cent:2182
      - clusterName=cent
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - advertisedAddress=broker-cent
      - advertisedListeners=external:pulsar://broker-cent:6651
      - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-cent:
        condition: service_healthy
      bookie-cent:
        condition: service_started
    ports:
      - "6651:6651"
      - "8081:8081"
    command: bash -c "
      sed -i 's/brokerServicePort=6650/brokerServicePort=6651/' conf/broker.conf && \
      sed -i 's#webServiceUrl=http://localhost:8080/#webServiceUrl=http://localhost:8081/#' conf/client.conf && \
      sed -i 's/webServicePort=8080/webServicePort=8081/' conf/broker.conf && \
      bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"
  
  zk-east:
    image: apachepulsar/pulsar:latest
    container_name: zk-east
    restart: on-failure
    volumes:
      - ./data/east/zookeeper:/pulsar/data/zookeeper
    environment:
      - metadataStoreUrl=zk:zk-east:2183
      - clusterName=east
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
    command: >
      bash -c "sed -i 's/clusterName=standalone/clusterName=east/' conf/standalone.conf && \
        sed -i 's/clientPort=2181/clientPort=2183/' conf/zookeeper.conf && \
        bin/apply-config-from-env.py conf/zookeeper.conf && \
        bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
        exec bin/pulsar zookeeper"
    healthcheck:
      test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
      interval: 10s
      timeout: 5s
      retries: 30
    ports:
      - 2183:2183

  # Init cluster metadata
  pulsar-init-east:
    container_name: pulsar-init-east
    hostname: pulsar-init-east
    image: apachepulsar/pulsar:latest
    command: >
      bin/pulsar initialize-cluster-metadata \
        --cluster east \
        --zookeeper zk-east:2183 \
        --configuration-store zk-east:2183 \
        --web-service-url http://broker-east:8082 \
        --broker-service-url pulsar://broker-east:6652
    depends_on:
      zk-east:
        condition: service_healthy
  
  # Start bookie
  bookie-east:
    image: apachepulsar/pulsar:latest
    container_name: bookie-east
    restart: on-failure
    environment:
      - clusterName=east
      - zkServers=zk-east:2183
      - metadataServiceUri=metadata-store:zk:zk-east:2183
      # otherwise every time we run docker compose uo or down we fail to start due to Cookie
      # See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
      - advertisedAddress=bookie-east
      - BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-east:
        condition: service_healthy
      pulsar-init-east:
        condition: service_completed_successfully
    # Map the local directory to the container to avoid bookie startup failure due to insufficient container disks.
    volumes:
      - ./data/east/bookkeeper:/pulsar/data/bookkeeper
    command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"

  # Start broker
  broker-east:
    image: apachepulsar/pulsar:latest
    container_name: broker-east
    hostname: broker-east
    restart: on-failure
    environment:
      - metadataStoreUrl=zk:zk-east:2183
      - zookeeperServers=zk-east:2183
      - clusterName=east
      - managedLedgerDefaultEnsembleSize=1
      - managedLedgerDefaultWriteQuorum=1
      - managedLedgerDefaultAckQuorum=1
      - advertisedAddress=broker-east
      - advertisedListeners=external:pulsar://broker-east:6652
      - PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
    depends_on:
      zk-east:
        condition: service_healthy
      bookie-east:
        condition: service_started
    ports:
      - "6652:6652"
      - "8082:8082"
    command: bash -c "
      sed -i 's/brokerServicePort=6650/brokerServicePort=6652/' conf/broker.conf && \
      sed -i 's#webServiceUrl=http://localhost:8080/#webServiceUrl=http://localhost:8082/#' conf/client.conf && \
      sed -i 's/webServicePort=8080/webServicePort=8082/' conf/broker.conf && \
      bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"

0 replies

paulm17 · 2024-03-12T19:05:25Z

paulm17
Mar 12, 2024
Author

# docker setup
server 1 - west
server 2 - cent
server 3 - east

# run on broker west:
bin/pulsar-admin clusters create --broker-url pulsar://broker-east:6652 --url http://broker-east:8082 east
bin/pulsar-admin clusters create --broker-url pulsar://broker-cent:6651 --url http://broker-cent:8081 cent

# run on broker cent:
bin/pulsar-admin clusters create --broker-url pulsar://broker-west:6650 --url http://broker-west:8080 west
bin/pulsar-admin clusters create --broker-url pulsar://broker-east:6652 --url http://broker-east:8082 east

# run on broker east:
bin/pulsar-admin clusters create --broker-url pulsar://broker-cent:6651 --url http://broker-cent:8081 cent
bin/pulsar-admin clusters create --broker-url pulsar://broker-west:6650 --url http://broker-west:8080 west

----

# do on all brokers
bin/pulsar-admin tenants create my-tenant --allowed-clusters west,east,cent
bin/pulsar-admin namespaces create my-tenant/my-namespace --clusters west,east,cent

# set topic and replication (do on west broker only)
bin/pulsar-admin topics create persistent://my-tenant/my-namespace/my-topic

# do on all brokers
bin/pulsar-admin topics set-replication-clusters --clusters west,east,cent my-tenant/my-namespace/my-topic

Can confirm this works now! A publisher can send messages to a consumer that is connected to each cluster. 🥳

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy cluster on docker, with 3 clusters #22228

{{title}}

Replies: 3 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Deploy cluster on docker, with 3 clusters #22228

paulm17 Mar 8, 2024

Replies: 3 comments

paulm17 Mar 9, 2024 Author

paulm17 Mar 12, 2024 Author

paulm17 Mar 12, 2024 Author

paulm17
Mar 8, 2024

paulm17
Mar 9, 2024
Author

paulm17
Mar 12, 2024
Author

paulm17
Mar 12, 2024
Author