-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Faile to connect to namenode:8020 #52
Comments
Hi! The application from the root Makefile is not for swarm. To test it, run one of the provided examples from spark distro. Also, what I can see from the execution namenode exposes port 9000 by default. Did you change the port? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I used this repo to pick up a docker swarm cluster and follow everything in swarm directory step by step and also modified the Makefile in main directory as follow ->
get-example:
if [ ! -f example/SparkWriteApplication.jar ]; then
wget -O example/SparkWriteApplication.jar https://www.dropbox.com/s/7dn0horm8ocbu0p/SparkWriteApplication.jar ;
fi
example: get-example
docker run --rm -it --network workbench --env-file ./swarm/hadoop.env -e SPARK_MASTER=spark://sparm-master:7077 --volume $(shell pwd)/example:/example bde2020/spark-base:2.2.0-hadoop2.8-hive-java8 /spark/bin/spark-submit --master spark://spark-master:7077 /example/SparkWriteApplication.jar
docker exec -it namenode hadoop fs -cat /tmp/numbers-as-text/part-00000
and when execute make example it reads the file and give me exception which tells me ->
Failed to connect to server: namenode/10.0.0.102:8020: try once and fail.
java.net.ConnectException: Connection refused
I have allowed all of the necessary ports and still cannot connect to namenode.... any suggestions ?
The text was updated successfully, but these errors were encountered: