Spark

From Chorke Wiki
Revision as of 04:38, 25 September 2022 by Shahed (talk | contribs)
Jump to navigation Jump to search

Spark Master

sudo tee -a /etc/systemd/system/spark-master.service >/dev/null <<EOF
[Unit]
Description=Apache Spark Master
After=network-online.target
After=systemd-user-sessions.service

[Service]
User=ubuntu
Type=forking
EnvironmentFile=${SPARK_HOME}/etc/.env
ExecStart=${SPARK_HOME}/sbin/start-master.sh
ExecStop=${SPARK_HOME}/sbin/stop-master.sh

[Install]
WantedBy=multi-user.target
EOF

Slave Daemon

sudo tee -a /etc/systemd/system/spark-slave.service >/dev/null <<EOF
[Unit]
Description=Apache Spark Slave
After=network.target

[Service]
User=spark-user
WorkingDirectory=/usr/lib/spark/sbin
ExecStart=/usr/lib/spark/sbin/start-slave.sh spark://10.20.22.10:7077
Restart=on-failure
RestartSec=10s

[Install]
WantedBy=multi-user.target
EOF

References

Retrieved from "https://cdn.chorke.org/weke/index.php?title=Spark&oldid=8386"