Hadoop

Hadoop is a Java-based programming framework that supports the processing and storage of extremely large datasets on a cluster of inexpensive machines. It was the first major open source project in the big data playing field and is sponsored by the Apache Software Foundation. Hadoop is comprised of four main layers:

1. Hadoop Common is the collection of utilities and libraries that support other Hadoop modules.
2. HDFS, which stands for Hadoop Distributed File System, is responsible for persisting data to disk.
3. YARN, short for Yet Another Resource Negotiator, is the "operating system" for HDFS.
4. MapReduce is the original processing model for Hadoop clusters. It distributes work within the cluster or map, then organizes and reduces the results from the nodes into a response to a query. Many other processing models are available for the 3.x version of Hadoop

Knowledge

sudo apt install ssh
sudo apt install pdsh

sudo apt dist-upgrade
sudo do-release-upgrade

sudo usermod -aG sudo hadoop
readlink -f /usr/bin/java | sed "s:bin/java::"

References

Hadoop » Setting up a Single Node Cluster Hadoop » Download » Archive Hadoop » Download » Current Hadoop » Download » Stable Hadoop » An Introduction VS Code on iPad Pro Machine Learning Jupyter Spark NLP

Hadoop » Install Standalone Mode on Ubuntu 20.04 Hadoop » Install & Configure on Ubuntu 20.04 Hadoop » Big Data Concepts & Terminology Create a Sudo User & Manage Access Hadoop » Install as a Daemon