Hadoop: Difference between revisions
Jump to navigation
Jump to search
| Line 38: | Line 38: | ||
{| | {| | ||
| valign="top" | | | valign="top" | | ||
* [https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-20-04 Hadoop » Install Standalone Mode on Ubuntu 20.04] | |||
* [https://www.vultr.com/docs/install-and-configure-apache-hadoop-on-ubuntu-20-04/ Hadoop » Install & Configure on Ubuntu 20.04] | |||
* [https://www.digitalocean.com/community/tutorials/an-introduction-to-big-data-concepts-and-terminology Hadoop » Big Data Concepts & Terminology] | |||
* [https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html Hadoop » Setting up a Single Node Cluster] | * [https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html Hadoop » Setting up a Single Node Cluster] | ||
* [https://stackoverflow.com/questions/40398280/ Hadoop » Install as a Daemon] | |||
* [https://archive.apache.org/dist/hadoop/common/ Hadoop » Download » Archive] | * [https://archive.apache.org/dist/hadoop/common/ Hadoop » Download » Archive] | ||
* [https://archive.apache.org/dist/hadoop/common/current/ Hadoop » Download » Current] | * [https://archive.apache.org/dist/hadoop/common/current/ Hadoop » Download » Current] | ||
* [https://archive.apache.org/dist/hadoop/common/stable/ Hadoop » Download » Stable] | * [https://archive.apache.org/dist/hadoop/common/stable/ Hadoop » Download » Stable] | ||
* [https://www.digitalocean.com/community/tutorials/an-introduction-to-hadoop Hadoop » An Introduction] | * [https://www.digitalocean.com/community/tutorials/an-introduction-to-hadoop Hadoop » An Introduction] | ||
* [https://hadoop.apache.org/docs/stable/ Hadoop » Docs » Stable] | |||
| valign="top" | | |||
* [https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Java+Versions Hadoop » Java Versions] | |||
* [[VS Code on iPad Pro]] | * [[VS Code on iPad Pro]] | ||
* [[Machine Learning]] | * [[Machine Learning]] | ||
| Line 50: | Line 58: | ||
| valign="top" | | | valign="top" | | ||
|- | |||
* [https:// | | colspan="3" | | ||
---- | |||
|- | |||
| valign="top" | | |||
* [https://stackoverflow.com/questions/1988249/ Bash » Switch user & execute remaining script] | |||
* [https://unix.stackexchange.com/questions/291454/ Difference between sudo user vs. root user] | * [https://unix.stackexchange.com/questions/291454/ Difference between sudo user vs. root user] | ||
* [https://jumpcloud.com/blog/how-to-create-a-new-sudo-user-manage-sudo-access-on-ubuntu-20-04 Create a Sudo User & Manage Access] | * [https://jumpcloud.com/blog/how-to-create-a-new-sudo-user-manage-sudo-access-on-ubuntu-20-04 Create a Sudo User & Manage Access] | ||
* [https:// | * [https://superuser.com/questions/468161/ Bash » Switch user & execute script] | ||
| valign="top" | | |||
| valign="top" | | | valign="top" | | ||
|} | |} | ||
Revision as of 14:40, 19 October 2022
Hadoop is a Java-based programming framework that supports the processing and storage of extremely large datasets on a cluster of inexpensive machines. It was the first major open source project in the big data playing field and is sponsored by the Apache Software Foundation. Hadoop is comprised of four main layers:
- Hadoop Common is the collection of utilities and libraries that support other Hadoop modules.
- HDFS, which stands for Hadoop Distributed File System, is responsible for persisting data to disk.
- YARN, short for Yet Another Resource Negotiator, is the "operating system" for HDFS.
- MapReduce is the original processing model for Hadoop clusters. It distributes work within the cluster or map, then organizes and reduces the results from the nodes into a response to a query. Many other processing models are available for the 3.x version of Hadoop
Knowledge
readlink -f /usr/bin/java | sed "s:bin/java::" sudo apt-get install pdsh sudo apt-get install ssh | |
|
| |
sudo apt dist-upgrade sudo do-release-upgrade sudo apt --fix-broken install sudo apt install ubuntu-desktop |
[Service]
User=hadoop
Group=hadoop
Type=forking
SuccessExitStatus=143
|