In this tutorial I will show you how to install Spark on ubuntu machine. Spark do not have particular dependency on Hadoop or other tools, but if you are planning to use Spark with Hadoop then you should follow my Part-1, Part-2 and Part-3 tutorial which covers installation of HDP platform, Hive and Sqoop.

To install Spark, first you need to ensure you have java installed. Run command “java -version” to check installed version.

If you don’t see messages like above and see message like java is not installed then you can install it with below command.

For Spark to work, Scala needs to be installed.

Once installed, type “scala” in terminal and you should see scala prompt like below.

You can then quit Scala prompt with “:q” command.

Visit below link and get latest link for pre-built version of Spark depending on your hadoop major version. For example if you are using hadoop version 2.x.x then get link for .tgz file with 2.x.x version.

https://spark.apache.org/downloads.html

Extract downloaded file,

Move extracted spark folder to /usr/lib/spark/ folder.

Set Spark environment variable in .bashrc file.

Add below lines in .bashrc file at the end.

Save and exit .bashrc file. Then reload .bashrc file

Now Spark installation is complete. To start Spark master and all other services, use below command.

Now you can access Spark WebUI using below URL.
http://localhost:7180

You should see webpage like below.

Spark installation is complete now.

To make it easier in future we should add Spark service in start.sh file.

Then add below line at the end.