How to Build Eclipse Project:
--------------------------------------------
1. Scala + Maven + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanScalaProjectMaven
mvn clean eclipse:eclipse
mvn package
scala -cp /home/orienit/spark/workspace/KalyanScalaProjectMaven/target/KalyanScalaProjectMaven-0.0.1-SNAPSHOT.jar com.orienit.scala.training.HelloWorld
2. Scala + Sbt + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanScalaProjectSbt
sbt clean eclipse
sbt package
scala -cp /home/orienit/spark/workspace/KalyanScalaProjectSbt/target/scala-2.11/kalyanscalaprojectsbt_2.11-1.0.jar com.orienit.scala.training.HelloWorld
3. Spark + Maven + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanSparkProjectMaven
mvn clean eclipse:eclipse
mvn package
Scala Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.scala.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectMaven/target/KalyanSparkProjectMaven-0.0.1-SNAPSHOT.jar /etc/hosts /home/orienit/spark/ouput/maven-op-1
Java Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.java.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectMaven/target/KalyanSparkProjectMaven-0.0.1-SNAPSHOT.jar /etc/hosts /home/orienit/spark/ouput/maven-op-2
Python Code:
------------------
$SPARK_HOME/bin/spark-submit /home/orienit/spark/workspace/KalyanSparkProjectMaven/src/main/python/com/orienit/spark/python/training/WordCount.py /etc/hosts /home/orienit/spark/ouput/maven-op-3
4. Spark + Sbt + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanSparkProjectSbt
sbt clean eclipse
sbt package
Scala Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.scala.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectSbt/target/scala-2.11/kalyansparkprojectsbt_2.11-1.0.jar /etc/hosts /home/orienit/spark/ouput/sbt-op-1
Java Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.java.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectSbt/target/scala-2.11/kalyansparkprojectsbt_2.11-1.0.jar /etc/hosts /home/orienit/spark/ouput/sbt-op-2
Python Code:
------------------
$SPARK_HOME/bin/spark-submit /home/orienit/spark/workspace/KalyanSparkProjectSbt/src/main/python/com/orienit/spark/python/training/WordCount.py /etc/hosts /home/orienit/spark/ouput/sbt-op-3
--------------------------------------------
1. Scala + Maven + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanScalaProjectMaven
mvn clean eclipse:eclipse
mvn package
scala -cp /home/orienit/spark/workspace/KalyanScalaProjectMaven/target/KalyanScalaProjectMaven-0.0.1-SNAPSHOT.jar com.orienit.scala.training.HelloWorld
2. Scala + Sbt + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanScalaProjectSbt
sbt clean eclipse
sbt package
scala -cp /home/orienit/spark/workspace/KalyanScalaProjectSbt/target/scala-2.11/kalyanscalaprojectsbt_2.11-1.0.jar com.orienit.scala.training.HelloWorld
3. Spark + Maven + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanSparkProjectMaven
mvn clean eclipse:eclipse
mvn package
Scala Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.scala.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectMaven/target/KalyanSparkProjectMaven-0.0.1-SNAPSHOT.jar /etc/hosts /home/orienit/spark/ouput/maven-op-1
Java Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.java.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectMaven/target/KalyanSparkProjectMaven-0.0.1-SNAPSHOT.jar /etc/hosts /home/orienit/spark/ouput/maven-op-2
Python Code:
------------------
$SPARK_HOME/bin/spark-submit /home/orienit/spark/workspace/KalyanSparkProjectMaven/src/main/python/com/orienit/spark/python/training/WordCount.py /etc/hosts /home/orienit/spark/ouput/maven-op-3
4. Spark + Sbt + Eclipse project
--------------------------------------------
cd /home/orienit/spark/workspace/KalyanSparkProjectSbt
sbt clean eclipse
sbt package
Scala Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.scala.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectSbt/target/scala-2.11/kalyansparkprojectsbt_2.11-1.0.jar /etc/hosts /home/orienit/spark/ouput/sbt-op-1
Java Code:
------------------
$SPARK_HOME/bin/spark-submit --class com.orienit.spark.java.training.WordCount --master local /home/orienit/spark/workspace/KalyanSparkProjectSbt/target/scala-2.11/kalyansparkprojectsbt_2.11-1.0.jar /etc/hosts /home/orienit/spark/ouput/sbt-op-2
Python Code:
------------------
$SPARK_HOME/bin/spark-submit /home/orienit/spark/workspace/KalyanSparkProjectSbt/src/main/python/com/orienit/spark/python/training/WordCount.py /etc/hosts /home/orienit/spark/ouput/sbt-op-3
nice post. Keep sharing.
ReplyDeleteMachine Learning training in Pallikranai Chennai
Pytorch training in Pallikaranai chennai
Data science training in Pallikaranai
Python Training in Pallikaranai chennai
Deep learning with Pytorch training in Pallikaranai chennai
Bigdata training in Pallikaranai chennai
Mongodb Nosql training in Pallikaranai chennai
Spark with ML training in Pallikaranai chennai
Data science Python training in Pallikaranai
Bigdata Spark training in Pallikaranai chennai
good informatin
ReplyDeleteBig Data and Hadoop Online Training