Sunday, October 13, 2019

Run spark-submit

Trying to run spark-submit, but cannot see what's going on.


\apps\spark-2.4.4-bin-hadoop2.7\bin\spark-submit --class spark-tutorial.SimpleApp --master local target\scala-2.13\spark-tutorial_2.13-1.0.jar



Eventually, when I switch spark home folder to run, then I have some progress.

D:\apps\spark-2.4.4-bin-hadoop2.7>.\bin\spark-submit \dvp\spark\spark-tutorial\target\scala-2.13\spark-tutorial_2.13-1.0.jar --class spark-tutorial.SimpleApp







Next to look at what cause this exception.

1 comment:

Alex said...

In the build.sbt, just update scala version

scalaVersion := "2.11.11"

then fix the exception