![]() Spark-submit -class ch. -master yarn-cluster spark-maven-skeleton-0.0.1-SNAPSHOT-jar-with-dependencies.jar arg1 arg2. To submit your jar, bring it inside a node of the cluster (for example with scp, rsync or AFS), login a shell in the cluster, and use the "spark-submit" command like this: You will find the Jar called "spark-maven-skeleton-0.0.1-SNAPSHOT-jar-with-dependencies.jar" inside the "target" folder. ![]() Right click on the project root directory-> -> -> "clean package" in the Goals text box -> Click on and wait for the compilation to finish (just the first time you compile it will take some minute because it will download every dependencies). When you finish to modify the main function, you are ready to build the jar. Now you can modify the main function inside the SkeletonExample.scala file This may require some updates to be downloaded (if so, wait for the popup asking to reboot). Wait for the installation to finish and reboot Eclipse when done.Īt this link in the git repository you can find a Maven skeleton project to import as "Existing maven project", how:Ĭlick on -> -> -> select the directory (with the button) in which there is the pom.xml file -> click. Wait for the elements to appear in the bo and select all of them and continue with the installation until you click "Finish". Insert as Name and Location "Scala" and this: Install the Scala plugin going on -> -> click the button, Please download Eclipse IDE for Java EE Developers, the right version for your operative system, and install it (unzip it, in most cases).Ĭonnect Eclipse to you JDK installation: click on -> -> -> -> click button and select the folder in which you unzipped JDK -> select it and click OK. ![]() Then is suggested to install Eclipse EE Juno or Kepler (that have maven already included). Eclipse for Scala, you can download it here Python installed Getting Z3 jar file Clone the Z3 git repository or download it Go in the z3 folder and run the command python scripts/mkmake. Install any JDK release (suggested 1.7.*) at this link, and install it (unzip it, in most cases).Ĭurrent Scala version in the cluster is 2.10.4 (last update of this information 02/Feb/2015) which is not yet supported by the last Eclipse release (Luna). This is a step-by-step guide in which I will try not to overlook something, but if you are having some problem or something is not clear, please do not esitate to contact me: Steps (overview) How to create a Spark Scala project and build a Jar
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |