How to export results of Spark in Zeppelin

zeppelin
spark2
multivac-dsl
(Noe Gaumont) #1

Hi,

I’m using zepplin and the viz are very nice but I would like to try other viz locally.
Is there a way to download the result locally?

Noé

(Maziyar Panahi) #2

Hi Noe,

There are two ways for exporting the results of your Spark application:

  1. (Only works in Apache Zeppelin):
    If you have your data in DataFrame/Hive format you can simply display them by SparkSQL feature in Zeppelin that gives you visualisations and export options.
    image.png2792x868 274 KB

NOTE: You can use SparkSQL (livy.sql) on both Hive tables and your DataFrames variables.

  1. (Works in both Zeppelin and Spark-shell/Spark-submit):
    Simply save your results inside your HDFS home directory. For instance, to save a JSON results you can use spark native write function:

MyData.write.json("/user/YOUR_USERNAME/TARGET_DIRECTORY")

Here is the link for more details/options:
https://spark.apache.org/docs/latest/sql-programming-guide.html#generic-loadsave-functions

If your data is RDD, you can use saveAsTextFile:
rdd.saveAsTextFile(outputDirectory);

Let me know if you have any question,