spark streaming write data to spark-sql, but cann't query it

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP



spark streaming write data to spark-sql, but cann't query it



I have submitted a streaming application, and write DataFram to spark-sql.


df.show(5)
df.write.mode(SaveMode.Append).saveAsTable(s"industy.test1")



when i send some message to kafka, the line df.show(5) have printed in the log file, that means i have successfully consumed the data, but when i use the zeppelin and the spark-sql interactive command line to query industry.test1, there is no data that i have consumed. And the log file has not any errors, but when i reopened a spark-sql interactive command line to query table, i got the data. So i want to know why i need to reopen it. Thanks for any help.





have you tried with just .saveAsTable("industy.test1")
– Vijay Krishna
Aug 6 at 20:25





@Vijay Krishna yes
– Gaurav
Aug 7 at 1:26









By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Popular posts from this blog

Firebase Auth - with Email and Password - Check user already registered

Dynamically update html content plain JS

How to determine optimal route across keyboard