Recent questions tagged Spark

0 votes

1.1k views

1 answer

spark submit add multiple jars in classpath

I am trying to run a spark program where i have multiple jar files, if I had only one jar I am not ... lib/spark/assembly/lib/hive-serde.jar. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

spark Dataframe clarification on select

I have created one dataframe ordersDF.Below is the schema. root |-- order_id: long (nullable = ... //stackoverflow.com/questions/65914467/spark-dataframe-clarification-on-select...

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

spark unit testing with SharedSparkSession (org.apache.spark.sql.test.SharedSparkSession.eventually)

I tried to write a simple unit for Spark join from the tutorial - Apache Spark Unit Testing Part 2 - ... -unit-testing-with-sharedsparksession-org-apache-spark-sql-test-sharedspar...

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.4k views

1 answer

spark ar studio - How to solve javascript "Unhandled Promise Rejection" and "Property 'p' doesn't exist" errors?

I'm working on a game using spark ar, by following the tutorial from youtube (blinking game tutorial). ... -to-solve-javascript-unhandled-promise-rejection-and-property-p-doesnt...

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

spark dataframe drop duplicates and keep first

Question: in pandas when dropping duplicates you can specify which columns to keep. Is there an equivalent in ... /questions/38687212/spark-dataframe-drop-duplicates-and-keep-first...

asked Oct 7, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

920 views

1 answer

spark streaming 集成 kafka，使用window时出现错误

当使用 spark streaming 2.0.0 集成 kafka 0.10.0时出现 KafkaConsumer 多线程争用的问题部分代码如下： val ssc = new StreamingContext(sc, ... 60S 时会发生错误当 batchDuration 为 5s, slideDuration 为 10S 时不会发生错误...

asked Feb 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

3.4k views

1 answer

spark ada - Ada complaining that I've added a volatile object in a function call to generic type when not volatile

So I've got the following declaration: record String1 : String (1 .. 64); String2 : String (1 .. 64); Timestamp : Time; ... Integer; B: Integer; C: String(1 .. 64); end record;...

asked Jan 27, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags