What is a stage in Apache Spark?
Maher Marwani
so I understand that a stage is a set of tasks that work in the same node.
so why do I get two stages when I work in local?
gorros
A stage is a set of parallel tasks - one task per partition. Number of stages is defined by number of shuffle/wide transformations. 
So coming back to your case, if you have shuffle operation then it will result in two stages.
この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。
侵害の場合は、連絡してください[email protected]
編集
関連記事
Related 関連記事
- 1
Apache SparkとApache Ignite
- 2
Apache SparkとApache Ignite
- 3
Apache SparkとApache Ignite
- 4
Apache Spark: What is the equivalent implementation of RDD.groupByKey() using RDD.aggregateByKey()?
- 5
Apache Spark対Akka
- 6
Apache Spark：NoSuchMethodError
- 7
Apache Spark Task not Serializable
- 8
Writing to a file in Apache Spark
- 9
Apache spark text similarity
- 10
Apache SparkのPrintschema（）
- 11
Apache spark with python
- 12
Apache spark dataframe
- 13
ERROR Executor: Exception in task 0.0 in stage 6.0 spark scala?
- 14
Apache Spark-reducebyKey-Java-
- 15
How to compute the mean with Apache spark?
- 16
Apache Spark：SparkPiの例
- 17
Apache Spark broadcast variables are not reused
- 18
Case class equality in Apache Spark
- 19
Dataframe transpose with pyspark in Apache Spark
- 20
Apache Spark + DeltaLakeの概念
- 21
Compatibility of Apache Spark 2.3.1 and 2.0.0
- 22
Apache Spark History Server Logs
- 23
Kickstart @ Apache spark using java
- 24
Apache Spark Python UDF Failing
- 25
What is Lineage In Spark?
- 26
Apache Storm vs Apache Samza vs Apache Spark
- 27
Apache Storm vs Apache Samza vs Apache Spark
- 28
Apache Sparkでの行列乗算
- 29
Apache Spark：マップとmapPartitions？
コメントを追加