Certified Associate Developer for Apache Spark — Free Practice Questions
10 free sample questions from a bank of 180, with the correct answers and explanations. No signup required — start practising right now.
1Which of the following describes the Spark driver?
The Spark driver is responsible for performing all execution in all execution modes – it is the entire Spark application.
The Spare driver is fault tolerant – if it fails, it will recover the entire Spark application.
The Spark driver is the coarsest level of the Spark execution hierarchy – it is synonymous with the Spark application.
The Spark driver is the program space in which the Spark application’s main method runs coordinating the Spark entire application.
The Spark driver is horizontally scaled to increase overall processing throughput of a Spark application.
Answer: D
The short version
D — the driver is the process where the main method runs and steers the whole application. It builds the logical plan, schedules tasks, and collects results while executors do the partitioned work.
Key concepts in this question
Driver process: the JVM process hosting SparkContext and the user main routine that launches jobs and stages.
Cluster coordination: the driver negotiates resources with the cluster manager and tracks task placement across workers.
Task scheduling: the DAG scheduler in the driver splits jobs into stages and tasks and ships closures to executors.
Result aggregation: executors return partition results to the driver, which assembles the final answer for actions.
Single point of control: only one active driver coordinates each application, so its failure normally ends the run.
Why D is correct
Option D states the driver is the program space where the main method runs while coordinating the entire application. That matches the settled definition of the driver as planner plus coordinator rather than as a data processor. It receives user code, optimizes the execution plan, works with the manager for resources, and maintains application state until completion.
Why the others are wrong
A. Executors perform distributed task execution, so the driver alone is not the entire application.
B. The driver is not fault tolerant by itself, since its loss normally forces restart or recovery from outside.
C. Application is the whole program including executors, so equating driver with the coarsest hierarchy level is imprecise.
E. Throughput scales by adding executors and partitions, not by horizontally scaling the single driver process.
Certified-Associate-Developer-for-Apache-Spark exam tip
Picture driver as brain plus main method and executors as hands, then driver failure equals application failure becomes easy to recall.
2Which of the following DataFrame operations is classified as a wide transformation?
DataFrame.filter()
DataFrame.join()
DataFrame.select()
DataFrame.drop()
DataFrame.union()
Answer: B
The short version
B — join is a wide transformation because it forces data to move across the network. Records with matching keys must meet on the same partition, so Spark shuffles before computing.
Key concepts in this question
Narrow versus wide: narrow keeps each parent partition local to one child, while wide pulls from many parents per child.
Shuffle trigger: any operation that rekeys or regroups data by value needs an exchange across executors.
Join mechanics: matching rows from two sides must be co-located by join key, which demands redistribution.
Lazy planning: transformations only extend the logical plan, with the shuffle executed when an action runs.
Cost signal: wide steps add network plus disk plus sort cost, so they dominate tuning decisions.
Why B is correct
Option B names DataFrame join as the wide transformation. A join on keys spread across partitions cannot complete locally, so Spark hashes or sorts by key and shuffles both sides to bring matches together. That cross-network movement is the textbook definition of wide, unlike row-local filters or projections.
Why the others are wrong
A. Filter keeps or drops rows within each partition, so no data moves between partitions.
C. Select projects columns row by row, which is purely local and narrow.
D. Drop removes a column locally without repartitioning or exchanging rows.
E. Union in its basic form stacks partitions without rekeying, so it avoids a shuffle here.
Certified-Associate-Developer-for-Apache-Spark exam tip
Link join plus groupBy plus distinct to shuffle in your mind, and treat filter plus select plus drop as local narrow steps.
3The code block shown below contains an error. The code block is intended to return the exact number of distinct values in column division in DataFrame storesDF. Identify the error.
Code block:
storesDF.agg(approx_count_distinct(col(“division”)).alias(“divisionDistinct”))
The approx_count_distinct() operation needs a second argument to set the rsd parameter to ensure it returns the exact number of distinct values.
There is no alias() operation for the approx_count_distinct() operation's output.
There is no way to return an exact distinct number in Spark because the data Is distributed across partitions.
The approx_count_distinct()operation is not a standalone function - it should be used as a method from a Column object.
The approx_count_distinct() operation cannot determine an exact number of distinct values in a column.
Answer: E
The short version
E — approx_count_distinct is approximate by design. It uses a sketch algorithm and can never promise an exact distinct count, so the code as written cannot meet its stated goal.
Key concepts in this question
approx_count_distinct: HyperLogLog based estimator that trades accuracy for speed and low memory on large distributed data.
countDistinct: Exact distinct counter built on shuffle and deduplication, slower but precise.
rsd parameter: Relative standard deviation control for the approximation error, lower rsd means more accuracy but still not exact.
agg with alias: Aggregation wrapper that computes an expression over a group and renames the output column for downstream use.
Exact versus approximate semantics: Exam often tests whether you know which functions guarantee correctness and which only estimate.
Why E is correct
E is correct because approx_count_distinct is documented as an approximate aggregate implemented with HyperLogLog sketches. Even with low rsd it returns an estimate within an error bound, not an exact value. Therefore no alias or parameter tweak can make that call return the exact distinct count in division, which breaks the stated intent.
Why the others are wrong
A. Tuning rsd only narrows the error band, it does not convert an estimator into an exact counter.
B. alias is valid after an aggregate expression and is not the source of the logical error here.
C. Exact distinct counts are fully possible in Spark with countDistinct or dropDuplicates followed by count, distribution does not forbid them.
D. approx_count_distinct is a standalone PySpark SQL function taking a Column, not a Column method, so this description is backwards.
Certified-Associate-Developer-for-Apache-Spark exam tip
Remember approx means approximate: if the question says exact distinct, immediately look for countDistinct, never approx_count_distinct.
4Which of the following code blocks returns the number of rows in DataFrame storesDF for each distinct combination of values in column division and column storeCategory?
C — groupBy with two string names plus count gives per-combination tallies. The distractors miss the exact API or semantic requirement detailed below for this stem.
Key concepts in this question
DataFrame transform: transforms return new frames without mutating inputs.
Column expressions: col plus functions build per-row logic.
Action trigger: actions like count take or first execute the plan.
Exam scope: associate questions test canonical API shapes rather than exotic overloads.
Why C is correct
C calls groupBy once with both string keys division and storeCategory then count, building a composite key in a single shuffle and returning one row per distinct pair with its count. The string varargs overload is the documented stable form for multi-column grouping, keeping both key columns plus count in the result. This follows standard Spark documentation for this item, so the canonical shape leads directly and reliably to the settled letter.
Why the others are wrong
A. Wrong becausestoresDF.groupBy(Seq(col(“division”), col(“storeCategory”))).count()misses the required API or semantics..
B. Wrong becausestoresDF.groupBy(division, storeCategory).count()misses the required API or semantics..
D. Wrong becausestoresDF.groupBy(“division”).groupBy(“StoreCategory”).count()misses the required API or semantics..
E. Wrong becausestoresDF.groupBy(Seq(“division”, “storeCategory”)).count()misses the required API or semantics..
Certified-Associate-Developer-for-Apache-Spark exam tip
composite grouping in one call: one groupBy with both names then count. Match the exact method name, argument kind and ordering from the docs for speed.
5The code block shown below contains an error. The code block is intended to return a collection of summary statistics for column sqft in Data Frame storesDF. Identify the error.
Code block:
storesDF.describes(col(“sgft”))
The column sqft should be subsetted from DataFrame storesDF prior to computing summary statistics on it alone.
The describe() operation does not accept a Column object as an argument outside of a sequence — the sequence Seq(col(“sqft”)) should be specified instead.
The describe()operation doesn’t compute summary statistics for a single column — the summary() operation should be used instead.
The describe()operation doesn't compute summary statistics for numeric columns — the summary() operation should be used instead.
The describe()operation does not accept a Column object as an argument — the column name string “sqft” should be specified instead.
Answer: E
The short version
E — describe takes column name strings, not Column objects. Passing col inside describes is a type error, the fix is to pass the plain name sqft.
Key concepts in this question
describe: DataFrame action that computes count, mean, stddev, min and max for string-named columns.
Column objects: Symbolic expressions built by col that are accepted by select, filter and agg but not by every DataFrame method.
summary versus describe: summary extends describe with percentiles and accepts similar string arguments, neither takes bare Column objects.
Typo sensitivity: describes with an s does not exist, and sgft versus sqft shows why exact names matter on the exam.
API signature awareness: Associate exam heavily tests which methods want strings, Columns, or lists of them.
Why E is correct
E is correct because the DataFrame describe signature is describe followed by a variable list of string column names. Supplying a Column object created by col violates that contract and raises a type error. Changing the argument to the string sqft makes the call valid and returns summary statistics, which matches the stated intent without needing subsetting or a different method.
Why the others are wrong
A. Subsetting first is unnecessary, describe can operate directly on the DataFrame when given correct string names.
B. describe does not want a Seq of Columns, it wants plain strings, so wrapping in a sequence does not fix it.
C. describe does compute summary statistics for a single named column, summary is not required for this task.
D. describe handles numeric columns including means and standard deviations, so this limitation is invented.
Certified-Associate-Developer-for-Apache-Spark exam tip
Memorize describe takes strings: if you see describe with col inside, mark it wrong and replace col with a quoted name.
6The code block shown below should extract the integer value for column sqft from the first row of DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task.
Code block:
__1__.__2__.__3__[Int](__4__)
1. storesDF
2. first()
3. getAs()
4. “sqft”
1. storesDF
2. first
3. getAs
4. sqft
1. storesDF
2. first()
3. getAs
4. col(“sqft”)
1. storesDF
2. first
3. getAs
4. “sqft”
Answer: D
The short version
D — storesDF.first.getAs with sqft extracts the value. In Scala first is accessed without parentheses and getAs is typed by the caller.
Key concepts in this question
first: Scala DataFrame accessor that returns the first Row without parentheses, unlike Python where first is a method call.
Row.getAs: Typed Row accessor that casts a named field to the requested type parameter such as Int.
Scala versus Python syntax: Parentheses rules differ by language, the exam uses this to test attention to API dialect.
Column versus name: getAs expects a field name string, not a Column expression built by col.
Chained blank filling: Recognizing object, accessor, typed getter, and name in order is the core skill tested.
Why D is correct
D is correct because the Scala path is storesDF to obtain the DataFrame, then first as a parameterless accessor to get a Row, then getAs parameterized to Int to extract a typed value, then the string sqft to name the field. Options with first with parentheses or getAs with parentheses or col around sqft mix Python style or wrong types. Only D matches the Scala Row API exactly and returns the integer value from the first row.
Why the others are wrong
A. Using first with parentheses and getAs with parentheses follows Python style, not the Scala signature expected here.
B. Omitting quotes around sqft turns the name into an unresolved identifier instead of a field lookup.
C. Passing col around sqft gives getAs a Column object when it requires a plain field name string.
Certified-Associate-Developer-for-Apache-Spark exam tip
For Scala Row extraction recall first without parens plus getAs typed plus plain name: first, getAs, quoted name.
7The code block shown below should print the schema of DataFrame storesDF. Choose the response that correctly fills in the numbered blanks within the code block to complete this task.
Code block:
__1__.__2__
1. storesDF
2. printSchema(“all”)
1. storesDF
2. schema
1. storesDF
2. getAs[str]
1. storesDF
2. printSchema(true)
1. storesDF
2. printSchema
Answer: E
The short version
E — storesDF.printSchema with no arguments prints the schema tree. The distractors miss the exact API or semantic requirement detailed below for this stem.
Key concepts in this question
Schema tree: StructType lists field names types and nullability.
Display versus return: printSchema prints while schema returns the object.
Eager action: the call executes immediately for inspection.
Exam scope: associate questions test canonical API shapes rather than exotic overloads.
Why E is correct
E resolves to storesDF.printSchema(), which walks the StructType and prints names, types and nullability to stdout with no argument needed. The schema property alone only returns the object without printing, and the vendor hint confirms the bare call. This follows standard Spark documentation for this item, so the canonical shape leads directly and reliably to the settled letter.
Why the others are wrong
A. Wrong because1. storesDF 2. printSchema(“all”)misses the required API or semantics..
B. Wrong because1. storesDF 2. schemamisses the required API or semantics..
C. Wrong because1. storesDF 2. getAs[str]misses the required API or semantics..
D. Wrong because1. storesDF 2. printSchema(true)misses the required API or semantics..
Certified-Associate-Developer-for-Apache-Spark exam tip
print means printSchema call with empty parentheses. Match the exact method name, argument kind and ordering from the docs for speed.
8The code block shown below contains an error. The code block is intended to create and register a SQL UDF named “ASSESS_PERFORMANCE” using the Scala function assessPerformance() and apply it to column customerSatisfaction in the table stores. Identify the error.
Code block:
spark.udf.register(“ASSESS_PERFORMANCE”, assessPerforance)
spark.sql(“SELECT customerSatisfaction, assessPerformance(customerSatisfaction) AS result FROM stores”)
The customerSatisfaction column cannot be called twice inside the SQL statement.
Registered UDFs cannot be applied inside of a SQL statement.
The order of the arguments to spark.udf.register() should be reversed.
The wrong SQL function is used to compute column result - it should be ASSESS_PERFORMANCE instead of assessPerformance.
There is no sql() operation - the DataFrame API must be used to apply the UDF assessPerformance().
Answer: D
The short version
D — SQL must call the registered name ASSESS_PERFORMANCE. Registering under one name but querying another means the query cannot resolve.
Key concepts in this question
spark.udf.register: Binds a Scala function object to a SQL-visible name for use inside spark.sql strings.
Registered name resolution: SQL looks up functions by the registered string, not by the original Scala identifier.
Case and spelling: SQL names are matched literally, so ASSESS_PERFORMANCE and assessPerformance are different symbols.
UDF in SQL: Once registered, a UDF is fully usable inside SELECT expressions alongside columns.
Error isolation: The registration line itself is valid, the failure appears only when the SQL text references the wrong identifier.
Why D is correct
D is correct because the code registers the Scala function under the uppercase name ASSESS_PERFORMANCE but then the SQL string calls assessPerformance. Spark SQL resolves functions by the registered name present in the catalog, so the lowercase reference is unknown and the query fails. Replacing the SQL call with ASSESS_PERFORMANCE makes registration and usage consistent and allows the UDF to be applied to customerSatisfaction as intended.
Why the others are wrong
A. A column can appear multiple times in a SELECT list, reuse of customerSatisfaction is legal.
B. Registered UDFs are explicitly designed to be callable from SQL statements after registration.
C. The order name then function in register is correct, reversing would break it.
E. sql is a valid SparkSession operation, there is no requirement to use only the DataFrame API.
Certified-Associate-Developer-for-Apache-Spark exam tip
When you see udf.register, circle the registered name and check the SQL string uses exactly that same spelling.
9The code block shown below contains an error. The code block is intended to create the Scala UDF assessPerformanceUDF() and apply it to the integer column customers1t1sfaction in Data Frame storesDF. Identify the error.
Code block:
The input type of customerSatisfaction is not specified in the udf() operation.
The return type of assessPerformanceUDF() must be specified.
The withColumn() operation is not appropriate here - UDFs should be applied by iterating over rows instead.
The assessPerformanceUDF() must first be defined as a Scala function and then converted to a UDF.
UDFs can only be applied via SQL and not through the Data Frame API.
Answer: B
The short version
B — Scala udf construction is missing its declared return type. The distractors miss the exact API or semantic requirement detailed below for this stem.
Key concepts in this question
UDF wrapper: udf turns a Python or Scala function into a Column expression.
Return type: Catalyst needs the declared output type for planning.
withColumn apply: the UDF runs per row through DataFrame transforms.
Exam scope: associate questions test canonical API shapes rather than exotic overloads.
Why B is correct
B is the defect because the Scala udf factory needs the function plus its Catalyst return DataType to plan serialization and codegen. The input can be inferred from the function signature, but without the return type withColumn cannot build a valid new column. This follows standard Spark documentation for this item, so the canonical shape leads directly and reliably to the settled letter.
Why the others are wrong
A. Wrong becauseThe input type of customerSatisfaction is not specified in the udf() operation.misses the required API or semantics..
C. Wrong becauseThe withColumn() operation is not appropriate here - UDFs should be applied by i…misses the required API or semantics..
D. Wrong becauseThe assessPerformanceUDF() must first be defined as a Scala function and then co…misses the required API or semantics..
E. Wrong becauseUDFs can only be applied via SQL and not through the Data Frame API.misses the required API or semantics..
Certified-Associate-Developer-for-Apache-Spark exam tip
Scala udf error means check return type first. Match the exact method name, argument kind and ordering from the docs for speed.
10The code block shown below should create a single-column DataFrame from Scala list years which is made up of integers. Choose the response that correctly fills in the numbered blanks within the code block to complete this task.
Code block:
__1__.__2__(__3__).__4__
1. spark
2. createDataFrame
3. years
4. IntegerType
A — spark.createDataFrame over the integer list with IntegerType builds one row per element. The distractors miss the exact API or semantic requirement detailed below for this stem.
Key concepts in this question
DataFrame transform: transforms return new frames without mutating inputs.
Column expressions: col plus functions build per-row logic.
Action trigger: actions like count take or first execute the plan.
Exam scope: associate questions test canonical API shapes rather than exotic overloads.
Why A is correct
A distributes each element of Scala list years as its own row and applies IntegerType as the single-column schema in one step. createDataset would yield a Dataset needing conversion, and rewrapping as List of years nests the collection. This follows standard Spark documentation for this item, so the canonical shape leads directly and reliably to the settled letter.
Why the others are wrong
B. Wrong because1. spark 2. createDataset 3. years 4. IntegerTypemisses the required API or semantics..
C. Wrong because1. spark 2. createDataset 3. List(years) 4. toDFmisses the required API or semantics..
D. Wrong because1. spark 2. createDataFrame 3. List(years) 4. IntegerTypemisses the required API or semantics..
Certified-Associate-Developer-for-Apache-Spark exam tip
list to frame means createDataFrame with list plus element type. Match the exact method name, argument kind and ordering from the docs for speed.