[ChristopherWirt]

Spark reads of Vortex files fail on any cluster whose spark.serializer is Kryo, since the 0.91.0 sync (#91).

What breaks

• The sync made VortexScan and VortexFileFormat broadcast the Hadoop conf (VortexIo.broadcast).
• That broadcast holds SerializableHadoopConf, which keeps the conf in a transient field and relies on Java's writeObject/readObject.
• Broadcasts go through spark.serializer. Kryo skips the field, so every executor gets a null conf, and every read NPEs in HadoopReadable.open → Path.getFileSystem(null).
• Found writing an event-store hour with 0.91.0-qc39: the write succeeded, but every read of the written files failed.

Fix

• Broadcast SerializableWritable<Configuration> instead. It's public Spark API, and Spark's KryoSerializer registers it to use Java serialization, so it arrives intact under both serializers.
• SerializableHadoopConf stays for the non-broadcast create path, which travels with the task and so is always Java-serialized.

Tests

• Local mode can't see the bug: tasks there reuse the driver's broadcast object without serializing it.
• New VortexIoTest case: round-trips the broadcast value through JavaSerializer and KryoSerializer, as a remote executor does.
• VortexIoTest passes on the Spark 4.0 and 3.5 builds. The read-path suites (VortexListElementPruningIoTest, VortexNestedSchema*) pass as well.
• The full Spark 4.0 suite run in one JVM exits with code 52 partway through on my machine. Unmodified develop does the same (16 failures), so it isn't this change.

Upstream has the same code, so it should go upstream too.

🤖 Generated with Claude Code