What's Changed

• Replace type check by pattern match by @cutiechi in #193
• Set Jupyter version to Java 8 by @Jolanrensen in #200
• Update spark to 3.3.2 by @devongleeson in #198
• Spark yarn in Jupyter notebooks by @Jolanrensen in #201
• Version bumps by @Jolanrensen in #202

New Contributors

• @cutiechi made their first contribution in #193
• @devongleeson made their first contribution in #198

Full Changelog: 1.2.3...v1.2.4

Adds Java 8 support #189 requested by #186, #183, #190.
All modules except jupyter now target Java (1.)8 and should be compatible with Java (1.)8 servers.
Let us know if you have any issues with it!

• Added BigInteger support in #182 thanks to #181
• New Spark versions: 3.2.3, 3.3.1, 3.2.2
• New Scala versions: 2.12.17, 2.13.10
• Updated Kotlin to 1.7.20
• Small bugfix regarding Map encoding

What's Changed

• Reorganized project: Gradle by @Jolanrensen in #168.

• Reorganized project: Now supporting builds for each patch-version of Spark from 3.0.0 to 3.3.0 and Scala 2.12 / 2.13 by @Jolanrensen in #168.
  NOTE: The new naming scheme.

• Added: UDT Support for MLlib etc. by @Jolanrensen in #162. Example.

• Added: Better UD(A)F support by @Jolanrensen in #152. Example

• Added: seqOf() function by @Jolanrensen in #172.

• Added: dfOf() function by @Jolanrensen in #162.

• Added: JavaRDD extension functions by @Jolanrensen in #174. Example / Use case.

• Added: Spark properties now in use-magic in Jupyter in #176.

• Fixed: Reading files from Datalore by @Jolanrensen in #165.

• Fixed: Encoding "isSomething" names in data classes by @Jolanrensen in #171.

• Fixed: Conversion methods for Scala 2.12 / 2.13 by @Jolanrensen in #175.

• Updated wiki.

Full Changelog: https://github.com/Kotlin/kotlin-spark-api/milestone/10?closed=1.

Get it

Maven Central: https://search.maven.org/artifact/org.jetbrains.kotlinx.spark/kotlin-spark-api_3.3.0_2.13/1.2.1/jar.

For other Spark- and Scala versions, follow the naming scheme.

Read more

Blog post: Coming soon!

What's Changed

• Default master value to first check the system variables. by @Jolanrensen in #127
• More documentation by @Jolanrensen in #128, #130
• Rdd related helpers by @Jolanrensen in #132
• 3.2 Encoders, supporting scala case classes by @Jolanrensen in #134, #147
• Reorganizing API and adding conversions by @Jolanrensen in #136
• Sorting functions for key/value Datasets by @Jolanrensen in #138
• Tuple first by @Jolanrensen in #144
• Streaming support by @Jolanrensen in #133, #157
• Kotlin Jupyter notebook support by @Jolanrensen in #137, #155, #158
• Created and updated wiki

Full Changelog: kotlin-spark-3.2-1.0.3...3.2-1.1.0

Get it

Maven Central: https://search.maven.org/artifact/org.jetbrains.kotlinx.spark/kotlin-spark-api-3.2/1.1.0/jar

Read more

Blog post: https://blog.jetbrains.com/big-data-tools/2022/05/26/kotlin-api-for-apache-spark-streaming-jupyter-and-more/

What's Changed

• Default master value to first check the system variables. by @Jolanrensen in #127
• More documentation by @Jolanrensen in #128, #130
• Rdd related helpers by @Jolanrensen in #132
• 3.2 Encoders, supporting scala case classes by @Jolanrensen in #134, #147
• Reorganizing API and adding conversions by @Jolanrensen in #136
• Sorting functions for key/value Datasets by @Jolanrensen in #138
• Tuple first by @Jolanrensen in #144
• Streaming support by @Jolanrensen in #133, #157
• Kotlin Jupyter notebook support by @Jolanrensen in #137, #155, #158
• Created and updated wiki

Full Changelog: kotlin-spark-3.2-1.0.3...3.1-1.1.0

Get it

Maven Central: https://search.maven.org/artifact/org.jetbrains.kotlinx.spark/kotlin-spark-api-3.1/1.1.0/jar

Read more

Blog post: https://blog.jetbrains.com/big-data-tools/2022/05/26/kotlin-api-for-apache-spark-streaming-jupyter-and-more/

What's Changed

• Default master value to first check the system variables. by @Jolanrensen in #127
• More documentation by @Jolanrensen in #128, #130
• Rdd related helpers by @Jolanrensen in #132
• 3.2 Encoders, supporting scala case classes by @Jolanrensen in #134, #147
• Reorganizing API and adding conversions by @Jolanrensen in #136
• Sorting functions for key/value Datasets by @Jolanrensen in #138
• Tuple first by @Jolanrensen in #144
• Streaming support by @Jolanrensen in #133, #157
• Kotlin Jupyter notebook support by @Jolanrensen in #137, #155, #158
• Created and updated wiki

Full Changelog: kotlin-spark-3.2-1.0.3...3.0-1.1.0

Get it

Maven Central: https://search.maven.org/artifact/org.jetbrains.kotlinx.spark/kotlin-spark-api-3.0/1.1.0/jar

Read more

Blog post: https://blog.jetbrains.com/big-data-tools/2022/05/26/kotlin-api-for-apache-spark-streaming-jupyter-and-more/

Bug Fixes

• Adds test for nullable properties in data classes
• Start to fix array of generics support
• Adds support for List and List and relative structures
• Struct type for nullable field
• Perform fixes in README.md (#113)

Build

• Updates build.yml

Documentation

• Fixes name of spark-3 API in README

Features

• UDF support
• Better selectTyped behavior
• Spark 3.2 support

Miscellaneous Tasks

• Updates Kotlin to 1.5.30

Features

• Added support for enums (a888ea4)

Bug

• adds reduceK function to avoid resolution ambiguity for reduce (72fb5ea)
• fixes invalid package name kotlinx → kotlinx (8e7523a)

CI

• Adds maven wrapper (e951424)

Accidentally 1.0.0 release had no content, this release contains content, release to maven central with version 1.0.1

No other changes