CARVIEW |
Select Language
HTTP/2 301
date: Wed, 23 Jul 2025 17:41:21 GMT
content-type: text/html; charset=utf-8
content-length: 0
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With,Accept-Encoding, Accept, X-Requested-With
location: https://github.com/Kotlin/kotlin-spark-api/wiki/Column-functions
cache-control: no-cache
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
server: github.com
set-cookie: _gh_sess=R6EdEMaEw66hq7dkwJCXwul1s6bTVQoPrIY2n1%2FeVPKO0R2JJyGyNfaMsPl0%2FcuCH9TMz7G7RyiuvNFg6CyYuKxfaMRH5W1YbU0eu0Jubk4p3dU2ov4oMaGd%2Bwg4zablJ3tVRMCPUU%2BhnKVZtUTr%2FKyPMhZibmTh4IBW7qS42mK8X2X2i6m2BtbHCVhVaL1w9TXGUwHuBdKIn27rjds5PvOY4TFcEIVUys%2F3v38yPNIbRangr8LfVOSHt98kosXw1PvUtSjKQrbGue%2BF4Kl3pQ%3D%3D--7wjRWcKw4F5n3VHP--ZVrjGYc1BlT18oPkIvTaPg%3D%3D; Path=/; HttpOnly; Secure; SameSite=Lax
set-cookie: _octo=GH1.1.751797308.1753292481; Path=/; Domain=github.com; Expires=Thu, 23 Jul 2026 17:41:21 GMT; Secure; SameSite=Lax
set-cookie: logged_in=no; Path=/; Domain=github.com; Expires=Thu, 23 Jul 2026 17:41:21 GMT; HttpOnly; Secure; SameSite=Lax
x-github-request-id: 89F0:35EBF4:F625FE:122F76D:68811EC1
HTTP/2 200
date: Wed, 23 Jul 2025 17:41:22 GMT
content-type: text/html; charset=utf-8
vary: X-PJAX, X-PJAX-Container, Turbo-Visit, Turbo-Frame, X-Requested-With,Accept-Encoding, Accept, X-Requested-With
x-robots-tag: none
etag: W/"cdeefc1e96ee240918eb5a44dd2ef365"
cache-control: max-age=0, private, must-revalidate
strict-transport-security: max-age=31536000; includeSubdomains; preload
x-frame-options: deny
x-content-type-options: nosniff
x-xss-protection: 0
referrer-policy: no-referrer-when-downgrade
content-security-policy: default-src 'none'; base-uri 'self'; child-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/; connect-src 'self' uploads.github.com www.githubstatus.com collector.github.com raw.githubusercontent.com api.github.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com *.rel.tunnels.api.visualstudio.com wss://*.rel.tunnels.api.visualstudio.com objects-origin.githubusercontent.com copilot-proxy.githubusercontent.com proxy.individual.githubcopilot.com proxy.business.githubcopilot.com proxy.enterprise.githubcopilot.com *.actions.githubusercontent.com wss://*.actions.githubusercontent.com productionresultssa0.blob.core.windows.net/ productionresultssa1.blob.core.windows.net/ productionresultssa2.blob.core.windows.net/ productionresultssa3.blob.core.windows.net/ productionresultssa4.blob.core.windows.net/ productionresultssa5.blob.core.windows.net/ productionresultssa6.blob.core.windows.net/ productionresultssa7.blob.core.windows.net/ productionresultssa8.blob.core.windows.net/ productionresultssa9.blob.core.windows.net/ productionresultssa10.blob.core.windows.net/ productionresultssa11.blob.core.windows.net/ productionresultssa12.blob.core.windows.net/ productionresultssa13.blob.core.windows.net/ productionresultssa14.blob.core.windows.net/ productionresultssa15.blob.core.windows.net/ productionresultssa16.blob.core.windows.net/ productionresultssa17.blob.core.windows.net/ productionresultssa18.blob.core.windows.net/ productionresultssa19.blob.core.windows.net/ github-production-repository-image-32fea6.s3.amazonaws.com github-production-release-asset-2e65be.s3.amazonaws.com insights.github.com wss://alive.github.com api.githubcopilot.com api.individual.githubcopilot.com api.business.githubcopilot.com api.enterprise.githubcopilot.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com copilot-workspace.githubnext.com objects-origin.githubusercontent.com; frame-ancestors 'none'; frame-src viewscreen.githubusercontent.com notebooks.githubusercontent.com; img-src 'self' data: blob: github.githubassets.com media.githubusercontent.com camo.githubusercontent.com identicons.github.com avatars.githubusercontent.com private-avatars.githubusercontent.com github-cloud.s3.amazonaws.com objects.githubusercontent.com release-assets.githubusercontent.com secured-user-images.githubusercontent.com/ user-images.githubusercontent.com/ private-user-images.githubusercontent.com opengraph.githubassets.com copilotprodattachments.blob.core.windows.net/github-production-copilot-attachments/ github-production-user-asset-6210df.s3.amazonaws.com customer-stories-feed.github.com spotlights-feed.github.com objects-origin.githubusercontent.com *.githubusercontent.com; manifest-src 'self'; media-src github.com user-images.githubusercontent.com/ secured-user-images.githubusercontent.com/ private-user-images.githubusercontent.com github-production-user-asset-6210df.s3.amazonaws.com gist.github.com; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; upgrade-insecure-requests; worker-src github.githubassets.com github.com/assets-cdn/worker/ github.com/assets/ gist.github.com/assets-cdn/worker/
server: github.com
content-encoding: gzip
accept-ranges: bytes
x-github-request-id: 89F0:35EBF4:F6262E:122F79A:68811EC1
Column functions · Kotlin/kotlin-spark-api Wiki · GitHub
Skip to content
Navigation Menu
{{ message }}
-
Notifications
You must be signed in to change notification settings - Fork 35
Column functions
Jolan Rensen edited this page Aug 2, 2022
·
3 revisions
Similar to the Scala API for Columns
, many of the operator functions could be ported over.
For example:
ds.select( col("colA") + 5 )
// datasets can also be invoked to get a column
ds.select( ds("colA") / ds("colB") )
dataset.where( col("colA") `===` 6 )
// or alternatively
dataset.where( col("colA") eq 6)
In short, all supported operators are:
-
==
- same as
equals()
- same as
-
!=
- same as
!equals()
- same as
-
eq
/`===`
- in Scala:
===
- in Java:
equalTo()
- in Scala:
-
neq
/`=!=`
- in Scala:
=!=
- in Java:
notEqual()
- in Scala:
-
-col(...)
- same in Scala
- in Java:
negate(col())
-
!col(...)
- same in Scala
- in Java:
not(col())
-
gt
- in Scala:
>
- same in Java but also infix
- in Scala:
-
lt
- in Scala:
<
- same in Java but also infix
- in Scala:
-
geq
- in Scala:
>=
- same in Java but also infix
- in Scala:
-
leq
- in Scala:
<=
- same in Java but also infix
- in Scala:
-
or
- in Scala:
||
- same in Java but also infix
-
`||`
is unfortunately an illegal function name on Windows
- in Scala:
-
and
/`&&`
- in Scala:
&&
- in Java:
and()
- in Scala:
-
+
- same in Scala
- in Java:
plus()
-
-
- same in Scala
- in Java:
minus()
-
*
- same in Scala
- in Java:
multiply()
-
/
- same in Scala
- in Java:
divide()
-
%
- same in Scala
- in Java:
mod()
Secondly, there are some quality of life additions as well:
In Kotlin, Ranges are often
used to solve inclusive/exclusive situations for a range. So, instead of between(a, b)
you can now do:
dataset.where( col("colA") inRangeOf 0..2 )
Also, for columns containing map- or array-like types, instead of getItem()
we have:
dataset.where( col("colB")[0] geq 5 )
Finally, thanks to Kotlin reflection, we can provide a type- and refactor safe way
to create TypedColumn
s and with those, a new Dataset from pieces of another using the select()
function:
val dataset: Dataset<YourClass> = ...
val newDataset: Dataset<Tuple2<TypeA, TypeB>> = dataset.select(col(YourClass::colA), col(YourClass::colB))
// Alternatively, for instance when working with a Dataset<Row>
val typedDataset: Dataset<Tuple2<String, Int>> = otherDataset.select(col<_, String>("a"), col<_, Int>("b"))
Clone this wiki locally
You can’t perform that action at this time.