-
Notifications
You must be signed in to change notification settings - Fork 0
r-spark/sparkavroudf
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
---
title: "What the Package Does (Title Case)"
output:
github_document:
fig_width: 9
fig_height: 5
---
## Building
First build this package, then build its jars by running:
```{r eval=FALSE}
spec <- sparklyr::spark_compilation_spec(spark_version = "2.4", jar_name = "sparkavroudf-2.4-2.11.jar", jar_dep = normalizePath("deps/spark-avro_2.12-2.4.2.jar"))
sparklyr::compile_package_jars(spec = spec)
```
then build the R package as usual.
## Getting Started
Connect and test this package as follows:
```{r}
library(sparkavroudf)
library(sparklyr)
sc <- spark_connect(master = "local", version = "2.4")
sparklyr::invoke_static(sc, "sparkavroudf.AvroUtils", "toAvro", spark_dataframe(sdf_len(sc, 3)), invoke_new(sc, "org.apache.spark.sql.Column", "id")) %>% sdf_register()
```
```{r}
spark_disconnect_all()
```
About
Support for Avro with UDF-like interface
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published