SparkR + Cassandra query with complex data types - r

Question

We have SparkR setup to connect to Cassandra and we are able to successfully connect/query Cassandra data. However, many of our Cassandra column families have complex data types like MapType and we get errors when querying these types. Is there a way to coerce these before or during the query using SparkR? For example, cqlsh command of the same data would coerce a row of MapType column b below to a string like "{38: 262, 97: 21, 98: 470}"
Sys.setenv(SPARK_HOME = "/opt/spark")
library(SparkR, lib.loc = c(file.path(Sys.getenv("SPARK_HOME"), "R", "lib")))
mySparkPackages <- "datastax:spark-cassandra-connector:1.6.0-s_2.10"
mySparkEnvironment <- list(
spark.local.dir="...",
spark.eventLog.dir="...",
spark.cassandra.connection.host="...",
spark.cassandra.auth.username="...",
spark.cassandra.auth.password="...")
sc <- sparkR.init(master="...", sparkEnvir=mySparkEnvironment,sparkPackages=mySparkPackages)
sqlContext <- sparkRSQL.init(sc)
spark.df <- read.df(sqlContext,
source = "org.apache.spark.sql.cassandra",
keyspace = "...",
table = "...")
spark.df.sub <- subset(spark.df, (...)), select = c(1,2))
schema(spark.df.sub)
StructType
|-name = "a", type = "IntegerType", nullable = TRUE
|-name = "b", type = "MapType(IntegerType,IntegerType,true)", nullable = TRUE
r.df.sub <- collect(spark.df.sub, stringsAsFactors = FALSE)
Here we get this error from the collect():
16/07/13 12:13:50 INFO TaskSetManager: Finished task 1756.0 in stage 0.0 (TID 1756) in 1525 ms on ip-10-225-70-184.ec2.internal (1757/1758)
16/07/13 12:13:50 INFO TaskSetManager: Finished task 1755.0 in stage 0.0 (TID 1755) in 1661 ms on ip-10-225-70-184.ec2.internal (1758/1758)
16/07/13 12:13:50 INFO DAGScheduler: ResultStage 0 (dfToCols at NativeMethodAccessorImpl.java:-2) finished in 2587.670 s
16/07/13 12:13:50 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
16/07/13 12:13:50 INFO DAGScheduler: Job 0 finished: dfToCols at NativeMethodAccessorImpl.java:-2, took 2588.088830 s
16/07/13 12:13:51 ERROR RBackendHandler: dfToCols on org.apache.spark.sql.api.r.SQLUtils failed
Error in readBin(con, raw(), stringLen, endian = "big") :
invalid 'n' argument
Our stack:
Ubuntu 14.04.4 LTS Trusty Tahr
Cassandra v 2.1.14
Scala 2.10.6
Spark 1.6.2 with Hadoop libs 2.6
Spark-Cassandra connector 1.6.0 for Scala 2.10
DataStax Cassandra Java driver v3.0 (v3.0.1 actually)
Microsoft R Open aka Revo R version 3.2.5 with MTL
Rstudio server 0.99.902