Skip to content

Conversation

@tjhgit
Copy link

@tjhgit tjhgit commented Mar 16, 2018

DSX now runs on version 2.11.8 and scala 2.1.2. Also tone analyzer changed the return value since 2017.

  1. use new versions in build.sbt
  2. got rid of Logging -> only using log4j
  3. JSONObject is now provided by different package

Tested this pull request as follows:

To compile it, use this command: docker run -it -v $(pwd):/root/src -p 4040:4040 -p 8080:8080 -p 8081:8081 -h spark --name=spark p7hb/docker-spark:2.1.1 when inside the above spark.samples directory.

Inside the docker container:

cd /src/streaming-twitter

sbt update
sbt clean compile
sbt console

sbt assembly

Runs flawlessly in sbt console, however leads to the following error on DSX:

Name: Compile Error
Message: :42: error: type mismatch;
found : org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext
required: org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.org.apache.spark.SparkContext
demo.startTwitterStreaming(sc, Seconds(30))
^
StackTrace:

evotjh added 2 commits March 16, 2018 10:33
… current dsx scala and spark version, modified old Logging class to use only log4j, JSONObject now in a different package
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants