This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package recfun | |
object Main { | |
def main(args: Array[String]) { | |
println("Pascal's Triangle") | |
for (row <- 0 to 10) { | |
for (col <- 0 to row) | |
print(pascal(col, row) + " ") | |
println() | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package example | |
import java.util.NoSuchElementException | |
object Lists { | |
/** | |
* This method computes the sum of all elements in the list xs. There are | |
* multiple techniques that can be used for implementing this method, and |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class TextAnalyser(val sc: SparkContext, ...) { | |
... | |
// Instance variable | |
val _commonWords = sc.broadcast(TextAnalyser.loadCommonWords()) | |
... | |
// In a worker thread | |
def analyse(rdd: RDD[String]): TextStats = { | |
// To prevent the whole class to be 'sucked' into serialization | |
val commonWords = _commonWords | |
... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class TextAnalyser(val sc: SparkContext, ...) { | |
... | |
// Instance variables | |
val _totalChars = sc.accumulator(0, "Total Characters") | |
val _totalWords = sc.accumulator(0, "Total Words") | |
... | |
// In a worker thread | |
def analyse(rdd: RDD[String]): TextStats = { | |
... | |
// This limits serialization to the reference variables only |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rdd | |
.flatMap(_.split("\\s")) // Split on any white character | |
.map(_.replaceAll( | |
"[,.!?:;]", "") // Remove punctuation and transfer to lowercase | |
.trim | |
.toLowerCase) | |
.filter(!_.isEmpty) // Filter out any non-words | |
.map(word => (word, 1)) // Finally, count words | |
.reduceByKey(_ + _) | |
.sortByKey() // and sort the word counts in a lexical order |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
cwd="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" | |
project_name=${1:-sbt_eclipse_project} | |
project_version=${2:-1.0} | |
scala_version=${3:-2.11.7} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
plugins { | |
id 'net.saliman.cobertura' version '2.2.7' | |
} | |
configurations.all { | |
resolutionStrategy { | |
// Cobertura includes an ASM version that can't handle Java 8, ASM 5.0.3 handles Java8 | |
// Please note that doesn't resolve all of the issues, such as the lack of support for Lambda expressions | |
force 'org.ow2.asm:asm:5.0.3' | |
forcedModules = [ 'org.ow2.asm:asm:5.0.3' ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apply plugin: 'java' | |
apply plugin: 'jacoco' | |
repositories { | |
jcenter() | |
} | |
dependencies { | |
compile 'org.slf4j:slf4j-api:1.7.12' | |
testCompile 'junit:junit:4.12' |