Spark properties control most application settings and are configured separately/individually for each application. These properties can be set directly on a SparkConf passed to your SparkContext. SparkConf permits you to configure some of the common properties (e.g. application name and master URL), as well as arbitrary key-value pairs through the set() method.
For example, we could initialize an application with two threads as below:
Note that we run with local, meaning three threads - which represents minimal parallelism,
Which can help detect bugs that only exist when we run in a distributed context.
Val conf = new SparkConf() .setMaster("local") .setAppName("CountingSlightBooks") Val sc = new SparkContext(conf)