How is this Scala stream using 16 threads?

Keywords: scala parallel-processing stream lazy-evaluation


I’ve been experimenting using streams to generate prime numbers in Scala, and somehow it looks like the streams are running in parallel.

At first I was running into size limitations before it could really run for a long time, but now that I’ve changed it over to SafeLong and gotten rid of anything holding a head it’s been generating the primes up to 100,000,000 for the past 10-20 minutes (pretty slow, yeah, but that’s not the point).

Thing is, now that it can run for a while, I noticed that my CPU was sitting around 70% usage. That’s odd, so I checked task manager and that 70% is almost entirely the Java platform, and it’s evenly distributed across all 16 threads.

Here's the code:

import{File, PrintWriter}

import spire.math.SafeLong
import spire.implicits._

object PrimesTester {
  def main(args: Array[String]): Unit = {
    val max: SafeLong = 100000000

    println("Grabbing stream...")
    def primes: Stream[SafeLong] = getPrimesLazy(max)

    println("Opening file...")
    val oFile = new PrintWriter(new File(s"lazyPrimes_$max.txt"))

    println("Generating and saving...")
    for(p <- primes){

  def getPrimesLazy(max: SafeLong): Stream[SafeLong] = {
    //Generate all relevant odd numbers
    def odds: Stream[SafeLong] = Stream.iterate(3: SafeLong)(_ + 2).takeWhile(n => n*n <= max)

    //Generate all composite numbers in the range
    def composites: Stream[SafeLong] = odds.flatMap{ n =>
      Stream.iterate(n * n)(_ + 2 * n).takeWhile(_ <= max)

    //Take all composite numbers out of the range
    2 #:: Stream.iterate(3: SafeLong)(_ + 2).takeWhile(_ <= max).diff(composites)

I’m pretty sure streams are sequential, and I’ve only ever seen parallel collections behave like that.

This has me extremely curious, because working from my understanding of streams this shouldn’t happen. But here we are. Does anyone know what’s going on?