First steps with Scala revisited once more: Scala’s implicit revenge


Last post might have left a bitter taste in scala’s fans mouth. In there, after developing a very simple script with scala, bash and python, I found the latter one better suited for the task.

But my first attempt with scala, was just an excuse to show some pretty powerful scala features, like Type Inference, collections, higher order functions, functional programming and some other goodies.

So this time I’ll take a more pragmatic -and less pedagogic- approach, and will just try to mimic the python script.

Here’s my first attempt:

#!/bin/sh
exec scala -savecompiled "$0" "$@"
!#

import java.io._

val total_files = new File(".").listFiles.filter(_.getName.endsWith(".textile"))
val translated_files = total_files.filter { file => 
  new BufferedReader(new FileReader(file)).readLine.
    indexOf("Esta página todavía no ha sido traducida al castellano") == -1
}

val total_size = total_files.map(_.length).sum / 1000
val translated_size = translated_files.map(_.length).sum / 1000
val size_percent = translated_size * 100 / total_size

println( "translated size: %dkb/%dkb %d%% (pending %dkb %d%%)" 
  format(translated_size, total_size, size_percent, total_size-translated_size, 100-size_percent)
)

val total_count = total_files.length
val translated_count = translated_files.length
val count_percent = translated_count * 100 / total_count

println( "translated files: %d/%d %d%% (pending %d %d%%)" 
  format(translated_count, total_count, count_percent, total_count-translated_count, 100-count_percent)
)

You can compare it to the python version below:

#! /usr/bin/env python
# -*- coding: utf-8 -*-

import fnmatch
import os

total_files = [file for file in os.listdir('.') if fnmatch.fnmatch(file, '*.textile')]
translated_files = [file for file in total_files if "Esta página todavía no ha sido traducida al castellano" not in open(file).read()]

total_size = sum([os.path.getsize(file) for file in total_files]) / 1000
translated_size = sum([os.path.getsize(file) for file in translated_files]) / 1000
translated_percent= translated_size * 100 / total_size

print "translated size: %dkb/%dkb %d%% (pending %dkb %d%%)" % \
      (translated_size, total_size, translated_percent, total_size-translated_size, 100-translated_percent)

total_count=len(total_files)
translated_count=len(translated_files)
translated_percent= translated_count * 100 / total_count

print "translated files: %d/%d %d%% (pending %d %d%%)" % \
      (translated_count, total_count, translated_percent, total_count-translated_count, 100-translated_percent)

Well, no such a big deal, both scripts are functionally equivalent, number of lines is approximately the same, I guess it’s now just a matter of personal taste.

Of course there are quite a lot of differences, scala runs on the JVM, is type safe and compiled. On the other hand, Python is more lightweight and a little less verbose.

That’s specially true for the “new BufferedReader(new FileReader(file)).readLine” part of the script, so I think we can improve it.

Scala’s io package

We can take advantage of Scala’s io package. Add the following import: “import scala.io.Source._” and just issue:

val translated_files = total_files.filter { file => 
  fromFile(file).mkString.
    indexOf("Esta página todavía no ha sido traducida al castellano") == -1
}

This reads the whole file, in order to read just the first line (like we did using the BufferedReader) do like this:

val translated_files = total_files.filter { file => 
  fromFile(file).getLines.take(1).mkString.
    indexOf("Esta página todavía no ha sido traducida al castellano") == -1
}

The current scala io package isn’t quite mature as java’s libraries. In fact, if you look at the documentation, apparently it was meant to be used mainly for source files. Work is being done on a new standard io library named scala-io. Luckily, you can always turn to the good old java libraries.

Well, it’s a little better, but we can make it more elegant and, at the same time, learn a very powerful scala feature.

Implicit conversions: Scala’s answer to open classes

Ruby is pretty famous, among other things, for letting you easily add new methods to existing classes, like this:

class Integer
  def squared
    self * self
  end
end

That will allow you to do this:

>> 4.squared
=> 16

Now in Scala, being a type-safe statically compiled language, that sort of magic is not so easy to accomplish, nevertheless they found a pretty elegant and safe way to do it.

You don’t touch the original class, you just define a new one, with the desired methods to add, and define a function that will implicitly convert from one type to the other.

To give it a try we will use the interactive scala interpreter, better known as REPL, which stands for Read-Evaluate-Print Loop.

So just type “scala” and you’ll be taken to the REPL. There we’ll create the class with the new method to enchance the Int class.

scala> class EnhancedInt(x: Int) { def squared = x * x }

And with the following code we’ll tell scala to automatically convert a regular Int to a EnhancedInt where the context demands it:

scala> implicit def IntToEnchancedInt(x: Int) = new EnhancedInt(x)

and after that you can try it

scala> 4.squared
res0: Int = 16

You can check this article, from where I took the example, for a much detailed explanation.

Enhancing the File class

What we want to do, is to let the class File tell us if the file is translated or not.

To do that we’ll just create a DocumentationFile class, like this:

class DocumentationFile(val file: File) {
  val isTranslated = (
    fromFile(file).getLines.take(1).mkString.
      indexOf("Esta página todavía no ha sido traducida al castellano") == -1
  )
}

And then add the corresponding implicit def to tell scala how to convert from a regular File to a DocumentationFile:

implicit def fileToDocumentationFile(file: File): DocumentationFile = new DocumentationFile(file)

And now, we can get the translated files with this neat line of code

val translated_files = total_files.filter(_.isTranslated)

Note that just like we did in the example with the Int class, with a couple of lines of code, we are enhancing an existing -and closed- Java class using scala. As we said earlier, it’s a pretty powerful feature, that should be used with caution. After a couple of these tricks, with many implicit conversions going around, following the code can get a bit messy.

This is the complete source code:

#!/bin/sh
exec scala -savecompiled "$0" "$@"
!#

import java.io._
import scala.io.Source._

val total_files = new File(".").listFiles.filter(_.getName.endsWith(".textile"))
val translated_files = total_files.filter(_.isTranslated)

val total_size = total_files.map(_.length).sum / 1000
val translated_size = translated_files.map(_.length).sum / 1000
val size_percent = translated_size * 100 / total_size

println( "translated size: %dkb/%dkb %d%% (pending %dkb %d%%)" 
  format(translated_size, total_size, size_percent, total_size-translated_size, 100-size_percent)
)

val total_count = total_files.length
val translated_count = translated_files.length
val count_percent = translated_count * 100 / total_count

println( "translated files: %d/%d %d%% (pending %d %d%%)" 
  format(translated_count, total_count, count_percent, total_count-translated_count, 100-count_percent)
)

implicit def fileToDocumentationFile(file: File): DocumentationFile = new DocumentationFile(file)

class DocumentationFile(val file: File) {
  val isTranslated = (
    fromFile(file).getLines.take(1).mkString.
      indexOf("Esta página todavía no ha sido traducida al castellano") == -1
  )
}

Conclusion

Now the differences between the python and the scala version aren’t so huge anymore. It is true that right now scala is lacking a more powerful IO standard library for dealing with files, but you can always use the vastly tested java libraries.

After this I think that scala has somehow recovered it’s reputation for scripting purposes, and at the same time we could have a look at implicit conversions, a powerful scala feature.

About these ads

8 responses to this post.

  1. Posted by Captain on 16 January, 2012 at 4:50

    What do you mean by “type safe”?

    Reply

  2. I mean like in java’s type safety. Scala’s compiler will verify that each variable is of the correct type, at compile time. That’s different from how bash or python works…

    Reply

    • Posted by Captain on 17 January, 2012 at 8:06

      You could argue Python is more “type safe” with the following example:

      Java
      int a = 1;
      String b = “something”;
      System.out.println(a + b); // “1something”

      Python
      1 + “something”
      #TypeError: unsupported operand type(s) for +: ‘int’ and ‘str’

      Reply

      • mmm, I think that’s an isolated case, I’m no python expert, but I wonder how easy would be to refactor complex code without breaking anything…

  3. f# partial classes, coupled with extension methods, are just so much simpler…

    Reply

  4. Posted by EasyBob on 17 January, 2012 at 11:04

    What about performance now?

    Reply

    • I tried it again:

      sas@ubuntu:~/devel/apps/playdoces/documentation/1.2.4/manual$ time ./status2.scala
      translated size: 429kb/626kb 68% (pending 197kb 32%)
      translated files: 40/64 62% (pending 24 38%)

      real 0m0.553s
      user 0m0.416s
      sys 0m0.052s

      it seems to last a little bit longer…

      Reply

  5. Posted by Dan Spencer on 24 January, 2012 at 12:12

    >f# partial classes, coupled with extension methods, are just so much simpler…

    You can’t just say things like this. It’s similar to saying my dad’s better than your dad. I suggest you write an article as good as this one and provide a link to it; otherwise your comment is simply not worth anything and is annoying to people who like to think.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: