Using function passing to separate concerns

So I had a difficult problem today where I needed to access a service but I couldn’t inject it due to the service having slightly too many concerns and hence a circular dependency in the Dependency Injection. So there is right way of dealing with this, which is to refactor the concerns of the collaborator until the dependency goes away. Unfortunately there is also the incomplete user journey that is holding everyone up unless a quick hacky fix is found.

Enter function passing to the rescue! In my case we are talking about an MVC web application where the Model is generating the circular dependency but it is possible to inject the collaborator into the Controller. The trouble is that my code separates the concerns of the Controller and the Model such that the Controller asks the Model to perform an operation and the Model returns the result of executing the operation. This is clean to develop and read but it also means that the Controller cannot access any of the internal operations of the Model, which is what I need for the invocation of the collaborator.

For a moment I thought I had to break the encapsulation between the two and pass the internal information of the Model operation out with the operation result, however function passing is a more elegant solution that keeps my concerns separate. Essentially the Controller will ask the Model to do something and also provide a function for the Model to execute if it wishes to invoke the collaborator.

Let’s try to illustrate this in code

class Controller (stuff: StuffRepository, messages : MessageDispatcher) {

post("/handle/stuff/:stuffId") {

def messageCallback(id: StuffId, message : Message) { messages.send(Messages.StuffUpdated, message, Some(id)) }

stuff.storeChange(params("stuffId"), messageCallback) }}

class StuffRepository {

def storeChange(id : String, callback : (StuffId, Message) => Unit) = {

makeChanges(id) match {

case Success => callback(StuffId(id), Message("Changes completed")) ; Success

case Failure => Incomplete }}

Hopefully that’s all relatively obvious, you can also type the callback for clarity and make it an Option if you don’t always want the callback invoked. If you don’t like the closure you can probably re-write is as a function that partially applies MessageDispatcher and then returns the function.

Some of you are going to be thinking this is totally obvious (particularly if you a Javascript fiend) and that’s cool but I do think it is an interesting technique for keeping responsibilities separated without very complex patterns. It is also something that is only possible with proper first order functions.

Scala can be the replacement for Enterprise Java

Last week I had the chance to take a look at one of my worse fears about Scala: really bad Scala code. As I thought it might it did include a partial application, I was very grateful it didn’t abuse apply. It also managed to bring in a lot of classic bad behaviour like catching Exception and then swallowing it.

It has always been one of my fears that when someone went rogue with Scala the resulting mess would be so bad it would be a kind of code Chernobyl, its fearsome incomprehensibleness would be like the Lift code only without the brevity.

When I finally had to confront it I thought: this is actually no worse than really bad Java code.

I know that it could potentially be worse but actually it would take someone who was actually really good at Scala to construct the nightmare-ish worst case scenario I had imagined. In most cases terrible Scala code is going to look a lot like terrible Java code.

So that is really my last concern out of the way. We might as well use Scala for the good things it offers like immutability and first-order functions. The boost to productivity is going to outweigh the potential crap that might also be generated.

Mockito and Scala

Scala is a language that really cares about types and mocking is the art of passing one type off as another so it is not that surprising that the two don’t get along famously. It is also a bit off probably that we are using Java mocking frameworks with Scala code which means you sometimes need to know too much about how Scala creates its bytecode.

Here are two recent problems: the “ongoing stubbing” issue and optional parameters with defaults (which can generally be problematic anyway as they change the conventions of Scala function calling).

Ongoing stubbing is an error that appears when you want to mock untyped (i.e. Java 1.4) code. You can recognise it by the hateful “?” characters that appear in the error messages. Our example was wanting to mock the request parameters of Servlet 2.4. Now we all know that the request parameters (like everything else in a HTTP request) are Strings. But in Servlet 2.4 they are “?” or untyped. Servlet 2.5 is typed and the first thing I would say about an ongoing stubbing issue is to see if there is Java 1.5 compatible version of whatever it is you are trying to mock. If it is your own code, FFS, add generics now!

If it is a library that you have no control over (like Servlet) then I have some bad news, I don’t know of any way to get around this issue, Scala knows that the underlying code is unknown so even if you specify Strings in your mock code it won’t let it compile and if you don’t specify a type your code still won’t compile. In the end I created a Stub sub-class of HttpServletRequest that returned String types (which is exactly what happens at runtime, thank you very much Scala compiler).

Okay so optional parameters in mocked code? So I love named parameters and default values because I think they are probably 100% (no, perhaps 175%) better at communicating a valid operating state for the code than dependency injection. However when you want to mock and set an expectation on a Scala function that uses a default value you will often get an error message saying that the mock was not called or was not called with the correct parameters.

This is because when the Scala code is compiled you are effectively calling a chain of methods and your mock needs to set matchers for every possible argument not the ones that your Scala code is actually calling. The simplest solution is to use the any() matcher on any default argument you will not be explicitly setting. Remember that this means the verification must consist entirely of matchers, e.g. eq() and so on.

What to do when you want to verify that a default parameter was called with an explicit value? I think you do it based on the order of the parameters in the Scala declaration but I haven’t done it myself and I’m waiting for that requirement to become a necessary thing for me to know.

Control expressions are functions too

I am kind of simultaneously learning Clojure and Scala, the former through the London Clojure Dojo (come and join us) and the latter through work. It is an interesting experience but I am very glad that Clojure is part of the mix as it helps understand a lot of things that are seem strange in Scala.

In Clojure everything is essentially either a function or a sequence. It is therefore not surprising to see that an if statement works like a function.

(if false 1 2)

Evaluating the above in a Clojure REPL results in the answer 2: if is a function that evaluates its first argument returning the second parameter if the evaluation is true, the third if false.

The same is true of Scala, with the additional wrinkle that if the different clauses evaluate to different types you could be in type hell. Despite its superficial similarity to Java syntax it is in fact quite different and you can compose it just as you would any other function.

1 + (if(true) 2 else 3)

Evaluated in a Scala REPL gives the result 3.

Scala 2.8 seems much better about making everything return a value (and hence act like a function), val and for will both now return Unit, which is not useful but prevents the compiler moaning.

This kind of thing is much easier to appreciate and learn in a pure functional language than in Scala where you never really know whether something is going to operate in a functional or object-orientated way.

Continuous Testing

Continuous testing is one of those things that has crept up on me slowly. About two years ago I was aware of people using a trigger on their TextMate save to run tests and, if green, commit to git. At the time it felt too much effort for too little gain but it was a cool trick.

Now as we forage out into the post-Java world we are starting to get some pretty cool revisions of familiar tools and one of the most engaging for me are continuous build tools. The daddy is clearly SBT, which while simple is also tremendously sophisticated. Adding a REPL to a build is a simple change but has all kinds of nice consequences, my favourite of which currently is the continuous test (~test) target that detects changes in your source and test files, compiles them and runs your tests. SBT cuts out the whole compile-link-run cycle for you, you just make a change, hit save, see the consequences and code again. It’s very fast and far more effective in giving feedback than any of the current IDEs (all of which need to get on this bandwagon fast is they want to stay relevant).

Clojure by comparison has been suffering in this regard with Leiningen becoming an unfortunately early defacto standard despite it standing shoulder to shoulder with the benighted Maven. The key thing that Leiningen does wrong is stay at the command-line and force you to cold-boot a JVM with each new command (the second is dependency resolution, SBT favours Ivy). Fortunately Lazytest by Stuart Sierra can hopefully save us here. Although still alpha Lazytest is an awesome way of developing Clojure and it’s hard to beat that feeling of smug satisfaction as the tests go green.

It is these kind of step change enhancements in development that are going to carry us forward more than shopping list of features that the new languages have or lack.

Scala and Voldemort: Types matter

So I have been taking a good look at Java object and key-value stores recently. One of the more interesting (because it is very aligned with Java rather than being language-agnostic) is Voldemort. However I got it into my head that it would be a good idea to actually play around a bit using the Scala console. That involved a lot of classpath tweaking, made simpler because the Voldemort distribution comes bundled with its dependencies. However I still rather strangely needed the Voldemort Test jar to be able to access Upper Case View class. I’m not whether I am doing something wrong here or whether the packaging has gone wrong during the build.

With the classpath complete I followed the quickstart instructions and quickly found that actually Scala’s strict type-system was revealing something interesting about the Voldemort implementation. If you translate the Quickstart example to Scala literally then you end up with a Store Client that cannot store anything as it is a Client of type [Nothing, Nothing] and if you supply a value you get an error. I realised that I needed to interact with a lower level version of the API and was surprised to find that the signature of the constructor seemed to need a Resolver instead of defaulting one and that I had to supply a Factory instead of the Config that the Factory consumes.

This is the code I came up with (note that you need to paste this into an interactive Scala REPL rather than putting it in a script). However I’m a nub with Voldemort and I’m not exactly fly with Scala either. Any advice or input is appreciated.

After writing this I was wondering how to try and store arbitrary objects, initially I thought it would be as simple as declaring the Client to be [String, Any] but that failed due to a Class Cast exception in the String Serializer, it looks like the server config comes into play here and says how a value should be serialised and stored. It’s an interesting experience, I have also been looking at Oracle’s BDB but Voldemort puts a much nicer (read Java 1.5) interface on top of it.

The Java Developer’s Dilemmia

I believe that Java developers are under a tremendous amount of pressure at the moment. However you may not feel it if you believe that Java is going to be around for a long time and you are happy to be the one maintaining the legacy apps in their twilight. Elliotte Rusty Harold has it right in the comments when someone says that there are a lot of Java jobs still being posted. If you enjoy feasting off the corpse then feel free to ignore the rest of this post because it is going to say nothing to you.

Java is in a tricky situation due to competition on all fronts. C# has managed to rally a lot of support. Some people talk nonsense about C# being what Java will look like in the future. C# is what Java would like if you could break backwards compatibility and indeed even runtime and development compatibility in some cases (with Service Packs). C# is getting mind share by leapfrogging ahead technology-wise at the expense of its early adoptors. Microsoft also does a far better job of selling to IDE dependent developers and risk-adverse managers.

Ruby and Python have also eaten Java’s lunch in the web space. When I am working on web project for fun I work with things like Sinatra, Django and Google App Engine. That’s because they are actually fun to work with and highly productive. You focus on your problem a lot sooner than you do in Java.

The scripting languages have also done a far better job of providing solutions to the small constant problems you face in programming. Automating tasks, building and deployment, prototyping. All these things are far easier to do in your favourite scripting language than they are in Java which will have to wait for JDK7 for a decent Filesystem abstraction for example.

Where does this leave Java? Well in the Enterprise server-side niche, where I first started to use it. Even there though issues of concurrency and performance are making people look to things like Erlang and JVM alternatives like Scala and Clojure.

While, like COBOL and Fortran there will always be a market for Java skills and development. The truth is that for Java developers who want to create new applications that lead in their field; a choice about what to do next is fast approaching. For myself I find my Java projects starting to contain more and more Groovy and I am very frustrated about the lack of support for mixed Java/Groovy projects in IDEs (although I know SpringSource is putting a lot of funding into the Eclipse effort to solve the problem).

If a client asks for an application using the now treadworn combination of Spring MVC and Hibernate I think there needs to be a good answer as to why they don’t want to use Grails which I think would increase productivity a lot without sacrificing the good things about the Java stack. Companies doing heavy lifting in Java ought to be investigating languages like Scala, particularly if they are arguing for the inclusion of properties and closures in the Java language spec.

Oracle’s purchase of Sun makes this an opportune moment to assess where Java might be going and whether you are going to be on the ride with it. It is hard to predict what Oracle will do, except that they will act in their perceived economic interest. The painful thing is that whatever you decide to do there is no clear answer at the moment and no bandwagon seems to be gaining discernible momentum. It is a tough time to be a Java developer.

Toy Scala Static Webserver in less than 100 lines

Doh! The original link to the code was for the wrong file! Sorry. Corrected below.

Okay so there has been a craze for writing static webservers in new languages in less than 100 lines recently and I am not claiming that this code is anything special but I wanted to give to give the NetBeans Scala plugin a go (in NetBeans 6.5) so here’s my version of a Scala static webserver (in less than 100 lines, natch).

The code is more of a Toy at the moment as it assumes a very happy path. However it does work and it did provide some useful learning about Scala.

The good stuff includes: compact code, class imports, XML literals, map literal syntax and the NetBeans plugin does a good job of providing codealong feedback from the compiler.

Confusing stuff:

• accessing array indexes with () not [], it makes sense but you have to get used to when coming from Java
• implementing Java interfaces in Scala: still not totally sure how you do that
• getting the right type to allow a Java API to be called: Array[Byte] seemed to take a long time to get right and not having type-coercion for Scala Lists to Java Lists means there is a anemic list variable in play
• functions that have many parameters make for confusing IDE errors; do you want Int, Int, Int, Int, String, String or Int, Int, Int, Int?

And finally the bad:

• the streaming IO for binary files is entirely imperative and is basically Java code, I’ve been told that Scalax can help put a prettier front on that
• nothing to do with Scala but the in-built Java HttpServer should have had the public API in interfaces and there should have had an NIO-based HttpExchange
• I.cant.stop.using.periods even.if.I.dont.have.to

Overall I am pretty happy with the quality of the final code and I feel I’m finding the balance of the language more and seeing Scala as more of an extension of Java than something entirely unique.

Splitting Loot with Scala

Over the last couple of days I have been trying to implement the Sharing Problem (Ruby Quiz #65: Splitting the Loot) in Scala. So far I have implemented the greedy pick algorithim and still need to implement a recursive solution that will brute force the edge cases.

However on the way I think I have picked up some important lessons. You can see the code for yourself in the GitHub project related to the problem and the solution (it’s in the Scala directory) but I will be highlighting some of the code in this post.

This was the first set of Scala I’ve done that was actually an attempt to use some of the functional aspects of Scala rather than just porting Java code in a more or less literal and imperative way. After struggling a little bit with the implict return and list concatenation I implemented the greedy heuristic. This is what it looked like (link to the code file).

import gems._

package splitter {

  class LootSplitter

  object LootSplitter {
    def splitLoot(gembag: GemBag, shares: Int): List[GemBag] = {

      if((gembag.totalValue % shares) != 0) {
        return List[GemBag]()
      }

      val individualShareValue = gembag.totalValue / shares

      var partShares: List[GemBag] = List.make(shares, new GemBag(List()))

      gembag.gems.sort(_.value > _.value).foreach((gem: Gem) => {
        partShares = partShares.sort(_.totalValue < _.totalValue)
        partShares = List(partShares(0) add gem) ::: (partShares drop 1)
      })

      partShares
    }

    }
  }
}

I was struck initially by how compact this code was, you are looking at about 28 lines of code. However as I was looking at it I began to wonder whether it was concise or just terse. How is someone meant to interpret something like _.value > _.value? I showed it to a collegue and his first reaction was “I don’t know functional languages so I wouldn’t know what this means”.

This was exactly the kind of reaction I was afraid of because I have been converted heavily to the principal of readable code. Someone should be able to scan code and understand, in principle, what is happening here. If they don’t then the cost of maintaining that code is going to be higher and we have actually lost something in the concise syntax.

I decided to try and implement a readable version of the same file which added about 6 lines (only two according to GitHub!). You can read this version here but now I want to throw it open to the public. Is this version actually easier to read? Are things like the underscore variable actually part of the price of comprehending Scala?

In my rewritten version I use some of the nice features of Scala such as first order functions but you still have lines like this:

List(sortedBags(0) add gem) ::: (sortedBags drop 1)

I hope you are reading this as “add a gem to the first sortedBag and make a List of it and then add all the other bags except the first one to the new list” but I am worried that this is far from obvious. Is that because I’ve done something that isn’t idiomatic or is it because actually the operators and the library API are too obscure?

Scala represents a significant evolution of Java in terms of absorbing all the lessons learned during the evolution of the language. When porting Java code I feel far more comfortable with it than when I am trying to create new code that uses the core language libraries. I don’t want to evolve to a new set of problems and best practices that try to avoid them.

Baby steps in Scala

Okay so following on from the User Group meeting I have started the exercise of porting a Java application to Scala. I’ve chosen Jay’s Secure Diary (or just JDiary for short) and I have commited some code to Launchpad.

So then, learnings so far… Well this is all being done via the command-line and a text editor as I have the Eclipse plugin has just not wanted to play ball with me. It’s also a tremendous pain in the ass to get stable updates of the plugin. I’m getting support from the Scala Tools forum but it feels a curious mix of features. A lot of things are very polished but the basic behaviour isn’t solid.

I was mightly relieved when I got Gant working and taking some of the strain of the build. It is surprisingly easy to get going. I based mine on this blog post. The harder part is referencing the Scala Home Directory which I have hardcoded for now and need to sort out.

I also gave Scala GUI a go but gave up quite quickly because it seems very opinionated about how you should be doing guis and while I like some of the ideas it didn’t allow me to use any of my existing Swing knowledge and didn’t have any easy documentation for how you are supposed to use it. In Profligacy for JRuby and Groovy you are allowed to mix the two approaches a lot more freely.

I also spent a ridiculous amount of time trying to implement the ActionListener trait/interface until I hit on the fact I was meant to be using the override key word. The compiler error complains that the parameter (ActionEvent in this case) is not defined, that’s true but it’s really not helping you understand what is wrong with the code.

I also spent too long fiddling around with Launchpad, but that’s another story for another post.

After investing eight hours or so in the effort I can be happy that my code is looking relatively clean compared to the original but I have to say that I was enjoying Scala more when I was sticking to just Scala.