The geeks were back in town...

Jay Zimmerman and crew descended upon southern Denver for the Fall 2008 No Fluff Just Stuff symposium. And I had the opportunity to attend.

This is my fourth No Fluff and as usual, it was great. Eleven hour-and-a-half long sessions, a great keynote, and an entertaining and informative panel discussion, coupled with some great networking filled the weekend. My brains were leaking out of my ears by conference end Sunday evening.

Here are the sessions I attended (as always, the biggest complaint I have is that there were some hard choices to make when choosing sessions):

Beginning Drools - Rule Engines in Java, Brian Sam-Bodden
What's Going On? : Complex Event Processing w/ Esper, Brian Sletton
Rapid Web Development with Grails and Ajax, Scott Davis
Boosting Programmer productivity with Mylyn, Brian Sam-Bodden
Design Patterns in Dynamic Languages, Neal Ford
Java.next #1: Common Ground, Stu Halloway
Architecture and Scaling, Ken Sipe
Git control of your source, Stu Halloway
Agile Test Driven Development With Groovy , Jeff Brown
Real World Hibernate Tips, Scott Leberknight
Google Your Domain Objects With Hibernate Search, Scott Leberknight
Keynote - Soft Skills and Organizational Dynamics, Ken Sipe

Here are some rambling take-aways from the sessions. The goal is to peak curiosity, not repeat the preso. Feel free to drop me some e-mail or leave a comment if you would like to discuss further.

The first two sessions I chose based upon professional curiosity, since, in a past life, I worked on a commercial Complex Event Processor. Rules engines are similar in nature, so I wanted to compare the two. Turns out, the biggest difference is that in a CEP, time is a de facto part of the system (you handle data in time order order the vast majority of the time). By default, the Drools rules engine processes rules in the order they were entered into the system. Another difference is that typically, a rules-based system is used when you already understand the business logic you wish to use for processing the data, whereas a CEP system might be used to discover what those rules are. Keep in mind that these differences are a bit fuzzy. I believe you could use either system to solve most problems without any serious complications. It was interesting to see that the user interface of Esper was not radically different from the commercial product I worked on (the underlying implementation could be another matter).

Most of my technically oriented friends know I am a Groovy fanboy. For that reason, I sought to expand my knowledge of this dynamic language and attended a couple of sessions dealing with the language. Unfortunately, both talks were a bit basic for my needs. The Grails talk was interesting, but most of the time was spent covering grails, so it was review. Because of time, the integration with Ajax was a bit rushed and not given the detail I had hoped for. The good news is that Scott runs the local JUG, so there is hope he will present the Ajax integration there in the near future (hint, hint). The Groovy testing talk suffered from a similar problem. Too much time covering the basics and not enough time discussing the interesting issues you can solve using GroovyTestCase. In particular, I had hoped to see more about how to test Java with Groovy. This is given a a lot of lip service, but examples are less common. (aside: I have done a little with this, see Exploring Groovy's Optional Typing Features). Instead, two very basic examples of using Groovy to test Groovy were presented. for folks new to Groovy, this was probably helpful. Another case of a talk that needs to be broken into basic and advanced sessions.

Neal's talk on Design Patterns in Dynamic languages, while not specific to Groovy, provided my Groovy fix for the weekend. The take away from this talk is that design patterns really have two purposes. As a mechanism for communication with other developers, they provide a set of common semantics. Neal feels this is a "Good Thing"(tm) and doesn't think this should change. The second purpose, on the other hand, is to provide band-aids to languages that lack features that can directly solve the problem a pattern is targeted toward. Neal argues that in the dynamic language arena, this reason is less relavent. Referring to the Gang of Four book, Neal relates anecdotally that one of the authors once stated that the sub-title of the book could have been "Making C++ suck less." To back this up, Neal went through several patterns, starting with the Command Pattern, that require a lot of scaffolding in traditional languages, but become so trivial in a dynamic language (examples used Groovy and Ruby) that it is almost not obvious you are using the pattern. He also introduces a new pattern, common in dynamic language solutions, call the Type Transmogrifier Pattern, which has a goal of transforming types as needed as part of a fluent interface call, such as a DSL that allows an end user to request services in convenient units (grams versus ounces) and letting the code handle the conversion.

The Mylin talk was another session taken principally to scratch the itch of curiosity. You get Mylin in a standard Eclipse download, but what the heck does it do? From Brian's talk, it appears one excellent use is to provide an Eclipse-based interface into defect tracking tools like Trac and Jira. Talking to another attendee after the talk, it sounds like Mylin can be also be used for task tracking without the need for an external tool to interface with. I find integration with external tools to be the most compelling argument for me. But even that seems a bit weak unless there is organizational use of the tool. Still, a good talk and now I know...

Source code control is important. It is one of those things that every good developer I have worked with feels strongly about. Having worked in systems like ClearCase for many years, CVS (and later, subversion, which is CVS++) felt like a breath of fresh air when I started using it. But even these tools are getting a bit long in the tooth. Git is an SCM tool that is garnering a lot of buzz. Stu's talk was a great intro to the tool, which represents a bit of a paradigm shift to the traditional tools. At its heart, it is really a distributed file system with some SCM tools layered on top. One bit of trivia I had not heard before, git was written by Linus Torvalds. The selling point for git is how easily it supports distributed development, where you do not have access to a central repository all the time, and the ability to do lightweight branching, again without having to phone home first. This is a cool tool and one I intend to start using at home right away.

The Scaling talk was good, if a bit web-centric in nature. My biggest take out of the talk is that scaling is not necessarily equivalent to performance. A system that scales may actually degrade performance as load increases, but it does so in a linear and measurable fashion. Ken stressed the need to do load testing as means to validate that your system behaves in this fashion. Another key take away that seems obvious after you hear it is that any agreed upon SLA must include load along with response time. Otherwise, you are almost guaranteed to violate the SLA at some point. Also discussed was the fact that systems can scale horizontally (add more boxes) or vertically (add more memory, increase processor speed). Given that increasing processor speed these days really means adding more cores rather than increasing the clock speed, I think the distinction is a bit artificial. Given this fact, Ken stressed the need to understand concurrency.

Beyond the technical details presented at NFJS, one of the really cool aspects of the conference is the way the presenters share their vision of the where the industry is going. Two very common memes present this year were concurrency and the JVM as a platform.

As mentioned earlier, improvements in processor speeds is leveling off and multi-core processors are now the de facto mechanism used to increase the performance of a system. But this presents new issues for software. The only way a software product can benefit from the perfromance increase provided by a multi-core system is if it is designed to operate in a concurrent fashion. And worse, if software is not written to operate in a concurrent environment (that is, not thread safe), not only will it not see a performance improvement, but it may fail when run in these environments. In past conferences, there have been some great presentations regarding this. This year, rather than a focus on thread safe Java, presentations were made regarding functional languages that run on the JVM and are inherently thread safe. Scala and Clojure seemed to be the popular languages among the presenters this year. Venkat has a book in the works on Scala and Stu has a book (out in beta) on Clojure. I just picked up the clojure book. Will hopefully blog on it soon (the folks reading this who know me now what soon means). Neal has coined the term "Polyglot Programming" and advocates using the correct language for the job, e.g., Groovy/Grails for a front end, Scala for the back end to provide thread safety, etc. When folks complain about the complexity this adds, he points out that we already do this, since most projects have custom XML property files, Spring property files, Ant (or Maven) build files, Java source code, etc. In other words, developers are already used to this.

The other meme, the JVM as a platform, has been building steam with the presenters for the last three years and really seems to have arrived this year. Stu has a series of presentations he is calling the Java.next series. I attended the Common Ground session, in which he presented four up and coming languages, Groovy, JRuby, Clojure, and Scala, that run on the JVM. He spent most of his time discussing the common features of all of these languages and why any of them represent a better language solution than Java. Rather than relist these features, I just point you to Stu's blog on the topic, here. As an aside, Stu was a huge Ruby advocate last year. This year, he seems to have decided that Groovy is a reasonable language as well, but is really into Clojure right now. He made a comment about being attracted to bright and shiny new objects. I can totally understand that.

A blog post cannot even begin to impart all of the knowledge that is shared at a No Fluff. And beyond the sessions, there are great side conferences, networking, and food (really good food!). One of my favoite meta-activities is to collect entertaining sound bites. In conclusion, here are a few gems I picked up this time...

Never burn a bridge (unless the orcs are behind you)

A compiler tells you your code is well formed, a unit test tells you your code is well behaved

Developers are responsible for keeping their knowledge portfolio up-to-date

Increase your digital footprint, blog

A stack trace means nothing to a human (I laughed my rear off on this one)

Eat like a bird, poop like an elephant (i.e., consume information in large quantities, share information in large quantities)

Agile development means sustainable progress, not necessarily fast progress

Be resourceful, not a resource

Greenspun's 10th Rule applies to Java

Evangelize, be passionate!

Exploring Groovy's Optional Typing Features

So I was talking to a friend over a beer after the last BJUG. That night it was the Venkat Show. If you have never had the chance to listen to Venkat talk, all I can say is try harder, the guy is amazing. He opened the JUG with a discussion of interesting features of Java that might have escaped a developers notice (or were forgotten along the way). His second talk was on testing with Groovy. As you will soon see, the testing topic turned out to be helpful with this blog.

Note: code for this blog can be downloaded from here.

After the jug, we started discussing Groovy and dynamic programming in general. As folks who know me well can attest, I am a card-carrying, drank the dynamic cool-aid, Groovy fanboy. So when my friend told me of something he learned at a recent NFJS seminar (he probably brought it up just to rub in the fact that he was there and I wasn't, but I digress), I had to re-think my choice of next gen language. Fear not, the story ends well...

Here is the basic challenge he threw down: "Write a Groovy method that takes a statically typed argument of List. Now call this method from another method, supplying an argument of type Stack. It will work just fine, type mismatch errors."

I didn't believe that this would work. I know Groovy is a dynamic language, but due to its close affinity with Java, I had believed that although optional, when type was provided, it would be honored. So rolling up my sleeves, I wrote some code (See README and code in test1 directory).

class GroovyAdder
{
  def doit(List list)
  {
    println "In list version of doit"

    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }
}

and then a Groovy test for this class (I told you Venkat's talk would be relevant):

class TestGroovyAdder extends GroovyTestCase
{
  def adder

  void setUp()
  {
    adder = new GroovyAdder()
  }

  void tearDown()
  {
    adder = null
  }

  void testList()
  {
    List list = [0,1,2]
    assertEquals 3, adder.doit(list)
  }

  void testStack()
  {
    Stack stack = new Stack()
    stack.push(3)
    stack.push(4)
    stack.push(5)
    assertEquals 12, adder.doit(stack)
  }
}

Running this test, I find that the original statement was true, calling the method with a Stack argument works.

.In list version of doit
.In list version of doit

Time: 0.249

OK (2 tests)

I was so confused! This brings us back tangentially to Venkat's first talk. While the problem is not something he addressed, it does fall into the category of "things forgotten". It took me a while, but eventually, I came around to the realization that Stack implements the List interface. Ah, a Stack IS A List, so all is well. To verify this, I added the following test (See README and code in test2 directory):

void testQueue()
{
  Queue queue = new PriorityQueue()
  queue.add(6)
  queue.add(7)
  queue.add(8)
  shouldFail(MissingMethodException) { adder.doit(queue) }
}

A Queue does not implement List, so if the typing information was honored, this test should fail, and it did! A MissingMethodException exception was thrown. Problem solved, I sent the solution to my friend. He agreed that this solved the initial problem, but also added that it seemed to be a royal pain that I had to run the program to see the problem, when it could be (and is in Java) detected during compilation. He's right if we are talking about a statically typed language, but Groovy is really a dynamic language at its core. The amount of compile time checking it can do and still adhere to the dynamic mantra is limited. Let's explore this a bit (See README and code in test3 directory).

First, lets just add a second method (and a couple of println's) to the Groovy Adder class:

class GroovyAdder
{
  def doit(List list)
  {
 println "In typed version of doit"
 
    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }

  def doit(def list)
  {
    println "In def version of doit"

    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }
}

We also need to change the test that uses Queue (hint, the call will succeed this time):

void testQueue()
{
  Queue queue = new PriorityQueue()
  queue.add(6)
  queue.add(7)
  queue.add(8)
  assertEquals 21, adder.doit(queue)
}

Running this test, here is the output we get:

.In list version of doit
.In list version of doit
.In def version of doit

Time: 0.226

OK (3 tests)

This is way cool. Groovy was smart enough to choose the typed version of the polymorphic method when it could and the dynamic version when nothing else could be found.

But, of course, this still could be decided at compile time. Let's try one more thing (See README and code in test4 directory). First, change the uninteresting method we just added as follows (it is now a typed method as well):

def doit(Queue list)
{
  println "In queue version of doit"

  def sum = 0

  list.each
  {
    entry ->

    sum += entry
  }

  return sum
}

So now we have a Groovy class, where the author provided for the use of Lists (which gives us Stack by default) and Queues. Of course, now I want to use a Set. I'm never happy. So let's add the following test and associated closure:

void testSet()
{
  def expando = new ExpandoMetaClass(GroovyAdder)
  expando.doit = dynamicDoit
  adder.metaClass = expando

  Set set = new HashSet()
  set.add(9)
  set.add(10)
  set.add(11)
  assertEquals 30, adder.doit(set)
}

Closure dynamicDoit = { Set set -> 
  println "In set version of doit"
  def sum = 0

  set.each
  {
    entry ->

    sum += entry
  }

  return sum
}

We have now crossed the boundary and are firmly entrenched in dynamic land. The compiler can't save us here. Prior to testing with the set, the test code alters the metaclass of the object we intend to test, adding a dynamic method that knows how to handle sets (it could just as easily have handled Iterable or even def and be more general, some folks never learn). Here is the output from running the test now:

.In list version of doit
.In list version of doit
.In queue version of doit
.In set version of doit

Time: 0.23

OK (4 tests)

As stated earlier, the compiler can't help when the code can dynamically change after compilation, and further, if the compiler tried, it would hinder the dynamic features of the language.

So there you go! But wait, there is more. As an added bonus, let's look at what happens when we are using Java (See README and code in test5 directory).

First, we create a simple Adder POJO:

import java.util.List;

public class Adder
{
  public int doit(List list)
  {
    int sum = 0;

    for (Object entry: list)
    {
      sum += (Integer)entry;
    }

    return sum;
  }
}

See the java test code on the download site for an example of how you get a compile time error if you attempt to pass a Queue into this doit method of this class. You can't get there from here, well at least not in Java. So let's see what we can do when using this class in Groovy. Here is the Groovy test code:

class TestJavaAdder extends GroovyTestCase
{
  def lister = new Adder()

  void testList()
  {
    List list = [0,1,2]
    assertEquals 3, lister.doit(list)
  }

  void testStack()
  {
    Stack stack = new Stack()
    stack.push(3)
    stack.push(4)
    stack.push(5)
    assertEquals 12, lister.doit(stack)
  }

  void testQueue()
  {
    Queue list3 = new PriorityQueue()
    list3.add(6)
    list3.add(7)
    list3.add(8)
    shouldFail(MissingMethodException) { lister.doit(list3) }
  }

  void testQueueWithMop()
  {
    Adder.metaClass.invokeMethod =
      { String name, args ->

        println "In Groovy version of Adder.doit"

        if ("doit" == name)
        {
          def sum = 0

          args[0].each
          {
            entry ->

            sum += entry
          }

          return sum
        }
      }

    Queue list3 = new PriorityQueue()
    list3.add(6)
    list3.add(7)
    list3.add(8)
    assertEquals 21, lister.doit(list3)
  }
}

Running these tests gives:

.In Java version of Adder.doit
.In Java version of Adder.doit
..In Groovy version of Adder.doit

Time: 0.227

OK (4 tests)

The results from TestList and TestStack are not surprising, as they would work the same way in Java. But notice in testQueue we were able to call the statically typed Java doit method, which requires a List, with a Queue. And we did not receive a compile error! We only found our error at runtime.

The reason, again, is that if Groovy strongly enforced the type expectations at compile time, there would be no way to use the dynamic trick demonstrated in testQueueWithMop (MOP = Meta Object Programming), in which we use another Groovy mechanism to dynamically extend the features of a class, this time a Java class, without the need for source code.

The purpose of this blog was primarily to answer the question of how Groovy's optional type mechanism works and provide my understanding of why things work the way they do. In doing so, I made use of some of Groovy's MOP features. To learn more about these, I refer you to the sources I used. Scott Davis' book, Groovy Recipes, has a great intro to Metaprogramming, along with easy to use code snippets that illustrate the topic. Venkat also has a book, Programming Groovy, that delves even deeper into the topic. I highly recommend both books.

I also did not go into the debate of dynamic versus static typing. Folks a lot smarter than me are discussing this issue. I am principally a spectator to the debate. I will say that for DSLs, which I am very interested in, testing, and script use, the dynamicism of Groovy rocks! And I am tending to lean that direction for other code as well, as long as the code is backed by good unit testing.

Hmmm, guess I picked a side. Well, no one has ever accused me of not having an opinion.

Two very Groovy books...

Over the 2007 holiday, I had the pleasure of doing technical reviews of two upcoming books published by The Pragmatic Programmers. First I reviewed Scott Davis' book, Groovy Recipes: Greasing the Wheels of Java by Scott Davis. I followed this up with a review of Programming Groovy: Dynamic Productivity for the Java Developer by Venkat Subramaniam. For the reader in a hurry, I will summarize by saying if you are just learning Groovy, Scott's book is a must have. If you have the Groovy Basics in hand, then Venkat's book will guide you to the next level. I highly recommend both books.

I met both Scott and Venkat at the Boulder Java Users Group. Scott both runs the JUG and is a frequent speaker. Venkat visits at least once a year to present talks.

The first talk of Scott's I had the pleasure of listening to dealt with agile software development. Scott is an amazing speaker whose excitement for the topic he discusses is truly infectious. I had already decided agile was the right way to go and Scott's talk helped me along the true path. I mention this because it was another talk of Scott's that introduced me to Groovy. Scott made two comments that caught my attention. First, he stated "Groovy is what Java would look like had it been written in the 21st Century." That is a pretty powerful statement. Second, he answered the question of why Groovy versus Ruby... the answer being the context switch is much easier for a Java developer using Groovy.

After listening to the talk, I downloaded the Groovy package and started playing with it. I soon realized there was a lot of power in this language. So I bought Groovy in Action, a.k.a. GINA, and really started digging into the language. At a later JUG, Scott mentioned that he and Venkat were planning to write their own book on Groovy. The language was evolving quickly and GINA was already getting out of date, so I started bugging Scott about the upcoming book. Long story short, eventually, the authors decided that they were each envisioning a different type of book and when Scott had a rough draft available, he asked me if I would be interested in being one of the technical reviewers. I jumped at the chance.

Scott's approach to the book is very agile, in that, after a brief discussion of the Groovy language, it essentially answers lots of very specific questions regarding "How to..." in Groovy. Each section starts with a code snippet that is designed to be copied directly into your code, letting you quickly solve a problem (e.g. parsing XMl documents) and move on. The detail in these examples ensures that this book will remain useful for some time to come. Along with the toolset you get, these examples really demonstrate the power of the Groovy language and provide an excellent way to learn the language.

While perhaps not as all encompassing a primer on Groovy as GINA, Scott does a good job, in the early part of the book, presenting most of the language's features in sufficient depth that I am comfortable suggesting this as your first book on Groovy. And once the basics are covered, Scott goes on to present some excellent examples of how to use idiomatic Groovy to solve common programming problems, including file I/O, XML processing, Web Services, basic metaprogramming, and a nice introduction to Grails, the web framework built with Groovy. After the Grails intro, Scott presents some nice examples of how to use Grails in a similar manner to his treatment of Groovy.

My approach to the review was to first read the text and understand what concept was being presented, followed by a test of the supplied code. This turned out to be very informative. There are three key ways to dynamically execute groovy code; as a script file from the command line, in the groovy shell, and in the groovy console. It turns out that the shell has slightly different execution behavior than either the console or script file approaches. This has to do with the fact that the shell has a binding that controls the visibility of variables that does not explicitly exist in the other environments. I found this issue and discussed with Scott. I have not seen the final version of the book, but if this is covered, it will represent my key contribution to the book.

Scott (or at least his publisher) must have liked the work I did on the book, because after I completed the review, I was asked to review Venkat's book.

I have had the opportunity to listen to Venkat speak on numerous occasions and at one point, described my self as a Venkat groupie! Watching Venkat speak is a truly humbling experience. Not only does he have an amazing ability to describe complex technical concepts clearly, but he does this while writing code to demonstrate these concepts at the same time. This ability to communicate crosses over to his books and his latest is no exception.

While the book does cover some of the basics of Groovy, Programming Groovy is probably not the best book to use for learning the language. Venkat spends more time covering the interesting "edge cases" and "gotchas" of the language than on the basics. For example, he goes into the details of how Groovy's == operator is not the same as == in Java. For someone who already has a basic understanding of the language, the exploration of these more advanced topics is a great way to move to the next level in mastering the language. But it is Venkat's exploration beyond these topics that make this a critical reference for the developer serious about learning about the paradigm shift that dynamic programming entails.

Venkat spends time comparing the strong type checking in Java to the optional typing that exists in Groovy. He makes the case for why the strong typing, while it provides the semblance of safety for the developer, is really very weak protection. he then carries this topic forward and discusses closures, which, when combined with dynamic typing, form the first building block for dynamic programming. He then proceeds to go into the most important topics in the book, Unit Testing and Metaprogramming. With Metaprogramming, which is the ability of a program to manipulate itself, it is possible to create new domain specific languages (DSLs) and sophisticated builders that simplify other programming tasks. Venkat provides great examples of how to use these techniques in Groovy programs.

Dynamic programming is becoming increasingly relevant to modern software development and Programing Groovy is a great introduction to these concepts along with concrete examples in Groovy.

My approach to reviewing this book was basically the same as for Scott's book. Examine the examples and make sure they make sense and they work. Hopefully, Venkat and his publisher found my comments helpful. I really enjoyed this opportunity and hope I have the chance to do more reviews in the future. It is a great way to learn a new technology, since you really have to focus on the details.