JEKLblog

Going Solar (It Rocks!)

2011-06-15T13:21:00.003-06:00

Almost a year ago, we went solar on our house. I tweet quite a bit about it and get asked lots of questions. It has taken a year, but I finally got around to putting this together to help answer some of these inquiries. My hope is that it will prove informative to all who are looking at solar on their own house.

We got quotes from the following companies:

Namaste - has lease option (SunRun)
REC solar - has lease option (SunRun)
Lighthouse - has lease option (their own leasing plan)
SimpleSolar - Did not have a lease option
RealGoods - has lease option (SunRun)

In addition, I contacted Bella, but they never got back to me.

I originally intended to purchase the system outright. Doing so, you get a 30% Federal tax credit, a Colorado State tax credit for your sales tax, and a rebate from Xcel based on the size of the system.

The Xcel rebates are based on meeting some predetermined goals as far as amount of electricity generated by solar systems. The rebates are based on two components. A flat payment per watt plus an additional Renewable Energy Credit (REC). The rebates are on a sliding scale which is decreasing over time. Had we purchased, we would have got in at $2/watt + $0.55/kW (and this is what Xcel paid the leasing company). Xcel is currently paying $1.75/watt + REC of $0.04/kW. See Xcel Solar Rewards for more info on this. If you are thinking of going solar, get in while the getting is good (the great getting is already in the past). A final note, Xcel will only reimburse you for systems that are less than 10 Kw and less than 120% of your last years use.

If you generate more power than you use, it is banked on a monthly basis. In January, if your bank is positive, you have two options, A.) keep it banked, B.) Sell it to Xcel for the Average Incremental Cost of Electricity (AHIC). Previous years AHICs:

2010 - 3.000¢/kWh
2009 - 3.058¢/kWh
2008 - 4.842¢/kWh
2007 - 3.414¢/kWh
2006 - 4.291¢/kWh

Note: If you choose Option A, it is forever, you can't go back. We have currently waived the decision, defaulting us to B. Given that we generate significantly more electricity than we use, banking the extra doesn't make much sense for us (we would never use it), so we will likely continue with option B in the future.

Even with the rebates, these systems are expensive. After looking at the cost of financing, we determined the payback would be minimal. So we looked at the leasing option. This turned out to be lucrative for a few reasons.

Here is why I like the lease:

Low upfront payment, fixed monthly payments for life of lease
Lease is for a guaranteed amount of energy generation per year
Leasing company takes care of insurance and maintenance of system. The inverter is going to go out in 10-12 years, they will have to replace (around $3500). Hail damage, they have to repair or replace.
If we generate more power than we use, it is ours, even if it exceeds the guaranteed amount. This extra can be banked or sold back to Xcel at our discretion.

Given the above, some things to consider with the leasing option:

Leasing company gets all of the tax credits and rebates
Slight headache if you sell your home, in that you need to get your buyers qualified for the lease (or prepay lease, or buy out system). It looks like SunRun makes this very easy, so this is a minor point.

As noted above, with one exception, all of the companies we talked to used SunRun as their leasing provider. Since we signed on the dotted line, two other leasing companies have arrived on the scene, Sunergy and Solar City. I do not have any experience with either company, but probably worth talking to them. Also, it is worth noting that these leasing companies only operate in states where the utilities have programs to make it lucrative to install systems. I know this after looking into the issue for some relatives in WY and UT.

The system we chose was a 7.56 kW system, installed by Namaste Solar and leased from SunRun. I was very happy with every company I talked to (except for Bella, who couldn't be bothered to even return my call). In the end, the decision was based on two things. I asked each company to "max" my roof out, figuring that as long as I was jumping in, I should go all the way. I also ran financials on each proposal, factoring in the downpayment, the leasing cost, our average usage based on the past year (1100 kWh per month), and assumed a 5% annual increase in the cost of electricity.

The Namaste proposal won out on both counts. My simple (and it turns out, conservative) financial analysis indicated we would save $20K over the 20 years. This system had the largest downpayment ($1800) due in part to the need to move a vent on the roof to accommodate the panels. The vent movement was the tactic that allowed them to propose the largest system. The system is guaranteed to generate 11 MWh a year (after 10 months, the system has averaged 1 MWh/month, so looking at 12 MWh+ for first year), which is approximately 80% of our 2009 average usage. Payments are $76 a month for 20 years.

To compare, this system would have cost us $28K to purchase outright. I am pretty sure this is after Xcel rebates, but before tax credits, but those details are a bit hazy, as I stopped looking at the purchase very early.

One other nice to have from Namaste that none of the other installers quoted was a screen guard around the base of the panels on the roof. This will keep critters out from under the panels and off of our roof.

So how are we doing? The system went live the last week of June, 2010. Since then, we have been net producers, generating more electricity than we have used. Since turning the system on, we have generated a total of 10,204 kWh energy, of which 3,338 kWh were put back on the grid (as of May 2011). Translation, for the past ten months, we have generated more power than we have used, by a significant amount!

But wait, I mentioned that the system was only sized for 80% of our average use last year? Well, at the same time that we got the system, we made some changes around the house to reduce our usage. Anyone who knows me knows that I like it COLD. Normally, my AC thermostat would be set to 70 and sometimes 68. This summer, we set it to 74 during the day and 72 for four hours in the evening. I know that this doesn't sound like a sacrifice, but those closest to me (and those who have observed how I dress in the winter) know that I have been less comfortable than I would have liked. The other change we made was to use small fans around the house to recirculate air instead of the HVAC fan. These changes have resulted in a considerable drop in our energy usage at the same time that we have added generation capability to our home. Net win!

Prior to getting the system, our electric bills averaged $120/month. Since we generate more than we use, our new bill should be just the lease payment minus any money we get from Xcel buying back the excess. This is almost the case, but there is one added fee, which is an $8/month charge for being connected to the Xcel grid. There has been a lot of complaint over this fee from some quarters, but I can understand the rationale. I am using Xcel's equipment both to transmit my excess to the grid and to augment my supply when the system is not generating. I think the fee is reasonable and even taking it into account, we are still already money ahead of where we were this time last year.

Finally, let me describe the system. On the roof, we have 24 SunPower 315-watt panels. These panels are all wired up and feed DC power into an inverter on the side of the house. This unit converts the DC power to AC power. This is fed into a meter used by SunRun to monitor production. A second meter follows the SunRun meter and is owned by Xcel to do the same thing (because they don't trust SunRun's meter!). Following this, the power is fed into our breaker box. The main house meter was replaced with a NetMeter (it runs forward and backward). This last meter is the one we are billed from. In addition, I have added a Ted5000 system to allow me to monitor our power generation and use in real time. The whole thing looks a bit Rube Goldberg in nature, but it works!

So during the day, the system generates power if the sun is shining. Namaste sent me an informational email informing me that the system would not generate power at night or on extremely cloudy days. That was almost a deal breaker, but we decided to continue! ;-) At night, we pull our power from the grid. As long as we pull less power from the grid than we put onto it during the day, our house is a power generation plant.

One more item to note. We do not have a storage system (batteries). Such systems are extremely expensive and have to be periodically replaced. The grid becomes our storage system, which is fine, but there is a caveat. If the grid power goes down, our system will shut off. This is to ensure that our system does not put power back onto the grid while someone is working on it, causing injury or death. So when the power is out for our neighbors, it is out for us.

I will continue to modify this as I receive questions and clarifications. I hope it provides useful information on going solar! Feel free to send questions my way and I'll help if I can.

If you choose to go solar and select Namaste and/or SunRun, please mention you heard about them from me as they have referral bonuses, and hey, any little bit helps.

My Ignite Talk Experience

2010-02-07T13:08:00.007-07:00

20 slides, 15 seconds per slide... a 5 minute talk. How hard could it be? Famous last words!

That was what I signed up for when I asked to present at the Denver Open Source User's Group Lightning Talk Night.

I had my topic in mind. I wanted to provide an overview to the basics of estimating tasks in an agile development. There are many things I find compelling about agile development philosophies, but none more so that the elegant simplicity of how estimates are made and progress tracked.

It might sound strange, but the first thing I focused on was a title. My thought was that for a five minute talk, the title would be an important way to set the tone for the talk. I started thinking about what I would be talking about and the title just seemed to fall in place. I took this as a good sign.

I then proceeded to procrastinate for three weeks (actually, I was slammed at work, but still...). The talk was a couple of weeks away and I knew I had to get cracking. Evidently, my subconscious agreed and had actually been on the job, because I woke up at 3 AM one morning with the entire outline of the talk running through my brain. I quickly grabbed my iPhone and started frantically typing into the notes App while everything was still fresh. I was surprised when I reviewed it the next day to find that it was a good outline. The final one did not deviate much from the original.

With outline in hand, I then started making my slides. I tried to go for a very simple title that conveyed the message of the slide and a graphic that supported that message, albeit in an abstract manner in some cases. Since I wanted to release the talk under Creative Commons, I had to search a bit for appropriately licensed media. FWIW, there are some very talented folks on the interwebs that totally ROCK!

After I had the slides completed, I wrote a script for the talk. This consisted of writing 2-3 short sentences for the slide, then speaking the sentences while timing to see if they fit within the 15 second allotment. There was a lot of iterative editing in this step. I wound up with a talk that was, in hindsight, a bit denser than it should have been. It fit the time limit, but barely.

I practiced the talk and thought I had it down fairly well. I had the script for each slide on a 3x5 card, just in case something happened. In practice, I found that glancing at the card was sufficient to guide me through the talk when I got stuck.

Then came the live performance. It was not the rousing success I had hoped for. I slipped up on my timing early in the talk. At 15 seconds per slide, it is very hard to recover from this, particularly when your talk is info-dense. I wound up speed reading my cards to keep up. I'm not saying it was a complete failure, but I would have liked it to go a bit smoother.

As far as the night in general, there was some amazing talks on a lot of different topics. This format is such a great venue for getting a peek at many different ideas and technologies. Slides for the talks are posted at DOSUG Slides.

A couple of days after the talk, I got the chance to reprise it at Tendril. It went a lot better (still not perfect).

The whole experience has been both educational and fun. If given an opportunity, I highly recommend you give it a try.

You can review my talk, with audio, here. Let me know what you think.

Also, full disclosure, the recorded talk is slightly longer than the 5 minutes allowed. This is mostly due to the fact that the introduction was not counted as part of the talk.

Agile Estimation: The story has a point, but the point is unitless

No Fluff Just Stuff, Fall 2009

2009-11-22T20:34:00.003-07:00

Another year, another NFJS behind me. You would think by now it would be old hat, but it seems like every time I attend, by the end of the weekend, I feel reenergized and ready to crank on some more code. My LOTTD also seems to get a lot longer!

Jay Zimmerman, the founder of No Fluff, just gets it and always manages to put together the perfect geek conference. The speakers are top notch, the topics are relevant, and the food is great. This year, the wireless networking was top notch as well!

Before I get into the sessions I attended, I want to go into one of the less tangible benefits of the conference that folks thinking of attending may not think of, which is the networking opportunities. My weekend started with lunch with some friends, one who is a speaker, prior to the start of the conference. While we don't focus our conversations exclusively on technical topics, they tend to dominate. In this case, I learned a lot about GIS that I was not aware of before.

Similar things happened during conference meals. The speakers are great for mingling, rather than being clickish, and again the discussions were very informative. One of the memes this year was whether speakers should use live coding or pre-recorded demos in their talks. The two camps had strong opinions on the issue and there was a lot of friendly ribbing that went on between them.

Personally, I got to see the good and bad for both approaches. In one talk, the speaker found that some web services APIs provided by Yahoo! he wanted to demonstrate had changed, which broke his demo. He was a bit embarrassed by this and it is possible that some folks in the session were unhappy, but I felt that most attendees were very engaged and it became a group debugging sessions, which in itself is a great learning experience. In another session that used live coding, a question from the audience sent the talk in a slightly different direction. Had the demo been canned, this would have been harder to do.

On the other side though, one of the speaker's Mac crashed right before he was scheduled to talk. Because his talks were canned, he was able to borrow a laptop and go through his talk without missing a beat. If he would have needed to install all of the tools used in his demo, it would have been a massive FAIL.

Jay summarized the debate saying if everyone agreed, someone would be redundant!

And now a summary of the talks I attended...

Spring 3, Groovier Spring - It has been a couple of years since I attended a Scott Leberknight talk. He has kept his tech edge and did great job of reviewing some cool features in Spring 3 (and some lesser known features from Spring 2.5). I'm not a Spring power user, but have some grand plans for incorporating Spring into the product I am currently working on. The plan is to use Spring Dependency Injection to allow third parties to extend the product without the need to access our source code. None of the features discussed in the talk seemed to add any capabilities that I would immediately need for this. My biggest take away is that the Spring platform is very powerful, but borders on being overly complex as well.

That being said, there are some cool features available in Spring. Annotations seem to have taken the place of most XML config. In general, I like this. Spring has added extensive support for ReSTful web services, including a work-around for the fact that browsers do not support the PUT and DELETE primitives. Spring has great support for TDD, including Junit 3 and 4 as well as TestNG. Transactional processing support is extensive, again the idiomatic use appears to be via annotations. The other big take away from this talk was that you can add a global exception handler to a Spring-based system.

Scott also presented information on a groovier Spring. I like Scott's definition of Groovy...

"Java minus all the code I don't like writing plus closures, metaprogramming and more."

SpringSource acquired G2One last year (SpringSource has since been acquired by VMWare). Based upon the first acquisition, a lot of folks felt that Groovy (and Grails) would get baked into Spring. These people were correct! Spring now has a a script factory object that supports beanshell, jruby, and for the purposes of this talk, groovy. Scott presented great examples of how to incorporate groovy scripts that can be run by the Spring app. There are several (too many?) mechanisms that can be used to do this. Fortunately, one of them appears to be the most straightforward. There are some security issues that need to be accounted for if you want to use this mechanism. In one example, the script is pulled out of a DB. Injection attack anyone?

Scott likes to open his talks with a joke, typically some kind of pun. This time, he started with an oldie, but a goodie. What is the object oriented approach to getting rich? Inheritance! But someone in the audience threw a curve ball I had not heard before... Marry a rich Singleton. LMAO!

Grails w/o GORM and ReSTful Grails - Scott Davis, the person who introduced me to Groovy and Grails, did a great job on these two complimentary talks. Both talks focused on using ReST in Grails, but from different perspectives. The first talk was about how to consume ReSTful web services. Scott demonstrated accessing Twitter, Yahoo Search, and a GIS web service from within a Grails application. The code to do this was surprisingly simple.

From the other side, Scott demonstrated how to build a Grails application that exposes a ReSTful web service. His focus in this talk was that while the UI is important, don't forget the data! As expected, it is very straightforward to implement a ReSTful interface in Grails. It turns out that there is a bit of boilerplate involved, so Scott recommends moving much of this to template functions.

I blogged about ReST last year. It is really taking off as the web service technology of choice. Scott compares ReST, which is resource (noun) oriented, to SOAP, which is service (verb) oriented. He presented stats which show that give a choice, consumers of web services overwhelmingly choose ReSTful interfaces to SOAP based ones.

Polyglot Persistence - Another Scott Leberknight talk. There is evidently a movement afoot, the No SQL Movement, which advocates the position that while SQL is not Evil, it is often not appropriate. In this talk, Scott presented some potential alternatives to the traditional RDBMS. These include document-based (CouchDB and Amazon SimpleDB), hash table (Project Voldemort), and big table (Google Big Table and Apache HBase) approaches.

I'm not a persistence guy. Most of my experience is with some simple Hibernate apps and GORM in Grails. It was great to see what else is out there, but my biggest take away from the talk is I'm not smart enough to work at Google. That Big Table stuff is pretty deep!

Git - Matthew McCullough has been on the No Fluff circuit for about a year and is really making his mark. He is a great speaker and really knows his stuff. He is also the speaker who had to recover from a crashed Mac.

Git is a distributed version control system written by Linus Torvalds of Linux fame that is quickly becoming the VCS all the cool kids are using. While an argument could possibly be made that some of this is hype, I think there is enough value add to the product that the hype is justified. In my case, the value is the ability to have the entire repository stored locally on my box, allowing me to work offline and still have access to the entire history. Matthew did a great job demonstrating how to use the Subversion to Git gateway. I am looking forward to trying this out.

The syntax is a bit quirky, as might be expected from the inventor of Linux, but it is easy to do the simple things and possible to do some powerful things!

Open Source Debugging Tools - Matthew provided a whirlwind tour of a dozen tools for debugging Java applications and another dozen or so tools for debugging web apps.

In the Java space, he provided summaries of tools for process debugging, stack and heap dump analysis, disassembly and decompilation. One of the tools that really excited me was the Eclipse TPTP comprehension tool. This tool monitors program execution and creates UML sequence diagrams that reflect the actual execution of the program. This tool can even show execution of dynamic methods created in Groovy objects. I asked about tying this tool into a CI server for automatic creation of developer documentation. The product is not there yet, but I can see this happening in the future.

In the web app space, the tools of note (for me) were curl, firebug, and firebug lite. I hope to find some time to learn these better. Matthew made the point that many developers find System.out.println to be the end all and be all of debugging. I definitely don't want to be in that camp!

Refactoring - Neal Ford has been a mainstay on the No Fluff tour since I have been attending (4 years now!). He often focuses more on the bigger picture of software development as opposed to specific technologies. This talk fits this description really well. Some of this was review from what I had learned reading the seminal refactoring book by Fowler, but there were a couple of good take aways from the talk.

Prior to refactoring, all that you typically have is piles of undifferentiated code. Following the techniques Neal presented, including the composed method and template patterns, will result in code that is more extensive and maintainable.

One soundbite I took from the talk was "It is OK for the framework to know about your code, but your code should not know about the framework".

Collections - This session was presented by Ted Neward, master of everything low level on the JVM (other stuff as well). I have been using java.util.* collections quite a bit lately. I have been frustrated by the lack of some seemingly obvious features, such as creating a list from an array. I went to this class to see if I was missing something. To some extent, I was. The Collections and Arrays classes have some of the features I was looking for, albeit with some warts that still make them less useful than they could be. And some features are still missing (I cannot directly create a List from a List). One take away from the class is I need to go look at Commons Collections.

Ted joked that every time a Java programmer uses an iterator, a kitten dies. Proper use of collections leads to a more functional programming style. I'm not really a cat person, but don't advocate cruelty either, so I really want to move that direction!

FWIW, closures and Groovy are another means to this end as well. So I'm already on the way.

JMS - Another Ted talk (pun intended). I had some very brief exposure to the JMS messaging bus a couple of years ago, but never got into the details. I try at every No Fluff to get some exposure to a technology or two that I may not necessarily need, but want to have a passing understanding of. So my take away from this session, JMS is a standards-based message passing protocol that handles both point-to-point and pub/sub messaging models. One of the benefits (provider dependent) can be guaranteed delivery. The API has some quirks, but seems pretty straightforward. There are many JMS providers out there. Ted stressed not coding to a specific provider jar, but rather compiling with the jar shipped with the JDK, then deploying with the provider jar to ensure you are adhering to the standard. The biggest disappointment was finding out that JMS messages from one vendor are not typically cross-compatible with other vendors.

So that's a wrap for this year. My head is full, there aren't enough hours in a day, but even with these constraints, I always feel like I have improved as a developer just by attending the show. And after I spend some playing with some code... look out!

I'll close with a teaser. Jay is going to have some really cool news for release in the next couple of week. I'll tweet/blog when he does. I don't want to steal his fire, so until then, I will leave you in suspense!

Brain Quenched, Actually Soggy to the Point of Leaking.

2009-03-08T21:20:00.002-06:00

In my last blog, I suggested that if you wanted to learn Groovy or Grails, that Scott and Andrew at ThirstyHead were the people to talk to. Well, last week I put my money where my mouth was and attended the inaugural class from ThirstyHead, "Advanced Grails", taught by Scott in Lone Tree, CO.

With the exception of a hard drive crash on my part that resulted in switching laptops mid-class, everything went fantastic. I really learned a lot in three short days.

I have extensive experience working at many levels of software development, from writing assembly and embedded C on 8-bit processors, to developing hard real time systems using a commercial RTOS (in C & C++), to multi-threaded Java development. One co-worker recently called me the Swiss army knife of programmers. I really liked that description. But the compliment of tools in this knife is not complete by a long shot and I am always looking to add new ones. Web development is one item lacking from my repertoire. Enter Grails.

A web framework targeted for the JVM, most first reactions tend to be "Hmmm, yet another web framework". The distinction Grails brings to the picture is a framework that embraces convention over configuration, dynamic programming, built-in support for testing, and tools that fit well in an agile shop, all this in a framework that is inherently Java friendly. While there is no arguing with the fact that Grails and it underlying language, Groovy, borrowed a lot of inspiration from the Ruby on Rails framework, it is relevant in its own right due to its strong integration story with Java. For a beginning look at Grails, check out Scott’s Mastering Grails series on IBM DeveloperWorks. There are also some good books out there, Grails in Action and The Definitive Guide to Grails. After you are comfortable with the tutorials, you have the basics under your belt, and you want to move to the next level, Scott’s class should be a prime consideration.

The goal of the class was to do a deep dive into the inner workings of Grails. In Scott’s words, by the end of the class, he wanted us to not only know how to use Grails, but how to make it sing… to a tune of our own selection. I think he got the job done.

We started with a deep dive into skinning a Grails app. Step one was looking at some existing CSS templates that are available via Creative Commons. After choosing one, we learned how to skin our app with it, modify the stock GSP (Groovy Server Pages, similar to Java Server Pages). In true agile fashion (Scott is an expert on Agile development as well), we added a feature at a time, first adding the correct tags and labels to the GSP and then learning how to break it into reusable templates. The View aspect of the MVC pattern is one of the areas I am weak in, so this section was extremely helpful. I will be attending the AJAX spike offered by ThirstyHead in April to further expand my knowledge in this area.

The next section covered Authentication and Authorization in a Grails application. Scott demonstrated, in a remarkably small amount of code, how to roll your own basic A&A module using Grails interceptors. He followed this up by introducing two popular security plugins, Authentication and JSecurity. Authentication looks fairly straightforward and provides some bells and whistles that would require more effort to develop from scratch. JSecurity appeared to be the nuclear option for security, powerful, but complicated.

Scott spent considerable time covering how Grails supports ReSTful web service support, including time spent looking at other ReSTful web sites. In the special case of sites that only provide read access to their data, Scott has coined the term GETful (since the only HTTP verb supported is GET) to describe the interface. ReST is becoming extremely popular due to the ease with which it can be implemented without having to compromise application functionality. Grails makes a ReSTful app very easy to implement.

We then covered several miscellaneous topics, including codec development (allows data conversion services to translate data to and from strings), operator overloading (supplied via Groovy), file upload, supplying syndication services from a site using Atom, indexing using Lucene and Compass, and integrating support for e-mail and JMX monitoring into your application.

We finished up the last day by digging into the details of the Grails build system, which is based on Gant, the Groovy DSL that wraps Ant. In this section, we learned how to extend the build system to perform custom actions during a build (like packaging special files into the jar or war file).

This was a LOT of info to fit into three days. Surprisingly, at no time did I feel that Scott had to rush things and we even had time to take a couple of detours based on questions from the class. The most memorable was when we figured out how the Twitter ReST interface worked. There was ample time to work on the lab exercises, which were written to build upon and reinforce the lecture material. Scott was always available when things went horribly wrong. The labs really brought everything home.

Another cool take away from class was that Scott bundled up all of the code he wrote during his lectures. Most of this code is commented to help remind us what the key points were for that code. This code is going to be a great resource as I start to use the material I learned.

Given what I already knew about Scott, his knowledge of the subject matter, excellent speaking skills, and cool sense of humor, I can’t really say I’m surprised about how well the class went. I’m looking forward to attending the tech spike in April.

If you are in need of a quick jumpstart in any of the topics they offer, try ThirstyHead. You won’t regret it.

Quench Your Head's Thirst!

2009-01-14T08:36:00.002-07:00

I am pleased to report that a good friend and really smart guy, Scott Davis, has formed a new company, ThirstyHead, with another great industry icon, Andrew Glover.

The company will provide top notch training focused primarily in Groovy and Grails initially, but if I know anything about these guys, we can expect to see other great offerings as well. Besides their excellent knowledge in all things Groovy and Grails, Scott is the GIS guy in the Java circles (he wrote the book!) and Andrew has done some amazing work in Behavior Driven Development (he wrote the code!).

If you have not had the chance to see Scott talk, you are truly missing a great learning opportunity and a fun time. Scott's speaking style is very informal and engaging. He is very passionate about the subjects he talks about and draws extensively from his own experience.

I have only had the opportunity to attend one talk by Andrew, but have listened to many of his podcasts on JavaWorld and read his entertaining Disco Blog. Based upon what I have heard, I think the Davis/Glover partnership will go far.

If you want to quickly bootstrap your understanding of cutting edge technologies that are bringing rapid, dynamic development tools to the Java platform, ThirstyHead is the place to go! I owe a lot professionally to Scott and wish him and Andrew much success in this endeavor!

The geeks were back in town...

2008-11-23T16:34:00.002-07:00

Jay Zimmerman and crew descended upon southern Denver for the Fall 2008 No Fluff Just Stuff symposium. And I had the opportunity to attend.

This is my fourth No Fluff and as usual, it was great. Eleven hour-and-a-half long sessions, a great keynote, and an entertaining and informative panel discussion, coupled with some great networking filled the weekend. My brains were leaking out of my ears by conference end Sunday evening.

Here are the sessions I attended (as always, the biggest complaint I have is that there were some hard choices to make when choosing sessions):

Beginning Drools - Rule Engines in Java, Brian Sam-Bodden
What's Going On? : Complex Event Processing w/ Esper, Brian Sletton
Rapid Web Development with Grails and Ajax, Scott Davis
Boosting Programmer productivity with Mylyn, Brian Sam-Bodden
Design Patterns in Dynamic Languages, Neal Ford
Java.next #1: Common Ground, Stu Halloway
Architecture and Scaling, Ken Sipe
Git control of your source, Stu Halloway
Agile Test Driven Development With Groovy , Jeff Brown
Real World Hibernate Tips, Scott Leberknight
Google Your Domain Objects With Hibernate Search, Scott Leberknight
Keynote - Soft Skills and Organizational Dynamics, Ken Sipe

Here are some rambling take-aways from the sessions. The goal is to peak curiosity, not repeat the preso. Feel free to drop me some e-mail or leave a comment if you would like to discuss further.

The first two sessions I chose based upon professional curiosity, since, in a past life, I worked on a commercial Complex Event Processor. Rules engines are similar in nature, so I wanted to compare the two. Turns out, the biggest difference is that in a CEP, time is a de facto part of the system (you handle data in time order order the vast majority of the time). By default, the Drools rules engine processes rules in the order they were entered into the system. Another difference is that typically, a rules-based system is used when you already understand the business logic you wish to use for processing the data, whereas a CEP system might be used to discover what those rules are. Keep in mind that these differences are a bit fuzzy. I believe you could use either system to solve most problems without any serious complications. It was interesting to see that the user interface of Esper was not radically different from the commercial product I worked on (the underlying implementation could be another matter).

Most of my technically oriented friends know I am a Groovy fanboy. For that reason, I sought to expand my knowledge of this dynamic language and attended a couple of sessions dealing with the language. Unfortunately, both talks were a bit basic for my needs. The Grails talk was interesting, but most of the time was spent covering grails, so it was review. Because of time, the integration with Ajax was a bit rushed and not given the detail I had hoped for. The good news is that Scott runs the local JUG, so there is hope he will present the Ajax integration there in the near future (hint, hint). The Groovy testing talk suffered from a similar problem. Too much time covering the basics and not enough time discussing the interesting issues you can solve using GroovyTestCase. In particular, I had hoped to see more about how to test Java with Groovy. This is given a a lot of lip service, but examples are less common. (aside: I have done a little with this, see Exploring Groovy's Optional Typing Features). Instead, two very basic examples of using Groovy to test Groovy were presented. for folks new to Groovy, this was probably helpful. Another case of a talk that needs to be broken into basic and advanced sessions.

Neal's talk on Design Patterns in Dynamic languages, while not specific to Groovy, provided my Groovy fix for the weekend. The take away from this talk is that design patterns really have two purposes. As a mechanism for communication with other developers, they provide a set of common semantics. Neal feels this is a "Good Thing"(tm) and doesn't think this should change. The second purpose, on the other hand, is to provide band-aids to languages that lack features that can directly solve the problem a pattern is targeted toward. Neal argues that in the dynamic language arena, this reason is less relavent. Referring to the Gang of Four book, Neal relates anecdotally that one of the authors once stated that the sub-title of the book could have been "Making C++ suck less." To back this up, Neal went through several patterns, starting with the Command Pattern, that require a lot of scaffolding in traditional languages, but become so trivial in a dynamic language (examples used Groovy and Ruby) that it is almost not obvious you are using the pattern. He also introduces a new pattern, common in dynamic language solutions, call the Type Transmogrifier Pattern, which has a goal of transforming types as needed as part of a ﬂuent interface call, such as a DSL that allows an end user to request services in convenient units (grams versus ounces) and letting the code handle the conversion.

The Mylin talk was another session taken principally to scratch the itch of curiosity. You get Mylin in a standard Eclipse download, but what the heck does it do? From Brian's talk, it appears one excellent use is to provide an Eclipse-based interface into defect tracking tools like Trac and Jira. Talking to another attendee after the talk, it sounds like Mylin can be also be used for task tracking without the need for an external tool to interface with. I find integration with external tools to be the most compelling argument for me. But even that seems a bit weak unless there is organizational use of the tool. Still, a good talk and now I know...

Source code control is important. It is one of those things that every good developer I have worked with feels strongly about. Having worked in systems like ClearCase for many years, CVS (and later, subversion, which is CVS++) felt like a breath of fresh air when I started using it. But even these tools are getting a bit long in the tooth. Git is an SCM tool that is garnering a lot of buzz. Stu's talk was a great intro to the tool, which represents a bit of a paradigm shift to the traditional tools. At its heart, it is really a distributed file system with some SCM tools layered on top. One bit of trivia I had not heard before, git was written by Linus Torvalds. The selling point for git is how easily it supports distributed development, where you do not have access to a central repository all the time, and the ability to do lightweight branching, again without having to phone home first. This is a cool tool and one I intend to start using at home right away.

The Scaling talk was good, if a bit web-centric in nature. My biggest take out of the talk is that scaling is not necessarily equivalent to performance. A system that scales may actually degrade performance as load increases, but it does so in a linear and measurable fashion. Ken stressed the need to do load testing as means to validate that your system behaves in this fashion. Another key take away that seems obvious after you hear it is that any agreed upon SLA must include load along with response time. Otherwise, you are almost guaranteed to violate the SLA at some point. Also discussed was the fact that systems can scale horizontally (add more boxes) or vertically (add more memory, increase processor speed). Given that increasing processor speed these days really means adding more cores rather than increasing the clock speed, I think the distinction is a bit artificial. Given this fact, Ken stressed the need to understand concurrency.

Beyond the technical details presented at NFJS, one of the really cool aspects of the conference is the way the presenters share their vision of the where the industry is going. Two very common memes present this year were concurrency and the JVM as a platform.

As mentioned earlier, improvements in processor speeds is leveling off and multi-core processors are now the de facto mechanism used to increase the performance of a system. But this presents new issues for software. The only way a software product can benefit from the perfromance increase provided by a multi-core system is if it is designed to operate in a concurrent fashion. And worse, if software is not written to operate in a concurrent environment (that is, not thread safe), not only will it not see a performance improvement, but it may fail when run in these environments. In past conferences, there have been some great presentations regarding this. This year, rather than a focus on thread safe Java, presentations were made regarding functional languages that run on the JVM and are inherently thread safe. Scala and Clojure seemed to be the popular languages among the presenters this year. Venkat has a book in the works on Scala and Stu has a book (out in beta) on Clojure. I just picked up the clojure book. Will hopefully blog on it soon (the folks reading this who know me now what soon means). Neal has coined the term "Polyglot Programming" and advocates using the correct language for the job, e.g., Groovy/Grails for a front end, Scala for the back end to provide thread safety, etc. When folks complain about the complexity this adds, he points out that we already do this, since most projects have custom XML property files, Spring property files, Ant (or Maven) build files, Java source code, etc. In other words, developers are already used to this.

The other meme, the JVM as a platform, has been building steam with the presenters for the last three years and really seems to have arrived this year. Stu has a series of presentations he is calling the Java.next series. I attended the Common Ground session, in which he presented four up and coming languages, Groovy, JRuby, Clojure, and Scala, that run on the JVM. He spent most of his time discussing the common features of all of these languages and why any of them represent a better language solution than Java. Rather than relist these features, I just point you to Stu's blog on the topic, here. As an aside, Stu was a huge Ruby advocate last year. This year, he seems to have decided that Groovy is a reasonable language as well, but is really into Clojure right now. He made a comment about being attracted to bright and shiny new objects. I can totally understand that.

A blog post cannot even begin to impart all of the knowledge that is shared at a No Fluff. And beyond the sessions, there are great side conferences, networking, and food (really good food!). One of my favoite meta-activities is to collect entertaining sound bites. In conclusion, here are a few gems I picked up this time...

Never burn a bridge (unless the orcs are behind you)

A compiler tells you your code is well formed, a unit test tells you your code is well behaved

Developers are responsible for keeping their knowledge portfolio up-to-date

Increase your digital footprint, blog

A stack trace means nothing to a human (I laughed my rear off on this one)

Eat like a bird, poop like an elephant (i.e., consume information in large quantities, share information in large quantities)

Agile development means sustainable progress, not necessarily fast progress

Be resourceful, not a resource

Greenspun's 10th Rule applies to Java

Evangelize, be passionate!

Exploring Groovy's Optional Typing Features

2008-07-20T22:23:00.004-06:00

So I was talking to a friend over a beer after the last BJUG. That night it was the Venkat Show. If you have never had the chance to listen to Venkat talk, all I can say is try harder, the guy is amazing. He opened the JUG with a discussion of interesting features of Java that might have escaped a developers notice (or were forgotten along the way). His second talk was on testing with Groovy. As you will soon see, the testing topic turned out to be helpful with this blog.

Note: code for this blog can be downloaded from here.

After the jug, we started discussing Groovy and dynamic programming in general. As folks who know me well can attest, I am a card-carrying, drank the dynamic cool-aid, Groovy fanboy. So when my friend told me of something he learned at a recent NFJS seminar (he probably brought it up just to rub in the fact that he was there and I wasn't, but I digress), I had to re-think my choice of next gen language. Fear not, the story ends well...

Here is the basic challenge he threw down: "Write a Groovy method that takes a statically typed argument of List. Now call this method from another method, supplying an argument of type Stack. It will work just fine, type mismatch errors."

I didn't believe that this would work. I know Groovy is a dynamic language, but due to its close affinity with Java, I had believed that although optional, when type was provided, it would be honored. So rolling up my sleeves, I wrote some code (See README and code in test1 directory).

class GroovyAdder
{
  def doit(List list)
  {
    println "In list version of doit"

    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }
}

and then a Groovy test for this class (I told you Venkat's talk would be relevant):

class TestGroovyAdder extends GroovyTestCase
{
  def adder

  void setUp()
  {
    adder = new GroovyAdder()
  }

  void tearDown()
  {
    adder = null
  }

  void testList()
  {
    List list = [0,1,2]
    assertEquals 3, adder.doit(list)
  }

  void testStack()
  {
    Stack stack = new Stack()
    stack.push(3)
    stack.push(4)
    stack.push(5)
    assertEquals 12, adder.doit(stack)
  }
}

Running this test, I find that the original statement was true, calling the method with a Stack argument works.

.In list version of doit
.In list version of doit

Time: 0.249

OK (2 tests)

I was so confused! This brings us back tangentially to Venkat's first talk. While the problem is not something he addressed, it does fall into the category of "things forgotten". It took me a while, but eventually, I came around to the realization that Stack implements the List interface. Ah, a Stack IS A List, so all is well. To verify this, I added the following test (See README and code in test2 directory):

void testQueue()
{
  Queue queue = new PriorityQueue()
  queue.add(6)
  queue.add(7)
  queue.add(8)
  shouldFail(MissingMethodException) { adder.doit(queue) }
}

A Queue does not implement List, so if the typing information was honored, this test should fail, and it did! A MissingMethodException exception was thrown. Problem solved, I sent the solution to my friend. He agreed that this solved the initial problem, but also added that it seemed to be a royal pain that I had to run the program to see the problem, when it could be (and is in Java) detected during compilation. He's right if we are talking about a statically typed language, but Groovy is really a dynamic language at its core. The amount of compile time checking it can do and still adhere to the dynamic mantra is limited. Let's explore this a bit (See README and code in test3 directory).

First, lets just add a second method (and a couple of println's) to the Groovy Adder class:

class GroovyAdder
{
  def doit(List list)
  {
 println "In typed version of doit"
 
    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }

  def doit(def list)
  {
    println "In def version of doit"

    def sum = 0

    list.each
    {
      entry ->

      sum += entry
    }

    return sum
  }
}

We also need to change the test that uses Queue (hint, the call will succeed this time):

void testQueue()
{
  Queue queue = new PriorityQueue()
  queue.add(6)
  queue.add(7)
  queue.add(8)
  assertEquals 21, adder.doit(queue)
}

Running this test, here is the output we get:

.In list version of doit
.In list version of doit
.In def version of doit

Time: 0.226

OK (3 tests)

This is way cool. Groovy was smart enough to choose the typed version of the polymorphic method when it could and the dynamic version when nothing else could be found.

But, of course, this still could be decided at compile time. Let's try one more thing (See README and code in test4 directory). First, change the uninteresting method we just added as follows (it is now a typed method as well):

def doit(Queue list)
{
  println "In queue version of doit"

  def sum = 0

  list.each
  {
    entry ->

    sum += entry
  }

  return sum
}

So now we have a Groovy class, where the author provided for the use of Lists (which gives us Stack by default) and Queues. Of course, now I want to use a Set. I'm never happy. So let's add the following test and associated closure:

void testSet()
{
  def expando = new ExpandoMetaClass(GroovyAdder)
  expando.doit = dynamicDoit
  adder.metaClass = expando

  Set set = new HashSet()
  set.add(9)
  set.add(10)
  set.add(11)
  assertEquals 30, adder.doit(set)
}

Closure dynamicDoit = { Set set -> 
  println "In set version of doit"
  def sum = 0

  set.each
  {
    entry ->

    sum += entry
  }

  return sum
}

We have now crossed the boundary and are firmly entrenched in dynamic land. The compiler can't save us here. Prior to testing with the set, the test code alters the metaclass of the object we intend to test, adding a dynamic method that knows how to handle sets (it could just as easily have handled Iterable or even def and be more general, some folks never learn). Here is the output from running the test now:

.In list version of doit
.In list version of doit
.In queue version of doit
.In set version of doit

Time: 0.23

OK (4 tests)

As stated earlier, the compiler can't help when the code can dynamically change after compilation, and further, if the compiler tried, it would hinder the dynamic features of the language.

So there you go! But wait, there is more. As an added bonus, let's look at what happens when we are using Java (See README and code in test5 directory).

First, we create a simple Adder POJO:

import java.util.List;

public class Adder
{
  public int doit(List list)
  {
    int sum = 0;

    for (Object entry: list)
    {
      sum += (Integer)entry;
    }

    return sum;
  }
}

See the java test code on the download site for an example of how you get a compile time error if you attempt to pass a Queue into this doit method of this class. You can't get there from here, well at least not in Java. So let's see what we can do when using this class in Groovy. Here is the Groovy test code:

class TestJavaAdder extends GroovyTestCase
{
  def lister = new Adder()

  void testList()
  {
    List list = [0,1,2]
    assertEquals 3, lister.doit(list)
  }

  void testStack()
  {
    Stack stack = new Stack()
    stack.push(3)
    stack.push(4)
    stack.push(5)
    assertEquals 12, lister.doit(stack)
  }

  void testQueue()
  {
    Queue list3 = new PriorityQueue()
    list3.add(6)
    list3.add(7)
    list3.add(8)
    shouldFail(MissingMethodException) { lister.doit(list3) }
  }

  void testQueueWithMop()
  {
    Adder.metaClass.invokeMethod =
      { String name, args ->

        println "In Groovy version of Adder.doit"

        if ("doit" == name)
        {
          def sum = 0

          args[0].each
          {
            entry ->

            sum += entry
          }

          return sum
        }
      }

    Queue list3 = new PriorityQueue()
    list3.add(6)
    list3.add(7)
    list3.add(8)
    assertEquals 21, lister.doit(list3)
  }
}

Running these tests gives:

.In Java version of Adder.doit
.In Java version of Adder.doit
..In Groovy version of Adder.doit

Time: 0.227

OK (4 tests)

The results from TestList and TestStack are not surprising, as they would work the same way in Java. But notice in testQueue we were able to call the statically typed Java doit method, which requires a List, with a Queue. And we did not receive a compile error! We only found our error at runtime.

The reason, again, is that if Groovy strongly enforced the type expectations at compile time, there would be no way to use the dynamic trick demonstrated in testQueueWithMop (MOP = Meta Object Programming), in which we use another Groovy mechanism to dynamically extend the features of a class, this time a Java class, without the need for source code.

The purpose of this blog was primarily to answer the question of how Groovy's optional type mechanism works and provide my understanding of why things work the way they do. In doing so, I made use of some of Groovy's MOP features. To learn more about these, I refer you to the sources I used. Scott Davis' book, Groovy Recipes, has a great intro to Metaprogramming, along with easy to use code snippets that illustrate the topic. Venkat also has a book, Programming Groovy, that delves even deeper into the topic. I highly recommend both books.

I also did not go into the debate of dynamic versus static typing. Folks a lot smarter than me are discussing this issue. I am principally a spectator to the debate. I will say that for DSLs, which I am very interested in, testing, and script use, the dynamicism of Groovy rocks! And I am tending to lean that direction for other code as well, as long as the code is backed by good unit testing.

Hmmm, guess I picked a side. Well, no one has ever accused me of not having an opinion.

Two very Groovy books...

2008-03-08T13:39:00.001-07:00

Over the 2007 holiday, I had the pleasure of doing technical reviews of two upcoming books published by The Pragmatic Programmers. First I reviewed Scott Davis' book, Groovy Recipes: Greasing the Wheels of Java by Scott Davis. I followed this up with a review of Programming Groovy: Dynamic Productivity for the Java Developer by Venkat Subramaniam. For the reader in a hurry, I will summarize by saying if you are just learning Groovy, Scott's book is a must have. If you have the Groovy Basics in hand, then Venkat's book will guide you to the next level. I highly recommend both books.

I met both Scott and Venkat at the Boulder Java Users Group. Scott both runs the JUG and is a frequent speaker. Venkat visits at least once a year to present talks.

The first talk of Scott's I had the pleasure of listening to dealt with agile software development. Scott is an amazing speaker whose excitement for the topic he discusses is truly infectious. I had already decided agile was the right way to go and Scott's talk helped me along the true path. I mention this because it was another talk of Scott's that introduced me to Groovy. Scott made two comments that caught my attention. First, he stated "Groovy is what Java would look like had it been written in the 21st Century." That is a pretty powerful statement. Second, he answered the question of why Groovy versus Ruby... the answer being the context switch is much easier for a Java developer using Groovy.

After listening to the talk, I downloaded the Groovy package and started playing with it. I soon realized there was a lot of power in this language. So I bought Groovy in Action, a.k.a. GINA, and really started digging into the language. At a later JUG, Scott mentioned that he and Venkat were planning to write their own book on Groovy. The language was evolving quickly and GINA was already getting out of date, so I started bugging Scott about the upcoming book. Long story short, eventually, the authors decided that they were each envisioning a different type of book and when Scott had a rough draft available, he asked me if I would be interested in being one of the technical reviewers. I jumped at the chance.

Scott's approach to the book is very agile, in that, after a brief discussion of the Groovy language, it essentially answers lots of very specific questions regarding "How to..." in Groovy. Each section starts with a code snippet that is designed to be copied directly into your code, letting you quickly solve a problem (e.g. parsing XMl documents) and move on. The detail in these examples ensures that this book will remain useful for some time to come. Along with the toolset you get, these examples really demonstrate the power of the Groovy language and provide an excellent way to learn the language.

While perhaps not as all encompassing a primer on Groovy as GINA, Scott does a good job, in the early part of the book, presenting most of the language's features in sufficient depth that I am comfortable suggesting this as your first book on Groovy. And once the basics are covered, Scott goes on to present some excellent examples of how to use idiomatic Groovy to solve common programming problems, including file I/O, XML processing, Web Services, basic metaprogramming, and a nice introduction to Grails, the web framework built with Groovy. After the Grails intro, Scott presents some nice examples of how to use Grails in a similar manner to his treatment of Groovy.

My approach to the review was to first read the text and understand what concept was being presented, followed by a test of the supplied code. This turned out to be very informative. There are three key ways to dynamically execute groovy code; as a script file from the command line, in the groovy shell, and in the groovy console. It turns out that the shell has slightly different execution behavior than either the console or script file approaches. This has to do with the fact that the shell has a binding that controls the visibility of variables that does not explicitly exist in the other environments. I found this issue and discussed with Scott. I have not seen the final version of the book, but if this is covered, it will represent my key contribution to the book.

Scott (or at least his publisher) must have liked the work I did on the book, because after I completed the review, I was asked to review Venkat's book.

I have had the opportunity to listen to Venkat speak on numerous occasions and at one point, described my self as a Venkat groupie! Watching Venkat speak is a truly humbling experience. Not only does he have an amazing ability to describe complex technical concepts clearly, but he does this while writing code to demonstrate these concepts at the same time. This ability to communicate crosses over to his books and his latest is no exception.

While the book does cover some of the basics of Groovy, Programming Groovy is probably not the best book to use for learning the language. Venkat spends more time covering the interesting "edge cases" and "gotchas" of the language than on the basics. For example, he goes into the details of how Groovy's == operator is not the same as == in Java. For someone who already has a basic understanding of the language, the exploration of these more advanced topics is a great way to move to the next level in mastering the language. But it is Venkat's exploration beyond these topics that make this a critical reference for the developer serious about learning about the paradigm shift that dynamic programming entails.

Venkat spends time comparing the strong type checking in Java to the optional typing that exists in Groovy. He makes the case for why the strong typing, while it provides the semblance of safety for the developer, is really very weak protection. he then carries this topic forward and discusses closures, which, when combined with dynamic typing, form the first building block for dynamic programming. He then proceeds to go into the most important topics in the book, Unit Testing and Metaprogramming. With Metaprogramming, which is the ability of a program to manipulate itself, it is possible to create new domain specific languages (DSLs) and sophisticated builders that simplify other programming tasks. Venkat provides great examples of how to use these techniques in Groovy programs.

Dynamic programming is becoming increasingly relevant to modern software development and Programing Groovy is a great introduction to these concepts along with concrete examples in Groovy.

My approach to reviewing this book was basically the same as for Scott's book. Examine the examples and make sure they make sense and they work. Hopefully, Venkat and his publisher found my comments helpful. I really enjoyed this opportunity and hope I have the chance to do more reviews in the future. It is a great way to learn a new technology, since you really have to focus on the details.

I want Java 6 on my Mac...

2007-11-13T19:41:00.000-07:00

So, 13949712720901ForOSX

Fred describes what this is all about here Vote For Java 6.

No Fluff Just Stuff, Fall 2007

2007-11-12T11:58:00.000-07:00

Jay and crew were back in town this weekend. My company, Avaya, sprung for the admission cost (props!) and I donated my weekend. My brains are leaking out of my ears a bit and I can’t wait to try out some of the new ideas I learned.

Before getting into the details, I need to brag about what I won! In Saturday’s lunchtime raffle, I won a license for the IntelliJ IDE ($500 retail). The vast majority of the NFJS speakers are hardcore advocates for IntelliJ and never pass up an opportunity to dis Eclipse. Now I will get to see what the hype is about. In particular, I have heard the support for Groovy is really good. It will be fun and who knows, maybe I will become an IntelliJ snob myself?

It seems like every time I attend NFJS, I end up spending most of my time focused on a specific theme. This time, it was web frameworks. I didn’t start out intending to do this. The first talk I attended was an intro to ReST. There is a lot of discussion and debate about SOAP versus ReST at Avaya (SOAP is currently winning) and I wanted to see what those wacky ReSTafarians were excited about.

Quick aside, quote from Brian Sam-Bodden at lunch, “Everytime I use SOAP, I feel dirty!".

Brian Sletten did a great job of describing why he thinks ReST is a better solution than SOAP for most web services. His argument really boils down to simplicity and flexibility. He dispels the myth that many folks (myself included) have that all ReST entails is exposing your web service via a URL. This is part of it, but he argues that a better way of thinking about it is as “Resource Oriented Computing" paradigm that easily enables separation of concerns in an application. Resources become the “nouns" in the system. They are operated on by the verbs, which in ReST are limited to GET, POST, PUT, and DELETE.. Operations on the nouns by the verbs result in unique views into the system.

As a defense about the simplicity of a ReST interface, Brian mentioned some anecdotal statistics that 85% of Amazon web services users use the ReST web service over the SOAP web service.

While ReST isn’t entirely about the URL, that is one of the most observable aspects of a ReST implementation. Brian spent some time discussing best practices for a ReSTful implementation of these URLs and also discussed the pros and cons of a non-opaque URL. Based both on what I heard and my use of ReSTful interfaces in the wild, I tend to agree with Brian that it is OK and even helpful to have non-opaque URLs for human use, but you should avoid exploiting this fact programmatically.

After hearing the ReST intro, I decided to sit in Brian’s next session, which covered a specific technology to implement ReST, called RESTlet. RESTlet is an open source API for developing a ReSTful system. I’m not going to go into a lot of detail here. Brian presented some great examples of how to use this and now I have yet another technology added to the “List of Things to Do"™.

Well, I had already attended two sessions on web technology, so why not keep it up.

The next three sessions I attended were all given by Scott Davis on Grails. Scott, who runs the Boulder JUG, has been pushing Groovy for quite a while. I am already a Groovy fan-boy and have been playing with it quite a bit. Scott and Venkat Subramaniam are working on a new book, Groovy Recipes, due out towards the end of the year. I have the honor of being a technical reviewer on the book. It is really looking good.

So Grails extends the use of Groovy to a web framework. Using Grails, typing four commands at the command line generates a completely functional web application including persistence! Seriously, this is a less than five minute exercise. So what took three sessions? Scott spent a lot of time showing how to customize the persistence and view behaviors within Grails. It is pretty easy to customize the environment in a global fashion such that after the customizations have been made, new projects can take advantage of these customizations quickly. Groovy rocks, and Grails is following in its footsteps.

Scott also presented the keynote, “No I Won’t Tell You What Web Framework to Use". Even if you don’t care about web frameworks, there were some great take always from this talk as well as some great humor. Scott’s main point is that since everyone has different needs and requirements, just asking someone what to use is not very helpful. He argues that you should really ask “What are you using?" and “Why?". And ask a lot of people. Then decide what the important decision points are for you (keep the number of criteria small). Use the feedback you have received to arrive at your decision. This summary doesn’t do justice to a talk that was well crafted and chock full of observations about how choice works in almost any aspect of life.

Well, that was it for the web frameworks. While this was my main focus, I did attend some other interesting sessions. I attended three sessions presented by Brian Goetz on concurrency and memory models in Java. In particular, he covered the changes in Java 5 surrounding concurrency, which are quite substantial. A key take away was that you almost never have to instantiate a thread directly in Java 5 and if you do, you better have a good reason. The concurrent library provides some great functionality for adding and controlling concurrency in a product. The timing of this information was helpful, as I am in the middle of a concurrent implementation and got some great ideas about how to approach it.

Attended two sessions by Ted Neward. Ted is the guy who has an opinion on everything, but also has the knowledge to back them up. His talks are some of the most information dense talks I have ever attended. The first was on debugging and monitoring in Java. It was interesting to see the tools he uses and the level of detail you can get to with them. Definitely good stuff. His second talk was on classloaders. I had seen this talk before, but needed to see it again to really absorb the info. It is starting to make sense and again, the information is timely, as I will be applying what I have learned immediately at work. According to Ted, now that we understand classloaders, we have a star on our shoulder that indicates we have attained Java Guru status. I wish!

The last session I attended, presented by Brian Sam-Bodden, was a basic overview of Eclipse Rich Client development. Not something I will be using immediately, but still was nice to see what is available. Eclipse 3.3 provides some nice wizards to bootstrap the process and Brian provided a nice overview on how to make use of them.

So I have tried to fit an entire weekend into a short blog. I barely scratched the surface. Didn’t really discuss the networking and technical discussion that occurs in between sessions. The great side discussions that occur with the speakers. The humor, the good food, … You really need to attend to get the whole picture!

I am a Nerd King

2007-09-22T11:32:00.000-06:00

Well, I rate lower than some friends, I suspect that fact that I am not a huge Star Wars fan affected my score (I am pretty sure I aced the Star Trek stuff). Also, Pokemon? WTF? I like MTG!!! Anyway, here are the results...

June 2007 BJUG

2007-06-19T07:35:00.000-06:00

I attended the BJUG last Thursday. I have been attending regularly for about a year and a half and even presented there last January. This meeting was perhaps the best yet. Jay Zimmerman and Scott Davis continue to run a top-notch Users Group!

The first talk was given by Howard Lewis Ship on Tapestry, which is a web development framework and is part of the apache project. While the topic fell a little out of the area where I am currently working, it was nonetheless interesting. I am planning to play with grails a bit and it is good to see how other frameworks work so I have something to compare against. Howard is a great speaker and author. We'll come back to Howard in a bit.

The only downer in the meeting was that the pizza was late. Fortunately, Howard took questions late, and our next speaker, Mike Cohn, agreed to break his talk into two pieces. Mike, who is a leading expert on agile development, presented Transitioning to Agile. I am a Scrum convert who has read both of Mike's agile books cover to cover. I was very interested to hear the talk, even though it comes a bit late as we have already transitioned. But it never hurts to review how you got there. Mike calls the technique we used "Don't Take No For an Answer". I liked the sound of that!

Mike made a point that is worth repeating. Dogmatically following "Best Practices" leads to stagnation. Only by understanding when it is appropriate to follow such practices and when deviation is a better path does a group learn to adapt. Mike argued that adaptation is a key tenet for a successful agile development.

The best part of the JUG, and arguably one of the best parts of JUGs in general, was when I got to speak to Mike one on one during the break to discuss a problem I was having with our Scrum development. Our team is tackling a large problem that we don't understand very well. The issue is so complex that a spike would not be sufficient to understand the problem. I explained this to Mike, who immediately replied, "Then Scrum is out the door!" He told me the team should focus on fixing the problem as quickly as possible, but that we should not try to fit the issue into a sprint structure. He did suggest that we keep the daily scrums running as a good starting point for communication and that as soon as possible, we restart our sprint planning and subsequent sprints. This is basically what we had done. This is one of the things I like so much about agile. It is pragmatic and usually, the most obvious path is the right one. Having said that, it was still good to hear from the expert that we are on the right track.

Another thing I like about the JUG is the ability to get together with some very smart folks and discuss software development. Geeky? You bet! Prior to the meeting, a friend, Chris, and I were discussing unit testing and the use of mocks. Chris was telling me that he had an epiphany and now understood when to use mocks in a way he hadn't before. I admitted that I still found the topic a bit of a mystery. In the fashion of all good teachers, rather than just give me the answer, Chris showed me how to have a similar epiphany. I will be blogging about this more later (and btw, I now "get it" as well), but for now, just wanted to highlight yet another reason I look forward to BJUG.

In the past, after the BJUG, I would go for beer with a couple of friends. Word about our get together spread and this month, about 12 of us went for beers. What a great time. Lots of shop talk intermixed with general conversation about other "stuff", including Howard introducing the team to a new game, Zertz. I totally suck at this game, but it is fun.

The plan is for this outing to become a regular ending to the meeting (we are going to look for a place that serves scotch for Demian). Should be a great time! If you live in the Denver/Boulder area, consider joining us.

Is Your Framework Adding Value?

2007-05-26T00:37:00.000-06:00

Trust me, this is going somewhere...

Pre-fab or Field-fit?

A friend of mine is a drilling engineer. He has traveled around the globe and has many interesting stories he has shared with me. One involved an experience he had while working in the former Soviet Union around 1993. He and some co-workers toured a factory where pre-fabricated apartments were being built. The idea was that the major components, such as walls, would be built in the factory and trucked to the site where they could be assembled, simplifying the construction process.

Sounds great in theory, but my friend noticed a flaw in the implementation. The walls were made by pouring concrete into large rectangular wooden forms. These forms had internal forms for windows. While the forms were precisely measured and designed to fit, the corners of the internal frames were not braced and as the concrete was poured, they shifted, and what should have been a rectangular shaped window in the middle of the wall wound up being a parallelogram with non-right angles for corners. What should have simplified construction in the field wound up creating more work than starting from scratch, as each section had to be "field fit".

When asked about the problem, the factory workers shrugged and indicated that since it wasn't their problem, they didn't feel any need to change what they were doing. Of course, it didn't matter, since the window glass never fit anyway. When they assembled the buildings, the panes of glass were often overlapped.

The Liquid Foundation

While watching the History Channel, I saw an interesting program about an earthquake in Kobe, Japan in 1995. It turns out that the Japanese have taken to creating artificial land to help ease the real-estate shortage that they are experiencing. This is done by dredging soil from the ocean floor and depositing in other water-filled areas until it forms artificial land ready-made for the construction of houses, businesses, and highways.

Unfortunately, when the earthquake hit, an unexpected state change, known as soil liquefaction, occurred in this artificial land. As the shocks occurred, the land acted, for all intents and purposes, like a liquid, flowing and oozing all over the place. Anything built on top of this now liquid foundation was destroyed.

Software as Architecture

It is fashionable to compare the creation of software with the creation of buildings. We go so far as to call many members of the software industry "architects". While I am not crazy about this metaphor, it does work at times.

Software organizations often attempt to go the prefab route by building frameworks composed of common utilities that can be "trucked" to different developments. The thought is that the framework provides a pre-fabricated foundation (to stick with the metaphor) on which to build the application.

Sounds great in theory. Hmmm, where have we heard that before? At issue is the quality of said framework. Were the architects of the framework familiar with good API design? Were there any architects or is the framework simply an amalgamation of code fragments gathered from other developments? Is the goal of the framework to provide the basics or does it represent a grand attempt to boil the ocean? And most importantly, does the framework simplify application development or does it complicate the application and leave it with "window frames" that are field-fit and "panes of glass" that overlap?

These are important questions for a development organization to ask when it assesses whether to go the prefabricated framework route. Not only is there a risk of wasting development effort (and slowing time to market) trying to fit an application into an awkward architecture, but after this occurs, the resulting product is now built on a shaky foundation that may liquefy at any time, but of course it will wait to do so until deployed at a large customer site.

I am not saying that frameworks in and of themselves are bad. At issue is how the framework was constructed and whether it truly provides the value its use implies. We can look to the open source communities for examples of frameworks that work. Spring, Hibernate, Commons Logging... solid and designed for easy integration.

I told you I was going somewhere.

No Fluff Just Stuff, Spring 2007

2007-05-06T20:35:00.000-06:00

NFJS, Spring 2007

This is the second year in a row I have attended the Denver leg of the No Fluff Just Stuff tour. Jay Zimmerman and crew have outdone themselves. The speakers were excellent, the topics were relevant, the venue was nice, and the food was good!

Due to budget issues, my company was not able to spring for the registration fee this year. So I paid for attendance out of my own pocket. It is that good! Kudos to Linda for supporting the financial decision.

I started out the seminar attending Venkat Subramaniam's Annotation Hammer talk. Annotations are near and dear to my heart, as they were the basis for my own talk on Hibernate with Annotations. It turns out Venkat doesn't feel that this is an appropriate use of annotations.

He made the distinction of intrinsic versus extrinsic metadata. According to Venkat, an intrinsic feature, which is one in which the semantics of the metadata are expressed in the code itself, is a good candidate for annotation use. Extrinsic features, in which the metadata expressed by the annotation is not expressed in the code itself, is not a good candidate for annotation use. Venkat believes that persistence semantics, such as in Hibernate, fall into the latter category. He also believes that using annotations with Spring and Hibernate results in vendor lock-in. I can buy into the extrinsic argument to some extent (but still like Hibernate annotations for small apps). I agree that Spring annotations result in vendor specific implementations. I disagree that annotations in Hibernate are vendor specific, since they are based on the JPA. If you don't like Hibernate, you can switch to another JPA provider (such as Ibatis). That's not lock-in where I come from.

Other interesting take aways from the talk include some info on Java 6, which will expand the use and support of annotations In Java 6, annotation processing is built in. For Java 5, the external Annotation Processing Tool is needed for pre-processing. A new annotation in J6, @WebService, provides all the scaffolding to support a web service without the need to write code.

Venkat showed some examples of junit 4.0 annotations. I really couldn't see any reason to use them over the standard junit API. No real value add.

Venkat concluded with a couple of observations: annotations "leak into" your code and can be hard to get rid of later. I don't like XML is not a good reason to favor annotations.

I'm guilty of the last one...

The next talk I attended was Neal Ford's 10 Ways to Improve Your Code. Neal used an interesting tool for his preso, MindManager, which is a mind mapping tool used for brainstorming and, it turns out, for making presentations.

Some key take-aways from this talk (some of them are "duhs", but still):

Version control and continuous integration are fundamental. An automated build machine should not have development tools on it.

Static code analysis should be favored over a compiler as the first step of testing. He mentioned FindBugs and a tool called PMD. PMD has a feature called "copy paste detector" that is useful for finding common code that can be refactored.

Singletons are glorified global variables. Overusing static classes is a sign of a procedural mindset

Persistent state is the enemy of unit testing. Keep your object state valid at all times. As an example, we discussed an Address class, with address, city, state, and zip fields. Neal argued that it made no sense to have setters for each field, since an address without a city makes no sense. Instead, he recommended using a constructor that takes all fields as arguments. I asked about a UI that might need to change the object. He suggested an update method. This is a great idea that I plan to use in my code.

Test driven development, avoiding feature creep, and analyzing dependencies (using a tool called JDepend) were discussed. He also presented an algorithm developed by an associate of his that used code coloring to identify code that provides the maximum business value. It was a bit complicated but looked like it could be useful if applied rigorously.

He touched on Domain Specific Languages (DSLs). This looks very interesting but we didn't get into much depth here, but the gist is that you should write APIs and name components using terminology that fit the domain.

Neal touched on a topic he calls Polyglot Programming, which was also the topic of his keynote talk. Summarizing, he argues that developers should use the most appropriate language for the task at hand and that it is acceptable to mix and match languages in a product. Given that there are 25 different languages that run on the JVM now, this is starting to make a lot of sense (see my discussion later on Java 6 for more on this).

The other critical argument Neal made in his keynote, and one that I whole-heartedly buy into, is that for software development to truly be considered an engineering discipline, the practitioner's of the discipline must embrace and commit to testing as the underpinning of everything the developer does. Neal is a test-driven development zealot. I am quickly becoming a convert. Of course it will take me some time to get good at it, but I have a goal.

The next talk I attended was OSGI: A Well Kept Secret, present by Venkat. OSGI is a component management platform that protects an application from "classpath hell". After hearing what this tool does, the first question the pops up is, "Why isn't this baked into the JVM". It turns out there are plans to do this, but there are two competing standards, JSR 277 and JSR291. And neither of thee standards appear to be learning from the OSGI team! So I'm not holding my breath. Like all Venkat talks, the demos were detailed and really showed the power of the product. I don't currently have need for this product, but it was good to get an introduction to it. Probably worth mentioning that OSGI is baked into apps like Eclipse.

Start of Day 2, first talk I attended was Build Teams, Not Products, presented by Jared Richardson. As a dev lead on a Scrum project, team building is a very important aspect of my job, and, in all honesty, not one of my strengths. So this was a relevant topic. I got some good ideas that I plan to try with my current team...

As the dev lead, it is important for me to share my own impediments so that other team members will be willing to do the same.

Focus on interactions, which will maximize human bandwidth.

Encourage informal code reviews as both a learning tool and a way to kick-start interactions. Verbalizing design decisions helps the whole team.

Randomize the order that the scrum reports in every day.

It was interesting to find that Jared worked at SAS, which rumor has it is the utopia of companies to work for if you are a software engineer. Jared dispelled this rumor a bit saying that, in regard to the work, there are good and bad areas within the company, just like any other company. Figures. He did say the bennies and culture were nice.

I followed this talk with another by Jared, Continuous Integration with Cruise Control. In keeping with my goal of getting better at TDD, I wanted to learn more about this tool so that I could use it more effectively. It was a good talk, with a great demo of the product. Nothing earth shattering but good review of what is available in the tool.

Neal's keynote got me to thinking and i decided it was time to look at Ruby again, so I attended his talk on JRuby. I'm still not sold on it. I really like Neal and respect a lot of his opinions, but he seems to be a bit hypocritical when it comes to Ruby. He spent a lot of time in the keynote discussing the warts in Java. During his Ruby talk, he glossed over similar issues in Ruby. One example is that arrays in Java being zero-based is bad, but when we run into this in a Ruby example, it is glossed over. Hmmm.

Neal stated that in Python, there is only one way to do something, while in Ruby, there are ten ways to do the same thing. He went on to argue this is a strength of Ruby. So put ten programmers in a room with Ruby and they will do the same task ten different ways. The maintenance team is gonna love that!

Neal argues for very clear concise class, method, and variable names, but the fact that the Ruby to string method is called to_s is not pointed out as a problem. Worse yet, there is Perl-like syntax that has crept into the language. How is this a clear, understandable piece of code:

list = ArrayList.new
%w(Red Green Blue).each { |color| list.add(color) }

%w? Give me a break. If I knew nothing else about the language, this would send me running... away!

Another Ruby annoyance is the suffixed if as opposed to the Java prefixed if.I am willing to concede that to some extent, this one is a learning curve, can't teach an old dog new tricks issue. Guilty as charged. But I'm still gonna use Groovy as my dynamic language of choice!

I should note that I really enjoy Neal's talks. He is intelligent, has a great sense of humor, is way smarter than I am, and does get me to look at things from a different angle. I just can't get excited about Ruby/JRuby.

I finished the day attending a talk, Making Architecture Work through Agility, presented by Mark Richards. Mark is an architect with IBM... and it shows. I am not very happy with the term architecture in software development. I really like Martin Fowler's essay Who needs an architect?. Mark's presentation reinforced this opinion. He showed a typical, "boil the ocean", eye-chart caliber architecture document and proceeded to discuss how an architect could get teams to implement the architecture in pieces. But this occurs "after" the architecture is delivered from on high. It is probably a bit better than what happens now, but I would argue that a better approach would be to use an agile process, and start implementing a product immediately, using refactoring to evolve the system over time as new features are identified and operation of the system is better understood. I attended this session hoping to change my mind abotu architecture, but instead, I find my opinion has been reaffirmed.

On the final day, I started by attending the Java 6 talk presented by Venkat. I knew that J6 was out there, but hadn't heard much about it. I hadn't heard anything about it that seemed like a must have or even a nice to have feature.

Java 6 is defined in JSR 270. It is currently available for Windows and Linux, but not for the Mac (a developer release is available). It turns out that both the developer Mac version and the Windows version running under Vista have problems. Linux users, you know who you are, this is your cue to start the heckling...

The most impressive new feature in J6 is its support for more than just Java. There are something like 25 languages that run on the JVM now and J6 provides a mechanism, called the ScriptEngineManager, that allows processing of these languages in Java code. This feature becomes very important with Neal's polyglot programming paradigm. J6 has a tool, jrunscript, that allows interactive testing for different scripting code.

J6 also provides built-in support for web services including a small webserver. Venkat feels that this feature is too little, too late. As a person who doesn't know web services very well, I might be able to take advantage of this feature.

J6 provides some tools to support monitoring and programmer support. JHAT provides some cool monitoring tools, JavaCompilerTool provides programmatic access to compiler, and there is built in annotation support, using -processor option on javac to specify annotation pre-processor.

J6 also provides localized UI support, splashscreen support (your splashscreen can be displayed BEFORE the JVM starts), console support, and integrated JDBC support. Venkat didn't go into these features in detail, but it is good to know they exist.

At lunch, there was a speakers panel. The speakers like to pick on Ted Neward, who seems to eat it up. They seem to have a lot of fun. Ted seems to enjoy taking the opposing side to the rest of the group. In particular, he made a comment that with careful programming, unit testing was not necessary. I was surprised that the rest of the group didn't hang him from the nearest tree. I later heard him mention that compilation WAS a weak from of unit testing. I think he likes the controversy. I hope so, because he seems like an intelligent guy in other respects.

The rest of the afternoon was spent in, here is the funny part, talks by Ted! He had a series on some quasi-basic stuff that I have never gotten around to looking into much. I took this opportunity to fix this.

The first one was on monitoring and debugging. Ted argues, effectively, that monitoring is more important because it proactively prevents errors, alleviating
the need to debug.

He presented some tools that come with Java that allow monitoring to occur. To demonstrate, he used a demo program available from sun called swingset2, which demonstrates many features of swing and is worth a look just for grins.

He started by looking at garbage collection, using the gc command line option, -verbose:gc, followed by a demo of jconsole to see the same information in a graphical format. He also demonstrated JMX hooks into the JDK logging libraries. which are cool. Unfortunately, they are not supported in log4j, which we use at work. The only criticism I had for this session is that towards the end, it got a little windoze-centric.

The last two sessions dealt with Reflection and Class Loaders. Lots of good technical information was imparted. Very information dense. My head hurt afterward. I'm not sure any of it is directly applicable to me right now, but it will be nice to have this background if I ever do run into uses of them.

The last thing I should mention is that I intentionally avoided talks by Scott Davis, but not because he is a bad speaker, which couldn't be further from the truth. Because he runs the Boulder JUG, which I attend regularly, I get to hear him talk on a regular basis. Were this not the case, I would probably attend everyone of his talks. Yup, he is that good! Wanted to put a plug in for him in case you haven't heard Scott talk and have the opportunity! Attend if you can... you won't regret it!

This blog is a short summary of a long weekend, but hopefully gives you a flavor of what the weekend was like. If you ever get the chance, I highly recommend attending a NFJS event.

A Geek's Groovy Valentine's Day Gift

2007-02-17T22:22:00.000-07:00

I have wondered at times, does my wife listen to my ramblings about what I am working on, or has she just gone off to her happy place and is on auto-pilot?

Well, on Valentine's Day, I found out that she is listening at least some of the time. I am pretty excited about Groovy and while I haven't done much with it yet, I am hoping to do so soon. Anway, Linda must have heard something about Groovy, as demonstrated by the following pictures.

Presenting at the BJUG, What Else Could Have Gone Wrong...

2007-01-13T14:48:00.000-07:00

Well, for starters, I could have had to drive from Monument to Boulder, got stuck in traffic, realized I didn't have the directions to get to the meeting, and barely made it on time for my presentation. That's what happened to the presenter that followed me, David, Geary, who went on to present an excellent talk on the Google Web Toolkit.

David has written an e-book on GWT Shortcuts, that was supposed to be available at Informit., but it is listed as coming soon. He also has a paper book in the works for later this year on GWT.

Anyway, back to my preso. I had signed up to present Hibernate With Annotations to the Boulder Java User's Group. Hibernate is an Object to Relational Mapping library that I am using for my pet/perpetual project to keep statistics for a hockey team. I am using a new feature in Hibernate, using Java 5 annotations to manage my Hibernate ORM. There isn't a lot of info on it currently, so I thought I would share.

So I realized that afternoon that I had forgotten my monitor adapter for my laptop (DVI to VGA). No way to hook up to the projector. D'Oh! So I left work early and picked up a cable at the Flat Irons Apple Store. Then I met some friends at a restaurant for a pre-meeting dinner. The waitress was very slow and we got out a little later that I had hoped, but we still had some time.

So I am driving into Boulder on 36, three friends in my car, just dropped over the top of the hill, and I see an HP in front of me. He is driving about 60 in a 65, so I pass him doing about 66. As soon as I pass him, he drops over and hits the lights. What the hell? Is he really gonna ticket me for 1 over? Turns out, no, my license plates have expired. So he is ticketing me for that and takes his sweet time. Now I am in trouble time-wise. When he finally gives me my ticket, I take off for the meeting. We get into the parking lot 5 minutes before I am on. I get my stuff and start into the building at a fast clip, leaving everyone else in the dust. On my way, I hit a patch of ice and fall. I barely miss a beat, jumping back up and heading in. My friends jokingly tell me to wait up. I found out later that my fall resembled my bowling style.

So I get into the meeting room and begin setting up my laptop. As I do, I look down to discover that my leg is covered in blood! Not knowing what to do, I continue setting up. My friend Demian arrives a minute later and asks if everything is going OK. I say not really and point to my leg. Demian immediately goes and grabs me a bunch of paper towels, wet and dry, which I use to start cleaning up. Fortunately, I am behind a podium, so I'm not freaking out everyone in the room. Anyway, by the time I finished setting up and futzing with getting the projector to work, my leg appears to have stopped bleeding and I am ready to start. Only five minutes late!

My friends told me that I did a great job on the presentation. They said it was not overly apparent that I had been a nervous wreck just before I started. I know that is what friends are for, but based on feedback I got from other folks in the room, I am gonna go out on a limb and say I did OK. Comparing my presentation to David's, it is obvious which one of us is a professional lecturer, but given that I am a newb, I am happy with how it went. I wanted to make sure that I didn't just "read the slides", and think I managed to avoid that particular sin of presentation. One good criticism I received is that I should have incorporated looking at my code into the presentation instead of waiting until the end to walk through it.

I agree that that would have been a better way to do it, particularly after seeing how David did this with great success. I will have to practice quickly getting in and out of PowerPoint if I want to do this successfully.

BTW, you can see my preso, and the code if you are interested, at my web site. My friend Demian, at my request, submitted the site to Digg, a community web site for sharing content. I was hoping that there would be some interest, but it flopped. I got 5 diggs, all from friends. you usually need at least a 100 to get noticed by the masses. The main reason I was hoping to get the exposure is to get more feedback on my solution. Ah well, guess I am on my own.

If you take away the fifteen minutes preceding my preso, I really enjoyed the experience. There is an old adage that the best way to learn something is to teach it. From my experience, this is definitely true!

A Serendipitous Post

2007-01-05T16:14:00.000-07:00

My daughter is having problems sending e-mail from Mail.app on her Mac while she is at school. I recently experienced the same problem at my sister-in-laws in Minneapolis. I was set to do some digging on this over the weekend when I found the following post from Scott Davis while reading some blogs...

Port 587

Problem solved. Thanks Scott!

Anyone for a Scrum?

2007-01-05T11:28:00.000-07:00

In preparation for work on a new project, I have been investigating the Scrum software development process. I have read “Agile Software Development with Scrum” by Ken Schwaber and Mike Beadle and am currently reading the follow on book “Agile Project Management with Scrum” by Ken Schwaber. Next in the queue is “Agile Estimating and Planning” by Mike Cohn. From what I have found on the net, these are essential reading for anyone getting started in Agile. Based on what I have read so far, I tend to agree.

At its heart, Scrum (and really all agile processes) attempts to inject some much-needed realism into project planning and development. The pillars of Scrum are visibility, inspection, and adaptation. Schwaber spent time with industrial process gurus reviewing software development. The consensus is that software development is complex and can only be successfully managed using an empirical process. Scrum provides such a process. This is in contrast to a defined process such as the waterfall model that is embodied in many companies’ development processes (including the one used in my company).

Given that I am reading three books just to get familiar with Scrum, there is no way I can provide the detail in this e-mail needed to fully sell anyone on the idea. Here are a couple of good sites that cover Scrum. The first provides a very good high-level overview and the second gets into details.

CodeProject

Mountain Goat Software

At a high level, here are some reasons why Scrum should be considered for use.

Visibility into project status with Scrum is excellent. At any given time, it is easy to see what has been done and what still needs to be done via the burn down chart and product backlog. These are quantitative tools for examining cost/time/quality throughout the duration of the project. After a couple of Sprints, the burn rate or velocity of the team can be calculated and the end date accurately projected. Need more visibility? Reduce sprints to 2 weeks in duration.

Since Sprints are only 30 days long, if priorities change or problems are encountered, only 30 days are lost. Before each Sprint, the direction of the project can be refined or even drastically changed if needed. Scrum is all about agility and adaptability.

For larger projects, a set of interlocking Scrum teams can be used to develop sub-components. This is an area that will take a little more work to get right, but will be worth the effort. To quote Dr. Venkat Subramaniam (technical author, speaker, and amazing developer), “Any team with more than 10 developers is a setup for failure.”

At a recent JUG in Boulder, author Neal Ford asked the audience, “How many of you are developing using a waterfall process that is being called agile?” A lot of hands went up, including a co-worker’s and mine. Going forward, I have resolved not to be quiet whenever we use the term agile to describe a waterfall process. They are not agile. That’s OK, particularly if you buy into waterfall as a successful software development process, but it isn’t agile. Until we stop using the word to mean something it doesn’t, we won’t get any closer to truly finding out if agile can help us. If you don’t have enough first hand evidence already, Schwaber recommends “Wicked Problems, Righteous Solutions” for discussions about why waterfall doesn’t work.

Schwaber makes the observation that technology plateaus are gone and that new projects + new technology = unpredictability. Scrum provides a tool to manage these challenges.

What’s next? I recommend getting Certified Scrum-Master training for all management and senior developers and Scrum product owner training for the product marketing team. Then implement what you have learned in creating great products!

The Passionate Worst...

2007-01-02T22:01:00.000-07:00

A follow up to my last blog on Chad Fowler’s book. I’m not going to summarize the entire book, which I highly recommend, but rather comment on a couple more points that really hit home. Fowler is discussing strategies a developer can use to protect their job. The “Be the worst programmer in the room” and “Love it or leave it” strategies are right on target.

In “Be the worst programmer”, Chad creates an analogy between being the worst musician in a band and being the “worst” programmer on a team. In both cases, he argues that people tend to perform at the level of those surrounding them. If those surrounding you surpass your abilities, it is often the case that you rise to the challenge and the quantity and quality of your work will surpass what you might normally have produced. He also argues that the opposite is true. If you find yourself on a team composed of lesser caliber people, your performance may take a hit because you are not challenged.

I have always enjoyed working with talented people. What I liked about this strategy is that it put into words what I think I have always instinctively been drawn too. While I have never felt like “the worst” (too much hubris?), I have always thrived on learning from those around me and have often felt like I had to step it up when surrounded by experienced developers. When I am the most experienced person in a team (which is where I find myself at the present time), I try to motivate the folks I am working with to stretch their talents as well. I hope I am successful, but if the truth were known, I think I prefer being “the worst”.

When I interview potential developers, I think the key trait I look for is passion (particularly for software development, but really passion in any area is a plus). This fits right into Fowler’s “Love it or leave it” strategy. A quote from the book, “What I found were a whole lot of people who were picking up a paycheck and a few incredibly passionate craftspeople,” summarizes my experience interviewing applicants. The blank stares I get when I ask “What technology or technological book in the past six months got you excited?” truly amazes me. One deer in the headlights applicant, in an attempt to hide the fact that she had nothing, answered that she had just read the user manuals for a commercial database she was using at her current job! As Chad says, “You can fake it for awhile, but a lack of passion will catch up with you and your work.”

I have been fortunate in the past two years to have been surrounded by developers who were both passionate and better than I am. I truly believe I am a better developer because of it.

Generalization and Specialization

2006-12-17T10:50:00.000-07:00

I have been reading "My Job Went to India... (and all I got was this lousy book)" by Chad Fowler. I'm still in the middle of it, but a couple of sections really resonated and I wanted to jot some thoughts down while they were fresh in my mind. It is probably worth mentioning for those not familiar with the book that it is not really a criticism of India or of outsourcing in general, but rather, a pragmatic discussion of how to survive in the midst of the current trend to outsource technical work. Fowler presents his thoughts on what a person can do to keep their job domestic when so many positions are leaving the country.

A couple of his ideas are contradictory, yet complimentary. He argues that a developer should decide to be a generalist or a specialist. He compares the specialist to an assembly line worker on a mass production line. The generalist, on the other hand, is the proverbial "Jack of All Trades, Master of None". I like that Fowler tries to dispel the negative connotation surrounding this label. In Fowler's opinion, either path can lead to better protection of your job.

The generalist is able to pick up the odds and ends and minutiae left behind in the wake of layoffs or outsourcing. He argues that it is important for the generalist to understand to some level the product at both a business and technical level. At the product level, understand the customer needs and business case for the product. At the technical level, not only should you understand the code used to realize the product, but also the architecture, test strategy, and deployment process. The generalist should also be careful not to fixate on specific technologies, e.g., code versus database, Java versus C#, etc.

A specialist can make his or her self safe in a different way. This is accomplished by understanding an area of technology to a depth that qualifies this person as a guru. By doing so, the developer becomes the indispensable resource of last resort, turned to when the really hard technical issue crops up that the rest of the team can't resolve. Fowler is quick to point out that this is a difficult state to attain. One example he uses is an interview question, "How would you write a program, in pure Java, that would crash the JVM?" He argues that unless you understand the internal workings of the JVM (which would include the threading model, memory management, and instruction generation), you aren't a Java specialist.

As an aside, I don't consider myself to be a Java specialist, but my first thought was to exhaust memory, but I am sure there are faster ways to crash the JVM.

While he didn't come right out and say it, I got the feeling that Fowler sees generalization versus specialization as an either/or proposition and that he maybe leans a bit towards favoring generalization. While I concur that generalization is a better survival strategy, I think there is room in the generalist’s repertoire for some degree of specialization. In my mind, a melding of the two ideas is very compelling.

I consider myself to be a generalist. I am a degreed electrical engineer. I have experience with digital electronics. I spent the first four years of my career coding in assembly for an embedded real-time system. I also wrote real-time firmware in C and C++. I have written graphical applications for Windows and the original MacOS. I am currently developing middleware apps in Java. I also have experience with scripting languages such as PERL, TCL, and Bash. I wrote some SQL and am currently learning the Hibernate ORM utility. Next up is Groovy. The list of things to learn is very long and is growing faster than there is time in the day.

I don't mention this to toot my own horn, but to support my hypothesis that I am a generalist. But I am also a specialist, albeit what I will term an opportunistic, short term specialist. Invariably, during certain stages of a project, I wind up learning something to a detail that meets Fowler's definition of a specialist. The area of the specialty varies from project to project and because of this variability, the half-life of my "specialist" qualification is short in duration (on the order of 1-2 years). For example, in the mid-90's, while working in the test and measurement industry, I became an expert on the IEEE 488.2 and SCPI command sets for instrument command and control. I knew the vagaries of the standards and what requirements needed the most work in the code. Ten years later, the details escape me, but at the time, when a question was raised about these protocols, folks came to me. The same has occurred in other industries, including understanding drug interaction for IV compounding, VME and VXI bus protocols, ISDN signaling (LAPD and Q.921), and currently, I am immersing myself in the agile methodology, SCRUM. In the spirit of "use it or lose it", when I move on to other projects, the details get fuzzy, but I always try to adopt at least one topic to focus on and embrace in a detailed sense.

The side-benefit of this strategy is that even when the details fade, much of what I learn can still be applied in a general fashion. Which supports my initial statement that I am a generalist. In summary, as a generalist, sometime specialist, I can attest to Fowler's thesis that developers who adopt these strategies maximize their chances of staying gainfully employed. It has worked for me.

The Open Source Community... Loony or Enlightened?

2006-12-03T21:29:00.000-07:00

I attended the Lightweight Java seminar with a colleague last week. The seminar was hosted by Virtuas. Virtuas is a services company specializing in open source software. Their stated mission is to assist their clients in becoming self-sufficient users of open-source software. More on this is a minute, but first a quick overview of the seminar itself, which was composed of four talks.

The first talk covered the Java Persistence Architecture, or JPA, which is part of EJB3. A simple description of JPA is annotation support for persistent objects. The JPA is defined in JSR 220 and represents an attempt to simplify J2EE. Since I'm not an "enterprisey" guy, I can't discuss how well it addresses this need. I do have some experience with JPA, as it is implemented in Hibernate. I will be giving a talk of my own on JPA use with Hibernate in January at the Boulder JUG. I will be focusing on the use of annotations in Hibernate without the use of XML configuration, so definately an anti-enterprise spin. I will post the preso after the JUG. At this talk, the speaker had a great example of persistence using a snake object that eats rabbit objects. Call me twisted, but I liked it!

The second talk covered Spring 2.0. Spring, in keeping with the "lightweight" theme of the seminar, provides a lightweight container environment that simplifies the use of other open source components including Hibernate, AOP, web services, LDAP, ACEGI security, an MVC controller, and more. Of course, "simple" is relative. In the case of Spring, this means that while you write less code, you substitute large XML configuration files. Anymore, every time I am told "less code through configuration", I feel like I am at a timeshare sales presentation. Caveat emptor! Even with the configuration overhead, I do think that Spring brings a lot to the table. It is definitely on the list of things to get up to speed on.

The third talk covered AJAX and JSF along with the DOJO library (word of advice, 4.0 is buggy, us 3.1 for now). These are tools that make some great web interfaces. Added to the list.

Finally, a talk on the AppFuse 2.0 framework. This framework ties all of the other pieces together when developing web apps. Since I don't do much web development at this point, some of the talk was lost on me. Probably the biggest take from the talk is that Maven is getting bigger and is now on the LOTTD (yeah, it's a long list).

Prior to the seminar, we weren't sure what to expect. We halfway expected to be subjected to a non-stop flood of markitecture. On top of that, the weather was really crappy. We really debated attending. Fortunately, we decided to go. It was definitely worth braving the weather. When we first arrived, we found that we were getting an amazing "goodie bag", which included three great technical books (on Java, Spring, and Eclipse) and a nice day-planner. The goodies were courtesy of SourceBeat. Want to impress a geek? Give them tech books! And it just got better from there. After a very short (less than 10 minutes) introduction of the company, they jumped right into the presos. Not only no markitecture, Virtuas took the soft sell to the extreme... almost a no sell? The great goodies, excellent talks, and lack of sales pressure all contributed to the inspiration of this blogs title. Can a company, which depends upon revenue from consulting on products that are free, be successful when using a marketing tool such as this seminar? I hope so.

As a way to say thanks, I started thinking about what I could do to help them. I'm not really in a position right now where I need their help, but I will keep them in mind for future issues. In the meantime, I am hoping that by blogging about them, I can help is some small way to get the word out about this enlightened (as opposed to loony) company.

NFJS - Spring ACEGI

2006-05-15T15:52:00.000-06:00

Presented by Stuart Halloway, Relevance, LLC

ACEGI is a generalized authentication and authorization system for Java, with specific hooks built-in for the Spring Framework. Since my experience with such systems is light, my ability to cover what was presented will be correspondingly basic.

The name ACEGI is not an acronym. It is derived by using the first 5 odd letters of the alphabet. Like most Spring-based services, ACEGI relies heavily on XML-configuration files. It is built using the Inversion of Control pattern and with dependency injection, it is possible to modify behaviors of the security system. One key example pointed out by Stuart in the talk was that JAAS security library, which (if I remember correctly) is part of the Java standard, can be used as the authentication server instead of the default service provided by the ACEGI distro.

The key components of an ACEGI based system include:

Authentication

Authentication Manager

Access Decision Manager

Secure Object Interceptor

Run As Manager

After Invocation Manager

The last two components are only used in special cases. A simple use case would be:

User submits principle (user) and credential (password) to Authentication object

The Authentication Manager populates the Authentication class with Authorities (permissions) based upon the provided Principal and Credential

User invokes the desired method, which is intercepted by the Secure Object Interceptor (shades of AOP at work!)

The SOI queries the Access Decision Manager regarding whether the request can proceed

The ADM passes this query on to the Authentication object, which accepts or rejects the request based upon the stored security data

Assuming the request was allowed, the SOI then invokes the actual method requested by the user

Because of the use of AOP, it is possible to add security to objects that were not even designed with security in mind!

Internally, the Authentication object and the Access Decision Manager use a collection of web filters that are responsible for the ultimate processing of the security requests. These filters are configured in an XML file (another example of IOC and DI use). Several examples of how these filters are used were presented in the class. This turns out to be one of the more difficult concepts in using ACEGI. It turns out that these configuration files must be carefully constructed and the order of the filters must follow explicit rules in order for the system to work correctly. Stuart is a big fan of this system and even he had harsh criticism to levy about this issue.

As mentioned earlier, a lot of the details are being elided here, mostly because of my ignorance of security. Based upon the examples shown in class however, I believe that it would be very straightforward to set up a secure system (typically web-based, but that isn’t required) using the ACEGI Security system.

NFJS - Spring AOP

2006-05-14T21:02:00.000-06:00

Presented by Stuart Halloway, Relevance, LLC

This seminar provided a deeper dive in to AOP with Spring. As mentioned earlier, Spring uses Dependency Injection and the IOC pattern to minimize dependencies. But sometimes, there are dependencies that are too broad to be removed using DI. In many of these instances, AOP can solve the problem. The simple (yet powerful) example Stuart provided was that of three inheritance trees:


SuperBeing                  SoftwareProject
 ^      ^                          ^
 |      |                          |
 |      --------                   |
 |             |                   |
SuperHero SuperVillain      OpenSourceProject         Server

Assume that each of the classes has an attribute name and we need to validate the name when it is set. Placing a name validation method in each of the four concrete classes would result in repeating the code four times. But since we understand OO concepts, we move the method into the base classes to reduce repetition. Now we only have to repeat the code three times!?! Not a great improvement. But with AOP (in Spring), we write the name validation method one time and using a Spring configuration file, attach the method to every class that needs it. This, in a nutshell, is what AOP is all about. Note that this mechanism is so powerful, we could add the name validation method to a class even if we don’t have the source code for it.

The name validation requirement in the previous example is referred to in AOP as a “cross cutting concern”. In non-AOP systems, cross cutting concerns often result in a lot of code duplication. When this code is isolated in one place with AOP, it is referred to as “advice”. The point where advice is applied in the code is called a “joinpoint”. A collection of all joinpoints for a given piece of advice is called a “pointcut”. The combination of a pointcut with its advice is called an “aspect”. Lots of new terminology, but it didn’t take long for it to make sense.

Stuart presented several examples that illustrated how aspects could be applied to a class, including method intercept, before method, after method, and throw only. The Spring framework implements AOP natively and is relatively easy to use. For higher efficiency implementations, Spring also supports AspectJ. It was mentioned that it is straightforward to start with Spring AOP and migrate to AspectJ.

It should be pointed out that the primary mechanism for implementing aspects is to modify the byte code of the class being aspected. It was interesting that Stuart offered the same advice on concerns about this that Dr. Venkat Subramaniam did at a JUG meeting a couple of months ago, which was, “Get over it!” Stuart followed this with an observation that wound up being a “Got it!” moment for me. He compared people who complain that they can’t directly see aspects in the code and therefore worry about what is being executed to similar criticisms in the past about OO and how you couldn’t see what code was being executed because of inheritance hierarchies. Not only do I remember these criticisms, I was guilty of making them early on (he admitted shamefully). So I am happy to say that I have “gotten over it” and am really looking forward to investigating AOP. I’ll probably start with the Spring version, but hope to try AspectJ soon as well.

NFJS - Introduction to Spring

2006-05-14T20:53:00.000-06:00

Presented by Stuart Halloway, Relevance, LLC

Stuart is a consultant out of North Carolina. As you will see, I attended many sessions presented by him. He is very dynamic, very knowledgeable, and has a great sense of humor. He also covered lots of Spring topics, which helped as well.

According to Stuart, Spring is Australian for no EJB. As an aside, he said that “having a pulse is Australian for no EJB 1.x or 2.x”. Kind of puts where he is at in perspective.

Spring is a framework that attempts to solve problems typically solved using J2EE, but in a more lightweight fashion. Based on what I heard over the weekend, I tend to agree. At its core, Spring is based on three core concepts. Configuration, Dependency Injection, and Aspect Oriented Programming.

Spring’s answer to configuration may be the most controversial of its features. The basics of a Spring application require minimal code, but the cost of this is large configuration files written in XML. I lost track of the number of times over the weekend a disparaging remark was made about XML. I do wonder why as much as the experts seem to dislike XML, it is still so prevalent. Anyway, if you are going to use Spring, you are going to use XML.

Dependency Injection is the mechanism that allows the details to be encoded in an XML configuration file. DI is based upon the Inversion of Control pattern. The idea is that an object doesn’t “reach out” to get other objects it needs (typically by calling new). Instead, the dependencies are “pushed to” the object from external sources. The means of this injection can be via constructor arguments or via bean-like setters. If the object is designed to depend only upon interfaces, DI can be used to modify object behavior by injecting specialized objects which implement these interfaces. And in Spring, the XML configuration files allow the injected objects to be changed without changing the code.

J2EE also uses an IOC pattern, but realizes it via context or JNDI lookups, which according to Stuart is more cumbersome. Also, evidently Spring minimizes the number of dependencies upon itself, while such dependencies are prolific in J2EE. Never having worked with J2EE, I am parroting what I heard at the seminar. I did find the explanation of Spring much easier to comprehend compared to some J2EE classes I took last year.

The final core component of Spring is Aspect Oriented Programming. If I took nothing else out the symposium, I learned that AOP rocks! I had some exposure to AOP prior to NFJS, but the message was really brought home in Stuart’s talks. I took an entire seminar on Spring and AOP, so I will cover the details in an upcoming blog. Again, the biggest downside to AOP in Spring is the massive XML burden it brings along.

During this talk, Stuart also presented examples of support services provided by Spring. He showed an example using a JDBC that showed how simple it was to connect to a database from a Spring container. He followed that up with a Hibernate example. I am really interested in Hibernate and it is dead easy to use in Spring, but for now, I plan to learn stand-alone Hibernate so that I really understand it well. After that, I will probably explore using it in Spring.

The last topic Stuart covered was Spring’s answer to security, called ACEGI. Again, I took an entire seminar on the topic, and so will reserve the details for a later blog.

NFJS - Intro

2006-05-14T20:32:00.000-06:00

One of my goals for the year was to improve my technical skills, particularly related to Java surround technology. One of the first things I did to realize this goal was to start attending meetings of the Boulder Java User’s Group and Denver Java User’s Group. Both JUGs attract excellent speakers who cover interesting and relevant topics. If you live in the Denver area, I highly recommend attending.

While I will probably blog more on these meetings at a later date, the point today is that through these meetings, I learned of the 2006 Rocky Mountain Software Symposium, a.k.a, the No Fluff Just Stuff Tour. The symposium was held near Avaya and after looking at the seminars available, I decided I wanted to attend. The first option, obviously, was to pay for it myself. While I was willing to do so, I decided to first see if Avaya would foot the bill. The company ended up sending me along with two co-workers. SCORE!!!

It was well worth the time. I learned a lot of great stuff.

Future blogs will summarize the seminars I attended. Along the way, I will also share some of the most interesting ideas and soundbites that I took away from them.

Until then, here are some general soundbites I took away from the speakers and attendees…

Java is not just a language, it is a framework.

Who needs architects?

Right to left scheduling. <= One of my favorites!

And a bonus quote from a speaker (I wish I could remember who) at a local JUG:
- Any project with more than 10 team members is bound to fail!

Why am I blogging...

2006-05-08T20:22:00.000-06:00

Short answer, cuz all the cool kids are doing it!

Longer answer. I've really enjoyed reading other peoples blogs. In particular, a friend of mine writes on some interesting things. A local technical author has also peaked my interest. And while Joel On Software isn't really a blog, Joel uses it as a forum to share ideas, which I hope to do as well.

My focus will be on some kind of geeky things, but I reserve the right to stray into unfamiliar territory when the mood strikes. I probably won't be the most consistent blogger in the world, but since I won't be the most scintillating either, you won't be missing much.

First up... The No Fluff Just Stuff seminar.