Mike Coyle’s Weblog

Scratch is a new kid-friendly visual programming environment built atop Squeak Smalltalk.

It is very intuitive, multimedia-oriented, and looks great. If you’re at all interested in getting your kids or non-technical friends/family into programming, I highly recommend checking it out.

…it is from kicking the shins of the giants around them.

I’ve read a couple of blog entries from folks coming back to Python after jumping on the Rails bandwagon, citing the temperament of the community as one of the reasons.

I never thought much of it, until I read the DHH’s response to Joel Spolsky’s Language Wars article, and the comments that follow. Sure, there are a number of comments that are constructive and forward looking. But you have to look for them. Carefully.

I think that Joel Spolsky has done a lot for the software development community, and was not paid the respect that he deserves in DHH’s response and a multitude of reader comments that followed.

I’ll be honest, I’ve considered jumping on the Rails bandwagon. Several times. The thing that has held me back is the notion that Ruby/RoR hasn’t really gone through its adolescence yet.

If you look at the folks who are doing great things with RoR, lots of them (37Signals, the Pragmatic Programmer guys, etc.) would be writing great software with _any_ tools that they were using. It’s like the fact that Chet Atkins could make a toy guitar sound great.

Maybe it’s my ignorance of the inner workings of the RoR community, but I get the sense that RoR hasn’t had its “holy $#%&” moment, when the hard lessons are learned by average-to-good corporate IT developers, and the sound of pagers at 3AM drowns out the cheers of the language evangelists.

I think that Microsoft had its when they started playing Paper-Scissors-Rock with DCOM and Java. I think that J2EE had its when the Microsoft Pet Shop spanked the canonical best practices of heavy EJB development. Python had its when the momentum of RoR highlighted the fact that there were 80 web frameworks that each did 80% of what people need, and no knee-jerk answer for which one to use for any given application.

But the Rails community? Not so much. I don’t think that the RoR honeymoon is over just yet. Someday it will, and the hard lessons that come will forge the legitimate counter-arguments to Joel’s points.

ITConversations recently published an OSCON 2005 session by David Heinemeier Hansson titled Secrets behind Ruby on Rails.

Even if you’re not a Ruby developer (which I’m not at the moment), this podcast episode is 16 minutes well-spent. In defending his position that “flexibility is overrated”, David makes the following comment:

“Too many technologists are chasing flexibility as thus it was free. It is not.”

Amen.

There’s no shortage of developers and architects who not only err on the side of configurability and/or future-proofing, but cut the wheel and swerve hard in that direction. My counter-arguments and counsel have traditionally drawn upon the Extreme Programming credo of YAGNI (You Aren’t Gonna Need It), and the Zen of Python, among other software writings to complement my first-person experiences. The rapid adoption and success of Rails provides additional points of reference that are well-grounded not just in theory, but in the heads-down, pragmatic implementations that moves our craft forward.

Even if you are not working on a Rails project, there are software architecture lessons to be learned from the successes and challenges that the Rails community faces as it moves through its adolescence.

I like to make the analogy that application architecture is much like a skeleton. It’s comprised of bones and joints. Bones are the well-defined elements of the application, the behaviors of the application that are well-understood, and therefore can be built rigid and strong. Joints are the points of flexion, accounting for uncertainty and/or future change either through dynamic behavior or external configuration.

IMO, the challenge of good software architecture lies in the proper layout of bones and joints in building the skeleton of your application.

If you have too much bone and too few joints, your architecture is unnaturally rigid, and making change involves literally breaking a bone and then coding your way through its repair.

Conversely, if you have too many joints and too few bones, your application will not have enough internal structure to stand on its own. Now your application probably needs to lean on an XML Configuration-based walker just to stand upright. It doesn’t get much less agile that that, metaphorically speaking.

The other issue that frequently arises is the misplacement of flexion points, making the application flexible in areas when far simpler solutions would suffice, and hardcoding specific behavior where flexibility is genuinely useful.

The classic example of misplaced flexibility is the case where an architect in a solid 100% Oracle (or 100% Microsoft) shop designs a complicated and opaque “data abstraction layer”, so that the application would require minimal changes in the event that the company nullifies long standing enterprise vendor relationships at some point in the future and forces the migration of all legacy applications built atop database vendor A to now run atop database vendor B, and the company chooses to not leverage such a seismic event to refactor, rewrite, or retire said application. In this example, the codebase is encumbered (and a chunk of the budget depleted) in the interest of building functionality that addresses a risk that is very, very unlikely to materialize.

If you’re that forward thinking, you might as well code your application to be flexible enough to address the Y10K Bug

By treating your application architecture as a skeleton, you will hopefully be able to put flexibility in your application where it makes sense, and reduce the risk that well-intentioned but misplaced flexibility becomes a skeleton in your application’s closet.

P.S. While some may consider the previous sentence to be a mixed metaphor, I disagree. Given the “Web 2.0″ world in which we’re living, I prefer to think of it as a Metaphor Mashup.

It seems that even the venerable Guido van Rossum (creator of Python) is weighing in on the ongoing Python web frameworks debate in two very good blog entries here and here.

Personally, I find this to be a fascinating time to be in this business. There have never been so many good technology stacks for building web applications. As I’ve mentioned in previous posts, Ruby on Rails is largely to thank for the current web application architecture renaissance. Python also has its share of very good frameworks to choose from.

However, I believe that in the coming years we will be moving away from a pure web/HTTP-based view of the world, and the value of a single server-side application being able to speak multiple protocols will become more and more apparent, as network-based applications stretch out beyond the traditional web browser.

The Python-based Twisted Networking Framework was developed to solve this specific problem of unifying multiple services in a single networked application. My favorite treatment of this topic is the chapter that Glyph contributed to Massively Multiplayer Game Development, titled Using the Twisted Framework for MMP Service Integration. Some of the code examples are a bit dated (it was published in 2003), but the conceptual material is worth the price of admission alone, IMO. Also, Abe Fettig’s new book Twisted Network Programming Essentials is quite good, with a couple of multi-protocol examples to boot.

Where will this leave the majority of the lightweight web frameworks? Unfortunately (and with absolutely no disrespect intended), it’s somewhat akin to the adage “Even if you win the rat race, you’re still a rat”. By this I’m referring to the fact that the web application framework that gains the most critical acclaim and popularity is still, by definition, a web application framework. Once you need your web app to talk to both web browsers and IM clients (or email, SMS, FTP, etc.), the game is fundamentally changed, and all bets are off.

Don’t get me wrong, web-based applications as we know them today are the dominant segment of the modern application development world, and will continue to be for the foreseeable future, but this doesn’t mean that there isn’t room for innovation beyond the typical capabilities of a mainstream web application. Specifically, presence and real-time messaging/collaboration will likely play increasingly important roles in the application development world, and Jabber/XMPP will likely become the de facto compliment to HTTP for stateful user interaction.

I’ll take this opportunity to cite a concrete example of the power of complimenting HTTP with XMPP-based messaging in a single server-centric application. We’re doing this exact type of thing with ProjectPipe today. ProjectPipe is our hosted project management solution. The web-based elements of the application talk traditional HTTP all day long between the browser and the server. However, we also provide rich integration between the web app and Microsoft Project and Excel residing on the user’s desktop. This is accomplished by using Jabber/XMPP as a compliment to HTTP (via a tiny client-side application that is installed on the user’s PC).

By building our application atop Twisted and Nevow, we are allowing our customers to do things that they otherwise couldn’t (without a bunch of additional strings attached). Having the user click on a web link and automagically generating an updated cut of their MS Project plan from server-based data is genuinely useful. Using the same technical underpinnings, users can also upload issue lists from Excel to the database server via a single click on a web page. From their perspective, transferring data back and forth between a remote database and their local desktop apps is easy to do.

Multi-protocol (HTTP/XMPP) server support has helped us take a reasonably complex integration scenario (bi-directional MS Project/Database integration) and make it appear unremarkable to our users, largely because we were able to envision and implement a solution that didn’t have to live within the architectural constraints imposed by a typical web application.

Bill Clementson has a great post where he describes two high-profile cases where large companies (Yahoo and Sony) using mainstream technology platforms acquire smaller firms (Viaweb and Naughty Dog) that have gained competitive advantage using a less-popular but uber-powerful technology, specifically the LISP programming language.

While, at face value, this is an interesting story about the capabilities of LISP, I believe that a more profound (albeit obvious) message lies just below the surface:
There are often material sacrifices to be made in standardizing on mainstream technologies.

Bill’s article describes the less-than-stellar success that arose when the LISP apps were either rewritten or sidelined in the interest of integration with the acquirer’s environment.

I think that we’re currently in a period of app development history where there’s growing endorsement for non-mainstream application platforms in mainstream (i.e. non-scientific or academic) business and consumer applications.

If you look around, there are a number of cool and/or popular applications that are built atop non-mainstream platforms these days such as:

• Dabble (using the Smalltalk-based Seaside)
• reddit (using the Python-based web.py)
• Basecamp (using the Ruby-based Rails)

(Not to mention our very own ProjectPipe.com which is built atop Python and the Twisted networking framework.)

They say that all competition utlimately narrows down to a two-horse race. For a while, it appeared that those two horses were going to be Java and .NET. Now, I believe that the two horses are going to be:

• Mainstream Toolsets, namely .NET and Java (and PHP, based upon sheer numbers)
• Non-Mainstream Toolsets, such as Python, Ruby, LISP, and Smalltalk -based frameworks

where the specific toolset you pick is of lesser importance than its mainstream vs. non-mainstream identification. In other words, if it’s not .NET or Java, few people will care what you’re using.

I think that there’s a decent amount of empirical evidence to suggest that the productivity/power/agility/etc. provided by these non-mainstream toolsets far outweigh their tangible (non-FUD) downsides.

History will likely remember Ruby on Rails as the key technology nugget that really legitimized non-J2EE or and non-.NET application frameworks. While non-mainstream tools will always be considered “weird” by some, my bet is that there will be sizable growth in mindshare for these non-mainstream alternative platforms in the next two years.

One of my favorite quotes is from Hunter S. Thompson: “When the going gets weird, the weird turn pro”.

In the next 2 years, I expect that when pros want to get their applications going, they’ll increasingly turn to the “weird”.