Mike Coyle’s Weblog

It seems that even the venerable Guido van Rossum (creator of Python) is weighing in on the ongoing Python web frameworks debate in two very good blog entries here and here.

Personally, I find this to be a fascinating time to be in this business. There have never been so many good technology stacks for building web applications. As I’ve mentioned in previous posts, Ruby on Rails is largely to thank for the current web application architecture renaissance. Python also has its share of very good frameworks to choose from.

However, I believe that in the coming years we will be moving away from a pure web/HTTP-based view of the world, and the value of a single server-side application being able to speak multiple protocols will become more and more apparent, as network-based applications stretch out beyond the traditional web browser.

The Python-based Twisted Networking Framework was developed to solve this specific problem of unifying multiple services in a single networked application. My favorite treatment of this topic is the chapter that Glyph contributed to Massively Multiplayer Game Development, titled Using the Twisted Framework for MMP Service Integration. Some of the code examples are a bit dated (it was published in 2003), but the conceptual material is worth the price of admission alone, IMO. Also, Abe Fettig’s new book Twisted Network Programming Essentials is quite good, with a couple of multi-protocol examples to boot.

Where will this leave the majority of the lightweight web frameworks? Unfortunately (and with absolutely no disrespect intended), it’s somewhat akin to the adage “Even if you win the rat race, you’re still a rat”. By this I’m referring to the fact that the web application framework that gains the most critical acclaim and popularity is still, by definition, a web application framework. Once you need your web app to talk to both web browsers and IM clients (or email, SMS, FTP, etc.), the game is fundamentally changed, and all bets are off.

Don’t get me wrong, web-based applications as we know them today are the dominant segment of the modern application development world, and will continue to be for the foreseeable future, but this doesn’t mean that there isn’t room for innovation beyond the typical capabilities of a mainstream web application. Specifically, presence and real-time messaging/collaboration will likely play increasingly important roles in the application development world, and Jabber/XMPP will likely become the de facto compliment to HTTP for stateful user interaction.

I’ll take this opportunity to cite a concrete example of the power of complimenting HTTP with XMPP-based messaging in a single server-centric application. We’re doing this exact type of thing with ProjectPipe today. ProjectPipe is our hosted project management solution. The web-based elements of the application talk traditional HTTP all day long between the browser and the server. However, we also provide rich integration between the web app and Microsoft Project and Excel residing on the user’s desktop. This is accomplished by using Jabber/XMPP as a compliment to HTTP (via a tiny client-side application that is installed on the user’s PC).

By building our application atop Twisted and Nevow, we are allowing our customers to do things that they otherwise couldn’t (without a bunch of additional strings attached). Having the user click on a web link and automagically generating an updated cut of their MS Project plan from server-based data is genuinely useful. Using the same technical underpinnings, users can also upload issue lists from Excel to the database server via a single click on a web page. From their perspective, transferring data back and forth between a remote database and their local desktop apps is easy to do.

Multi-protocol (HTTP/XMPP) server support has helped us take a reasonably complex integration scenario (bi-directional MS Project/Database integration) and make it appear unremarkable to our users, largely because we were able to envision and implement a solution that didn’t have to live within the architectural constraints imposed by a typical web application.

Preface: I came up with this “Three Ds” model in a post to a Radio Userland blog that I had written a few years ago. The blog subsequently died on the vine from neglect on my part. Since then, I’ve referenced the concept in conversations a bunch of times, and I have not been able to locate my original posting via Google or other search mechanisms. Since I think that the approach is still relevant, I’m taking the time to re-write it and post the article here for future reference.

I also believe that this model has relevance to the linkjacking that occurs on community sites such as reddit, which I’ll address at the end. So here we go:

There are three things that are required in moving content from the place where it happens to the eyes and ears of its final downstream consumer. These are what I refer to as The Three Ds of New Media: Discovery, Decoration, and Dissemination. The following is a description of each:

• Content Discovery: This is the work of actually finding and documenting a news story, typically performed by the folks who are first on the scene. Traditional media examples include the AP, newspaper staff reporters, and local network news personnel. Online examples include the author of original web content, or the “live blogging” that takes place at events like MacWorld.
• Content Decoration: This is the analytic, opinion, or comedic spin that the Leno/Colbert/O’Reilly crowd add to the story.
• Content Dissemination:This is the act of communicating the content out to the consumer. Note that this is not the same as being an aggregator, since aggregation isn’t always necessary (such as the real-time blogger mentioned above).

Where it gets interesting is that the Three Ds were largely “bundled” when you read the New York Times, or watched the network news. You bought the New York Times newspaper containing New York Times editorial opinions about stories covered by New York Times reporters. Now, given that there are a number of “news sources” to choose from online, the game has changed and one’s selection of news Discoverer, Decorator, and Disseminator are a function of the consumer’s trust levels and preferences. We are living in an increasingly mashed-up world.

For example, there are lots of folks whose insight and analysis I find interesting and/or useful, but I wouldn’t rely on them for sole-sourcing the authenticity of a story. These are good Decorators, but unproven Discoverers.

Conversely, as the opinions of the editorial page have been gradually creeping into front page in our major newspapers, I continue to trust the Discovery but question the Decoration.

And lastly, as it relates to the community-driven news sites, I don’t care who initially posted the content, so long as it is sourced from a Decorator or Discover (or other Disseminator) that I trust.

These relationships have a tendency to chain, given the number of content aggregators and cross-links among commentary sites. There is also often overlap, as many Decorators also own a channel for Dissemination. Rush Limbaugh is the classic case for overlap: his opinion/content is offered exclusively through his broadcast, and his broadcast is dedicated solely to his opinion/content. The AP home page is an example of the overlap between Discovery and Dissemination.

If we are able to look at news through the prism of the Three Ds and differentiate between the sourcing, commentary, and delivery of content, we may pick-and-choose our content providers somewhat freely, while still having an objective basis for ascertaining the trustworthiness of the content that we receive.

As it relates to the community-driven Disseminators (such as reddit, Digg, etc.), there is the ongoing problem of Linkjacking, a practice where someone finds a story, includes a link to the story (and not much else) in their own weblog, and then post a link to their weblog to reddit, in the interest of using public interest in the story to drive traffic to their website, even though their contribution in the value chain is minimal.

I think that the Three Ds approach provides a reasonable litmus test for posting things that you’ve found elsewhere:

• If you are simply Discovering an article that is already available on the web, then you should post the original article, rather than your weblog entry, to reddit.
• However, if you are Decorating the article with some analysis, perpective, or opinion, then I believe that linking to your weblog is totally appropriate, providing that you cite your source for the content in your weblog entry as well.

With this approach, your linkjacker-ness varies inversely with the quality of your Content Decoration. Here, linkjackers in reddit are akin to TV news analysis programs that offer no thoughtful analysis: Market forces will kick in and they will both find themselves off the air due to the empty nature of their commentary.

John Hagel has an interesting take on a Forbes article by Rich Karlgaard about Zero-Sum thinking, particularly as it relates to the business community’s unfortunate embrace of this often-flawed world view.

I had never considered zero-sum thinking as a “sickness” as Mr. Karlgaard suggests, but I think that this analogy is spot-on.

My personal rule of thumb for noticing that your falling into the Zero-Sum trap? When your inner Daffy Duck chimes in during planning or negotiations with:
“It’s mine, you understand? Mine, all mine! Get back in there! Down, down, down! Go, go, go! Mine, mine, mine!”, then you may be missing an opportunity to, as economists are apt to say, grow the pie.

Bill Clementson has a great post where he describes two high-profile cases where large companies (Yahoo and Sony) using mainstream technology platforms acquire smaller firms (Viaweb and Naughty Dog) that have gained competitive advantage using a less-popular but uber-powerful technology, specifically the LISP programming language.

While, at face value, this is an interesting story about the capabilities of LISP, I believe that a more profound (albeit obvious) message lies just below the surface:
There are often material sacrifices to be made in standardizing on mainstream technologies.

Bill’s article describes the less-than-stellar success that arose when the LISP apps were either rewritten or sidelined in the interest of integration with the acquirer’s environment.

I think that we’re currently in a period of app development history where there’s growing endorsement for non-mainstream application platforms in mainstream (i.e. non-scientific or academic) business and consumer applications.

If you look around, there are a number of cool and/or popular applications that are built atop non-mainstream platforms these days such as:

• Dabble (using the Smalltalk-based Seaside)
• reddit (using the Python-based web.py)
• Basecamp (using the Ruby-based Rails)

(Not to mention our very own ProjectPipe.com which is built atop Python and the Twisted networking framework.)

They say that all competition utlimately narrows down to a two-horse race. For a while, it appeared that those two horses were going to be Java and .NET. Now, I believe that the two horses are going to be:

• Mainstream Toolsets, namely .NET and Java (and PHP, based upon sheer numbers)
• Non-Mainstream Toolsets, such as Python, Ruby, LISP, and Smalltalk -based frameworks

where the specific toolset you pick is of lesser importance than its mainstream vs. non-mainstream identification. In other words, if it’s not .NET or Java, few people will care what you’re using.

I think that there’s a decent amount of empirical evidence to suggest that the productivity/power/agility/etc. provided by these non-mainstream toolsets far outweigh their tangible (non-FUD) downsides.

History will likely remember Ruby on Rails as the key technology nugget that really legitimized non-J2EE or and non-.NET application frameworks. While non-mainstream tools will always be considered “weird” by some, my bet is that there will be sizable growth in mindshare for these non-mainstream alternative platforms in the next two years.

One of my favorite quotes is from Hunter S. Thompson: “When the going gets weird, the weird turn pro”.

In the next 2 years, I expect that when pros want to get their applications going, they’ll increasingly turn to the “weird”.

Venture Chronicles mentions that the current Thin Client debate going on between two columnists (Mark Hall and Frank Hayes) at Computerworld looks like it was ripped from the pages of a 1999 edition.

Both thin and thick client approaches have their relative merits and downsides. At Botonomy, our applications (such as ProjectPipe) are built around a hybrid concept that we call the Right Client Architecture.

Our Right Client approach has three core tenets:

• Make the basic functionality as thin as possible (i.e. browser-based), and provide “Thickness” for a small set of high-value functions or services via local desktop software
• Where possible, leverage existing apps that the user already understands for Thick client capabilities. In many cases, this happens to be Microsoft Office (Excel, Word, Project, etc.)
• Let the user pick the right tool for the job, with the understanding that the user experience will vary by platform

We believe that this is a very compelling vision for computing across heterogeneous platforms (Windows, Macs, Palm, Phones, etc.) because we’re not trying to convince users to trade the rich desktop tools that they’re used to using for crude web-based substitutes, nor are we sitting around waiting until the base model El Cheapo PDA screams like a Cray supercomputer that’s liquid cooled with Red Bull and provides a desktop-like experience in a pocket-size form factor.

We realize that all computing devices are not equal, and that this fundamental capabilities gap will not change anytime soon. Trying to provide user experience parity across diverse platforms is a losing proposition. In that game, the best you can be is mediocre. In our Right Client model, we lay out the options and let the user pick the right tool for the job.

Let’s take the case of managing issues in ProjectPipe. When I’m using a Mac, I can input issues through a web interface. However, if I’m on a Windows PC, I can input issues in an Excel spreadsheet and then, by clicking the “upload” link on the Issues web page, automagically load the issue data from my local spreadsheet to the centralized database.

[In case you’re wondering how it works, ProjectPipe employs a small client program that makes an XMPP (Jabber) connection between your desktop and our server farm. This local client runs in your Windows system tray and allows for seamless yet totally optional desktop integration with the rest of the browser-based application.]

If we wanted a pure-play Thin Client experience that was 100% cross-platform and 100% browser-based, the guy with an existing spreadsheet with a few dozen issues that need to be entered into web form is going to cut-and-paste his way through a very tedious afternoon.

Conversely, if we built ProjectPipe as a Thick Client app in Windows, then the MS Office integration would be a cakewalk, but all of our Mac and Linux users would be out of luck.

We believe that our Right Client approach strikes an attractive and flexible balance between the two extremes. While this approach may not fit some pundits’ view of computing utopia, it keeps the heavy lifting of our applications 95% server-side, while providing a rich desktop integration experience where necessary.

]]>

The 1.0 Beta of IronPython (Python running atop .NET) is available at the IronPython Workspace Home at gotdotnet.

This is great news for Windows users because IronPython and Monad/MSH (the new Microsoft command line shell) give developers/sysadmins/power users/etc. two very attractive alternatives for scripting and automation.

Down the road, it could be viewed as a legitimate alternative to C# (and CPython) for more ambitious application scenarios as well.

Plus, Microsoft’s formal nurturing and release of IronPython should make a world of difference in Python’s acceptance rate among the “100% Microsoft Shop” crowd.

This is also great for the Python community because it makes Python “not irrelevant” to the scores of seasoned developers who have invested a ton of time and energy becoming one with the .NET Frameworks. They can Embrace python because it Extends their mental investment in .NET.

Embrace and Extend. That has a nice ring, doesn’t it

]]>