Mike Coyle’s Weblog

Here’s my recommendation for evaluating the Google App Engine: Think of it as the world’s biggest, baddest embedded device, like a smartphone designed by Chuck Norris.

Remember that, just like the folks that build PDA software, you are writing code to be deployed atop a purpose-built, specialized hardware/OS combination that has profound intentional differences from your Mac/XP/Linux desktop environment. And while the hardware/OS vendor (i.e. Google) does everything that they can to bridge that gap, many of these differences cannot be effectively abstracted away.

The teams that will successfully launch apps atop the Google App Engine are the ones that understand and embrace the nature of the platform (both pros and cons), rather than those that grudgingly work around its idiosyncrasies, pining for the Ubuntu-y goodness of their favorite deployment environments.

Folks who treat App Engine simply as an alternate deployment platform for Python-based web applications run the risk of hitting the wall before they get their app out the door. Anything but the simplest applications will likely require some serious RDBMS-to-”Datastore API” architectural transmogrification. And there are still a bunch of Python libraries (Paste, Mako, etc.) that cannot be used in the App Engine environment.

For time spent in the App Engine world will make for an interesting life:

Infinite scalability. Minimal administrative headaches. But no way to run a query with a “GROUP BY” clause.

Plan accordingly.

Google App Engine was announced last night. This is a service that allows you to write a web application in Python and deploy it to Google, where it can run atop Google’s uber-scalable infrastructure. Here is the transcript and video of the announcement, where Kevin Gibbs from Google explains the App Engine’s value proposition (Easy to use, Easy to scale, and Free to get started.)

Like most big announcements, it’s the most sensational aspects of the story that emerge first. But I think that the broader impact of Google App Engine is a fundamental rethinking of the relational database value proposition among the RDBMS’ staunchest allies. Google makes a compelling case for deployment, so long as your application can live within its constraints.

I see data management as the biggest impedence mismatch between the Google App Engine and the traditional web application design. However, over time I’m certain that a set of patterns will emerge to aid the masses in bridging the gap between the relational model and the Datastore API.

They have some great stuff in the Datastore API, and Christian and I are already at work putting together some offerings to help companies attracted to Google’s Infrastructure-as-a-Service model objectively evaluate and/or embrace the App Engine for their web application needs. Email me if you’d like to discuss.

So that being said, I’ll argue the following:

• Is it a giant piece of flypaper to help Google acquire the next generation of startups? Maybe, but I contend that the real killer next-generation apps will need more feature-rich networking capabilities. Let’s wait until we see what kind of XMPP/GoogleTalk integration comes along.
• Is it a sign that commodity web hosting providers are going the way of the corner video store? Perhaps.
• Is it a welcome counterbalance to the Ruby-on-Rails hype machine in the dynamic language arena? Yep.
• Is it a final blow in the in the Google vs. [INSERT NAME HERE] battle for the hearts, minds, and wallets of the IT crowd? No, the Infrastructure-as-a-Service game is just beginning. The Amazon infrastructure offerings are generalized and piecemeal (by design), whereas the Google App Engine is more of a “vertical” play tailored to traditional data-backed web applications.
• Is it a symbolic victory of Django over Pylons in the Python web frameworks world? No, it’s a victory for WSGI. Django just happens to be up on stage, ably and graciously accepting the award on WSGI’s behalf. I think that the stuff coming out of the App Engine ecosystem in the coming weeks will provide additional validation to the idea that WSGI is Python’s not-so-secret weapon.

Lastly, I’ll throw out some predictions, none of which are Earth-shattering:

• An Open Source application stack that emulates the Google App Engine SDK’s runtime environment will emerge from the community for folks that want to build applications that are compatible with the App Engine, but do not want to deploy their application to Google’s cloud at the current time. Also, this stack would provide an option for companies that deploy a custom app to the App Engine, but subsequently wish to take their application in-house.
• Within time, you’ll see integration with other elements of the Google infrastructure (Docs, GTalk, Search, etc.).
• The next language to be supported will be JavaScript, but that will not come for quite some time.

A couple of months back, I had the pleasure of being interviewed by Susan Ladika for PM Network magazine (a publication of the Project Management Institute). They were doing an article about managing small project teams for the March 2008 issue.

I put together a bunch of content in advance of the interview. Now that the magazine’s hit the streets, I figured that I’ll publish the stuff that I wrote that didn’t make it into the article.

I did my prep work by writing out a fake interview with myself. I’m certain that I’ll recycle some of this content in future writings and talks, but here it is in its original form for your reading enjoyment:

–[BEGIN FAKE INTERVIEW]—————————————–

1) Your company uses the slogan “We help small teams solve large problems.” Please explain.

The needs of small, disciplined project teams are largely unrecognized and under-served in the project management community. We started our company to bring the necessary people, process, and technology to small teams so that they can leverage their constraints and deliver value to their customers in a predictable and instrumented fashion.

Too many small project teams either wander their way through the project lifecycle alone, or end up suffocating under a heavy, inappropriate layer of bureaucratic oversight that makes project delivery slower and more painful, but doesn’t necessarily reduce any tangible delivery risks.

We know that there’s a better way, so we put our money where our mouth is. ProjectPipe.com is our project management software solution that embodies our approach to disciplined, high-performance small team project management. Our target customers in corporate settings are the small boats of ambition floating in the sea of big company mediocrity. While we’re not here today to discuss ProjectPipe, any interested readers can find out more at ProjectPipe.com.

We are convinced that the future of project delivery is small, distributed teams. We are fortunate to be in a position to expedite this trend through our software, consulting, and writing.

2) But isn’t bigger generally better?

No. Having more people on your team is not like having more money in the bank.

For example, jazz quartets don’t want to be orchestras when they grow up. The two are different beasts.

If you try to run a jazz quartet like a symphony, you get bad jazz. Run a 20-piece orchestra like a jam session, and you end up with chaos. The same goes for project teams. If you run a small project like a miniature large project, you end up with….pretty much what you see in corporate environments everyday. Namely, too many meetings, lots of low-value boilerplate documentation, and other adminstrative overhead that gets in the way of solving the real problems at hand.

Don’t get me wrong, a team of 4 people still needs to play by the book, but it’s a different book.

3) Why does the size of a team matter? Aren’t there universal best practices for project management?

At a high-level, there are a number of universal truths that the Project Management Institute, and other organizations have done a wonderful job categorizing and articulating. However, when it comes time to tactically implement these best practices, you see the entire spectrum of implementation choices, from hyper-bureaucratic process overkill to Wild West-style chaos.

In theory, practice and theory are the same. In practice, they are profoundly different.

This is why it is so important to have strict adherence to a lightweight process. As a project manager, you want to spend your overhead capital wisely, applying documentation and other process-related protocols to the specific areas of project risk that are relevant to your project.

4) What types of things are more difficult for small teams, aside from the shortage of manpower?

There are three general challenges for a small team. First of all, distractions and overhead can be a significant drain of time and attention. Second, context switching can be expensive, as members of small teams often wear multiple hats. Lastly, having fewer team members to draw insight from can result in less diverse domain expertise brought to the table.

5) Are there any advantages that come from having a small team?

Certainly. Small effective teams understand the nimble clarity that comes with having few moving parts. They have conversations rather than meetings.

Small teams separate the wheat from the chaff, since there’s nowhere for underperformers to hide. As a result, I’ve found that higher-skilled resources are more likely to prefer working in smaller teams, since there’s a bit of self-selection at play. For example, rock stars know that poor performers are more interested in being the 16th person on a 16-person team than the 3rd person on a 3-person team.

6) What kinds of special needs do small teams have?

Small teams need to behave like small teams, not miniature versions of large teams, because bureaucracy doesn’t scale down particularly well. Small teams need to be led by a project manager that understands the dynamics of small team delivery. Specifically, someone who picks their battles carefully and doesn’t mind getting their hands dirty from time-to-time. Additionally, small teams need to use the best tools available, as they cannot afford to arbitrarily throw labor at problems.

7) Are there particular issues that come when team members are dispersed around the globe?

Small teams should strive for “location transparency”, regardless of the team’s physical locale. Collaborative technology (and the day-to-day practices that support collaborative processes) should be employed whether team members are separated by a hallway or an ocean.

8) What ways can small teams work most effectively to accomplish big tasks?

There are four aspects that drive small-team effectiveness: Planning, Process, Automation, and Asset Management.

• Planning: Small teams need to focus their efforts on understanding how the key moving parts of their project fit together so that they can proactively categorize and manage delivery risk. This is far more important than trying to optimize the workload balancing in your project plan, since you’re not trying to distribute lots of work among interchangeable resources on a large team.
• Process: Small teams should maintain strict adherence to a lightweight process. Individual heroics are not scalable nor predictable over time. Even small teams need to play by the book, but they need to make sure that it’s the right book for them.
• Automation: Small teams need to maximize the amount of work that they don’t do, and focus their efforts on the important stuff. If work cannot be avoided, it should be automated. Computers are really good at doing predictable stuff really fast.
• Asset Management: Small teams should manage their important data using some sort of version control mechanism. A good version control system acts as a time machine in case users need to review and/or compare previous versions of a document or other artifact, and a bank vault to protect a single, current version of the truth for each project asset.

9) I also found the comment intriguing that large teams are usually bands of small teams in disguise, and I’d like your thoughts on that.

We coined that phrase to help decision makers understand the true nature and composition of their organizations, and soften any negative connotations that the “small teams” concept would have in larger organizations.

At the end of the day, projects get talked about by large teams and completed by small teams (or groups of small teams working together).

Any team can make a decision. Only small teams make great decisions.

Any team can do work. Only small teams do efficient work.

It’s true that there is strength in numbers, but sometimes that strength lies in small numbers.

–[END FAKE INTERVIEW]——————————————

IBM invests in PostgreSQL vendor EnterpriseDB.

This is good news for two reasons: First, it throws yet another (big) blue chip name behind the PostgreSQL brand and ecosystem. IBM was one of 4 firms participating in EnterpriseDB’s C round of venture financing, which raised $10 million for the firm.

Second, EnterpriseDB is also rebranding their value-add packagings of PostgreSQL as Postgres Plus and Postgres Plus Advanced Server. The products used to be branded as EnterpriseDB Server and EnterpriseDB Advanced Server respectively. This new naming scheme should bring more brand recognition to the underlying PostgreSQL project.

For those not familiar with EnterpriseDB, their main claim to fame is an Oracle compatibility layer that they’ve built atop PostgreSQL. This allows firms to save a few bucks by replacing Oracle databases with EnterpriseDB’s distribution of PostgreSQL. They also provide support and other value-added software and services for customers using PostgreSQL.

Speaking of databases, I highly recommend checking out FLOSS Weekly: Episode 26 where the guest is D. Richard Hipp, the creator of SQLite. SQLite is the most widely distributed relational database in the world. Aside from the interesting technical content, the interview can stand on the merits as a human interest story of how one man’s pet project can have a huge impact on the world.

Great article on Jive Software’s Community Blog: XMPP (a.k.a. Jabber) is the future for cloud services.

The Tivo example that Matt provides is good conversational fodder if you ever need to explain why Polling is Evil to a non-technical audience.

Personally, I don’t love the “Cloud Services” moniker, but that’s neither here nor there. Regardless of what they call it, we’re starting to see that the app networking world is sliding into standardization, using HTTP for simple round-trip synchronous communications, and XMPP for asynchronous communications and/or stateful conversational communications.

The consolidation of the major Instant Messaging providers around XMPP will do much to glue the “Web” and “Non-Web” elements of the Internet together in ways that we’ve only started to think about.

Also, here’s a link (found in the comments of the aforementioned article) about the new XMPP Logo.

We recently announced the re-branding of our consulting practice as PyGuys.com. By marketing on a brand-by-brand basis rather than at the company level, we think that we’ll be able to improve the signal-to-noise ratio for the targeted audiences of our various products and services. For example, our Python consulting customers may not have a need for our project management software, and vice-versa.

That being said, the casual observer may wonder why we’re so gung-ho over Python when it appears that all of the cool kids are using Ruby on Rails. When we started the company in 2005, there was still plenty of room on the Rails bandwagon. Our adoption of Python was deliberate based upon the maturity of its language, libraries, and community.

Quite simply, we’re in this for the long haul, and feel that Python has the best prospects of being the dominant dynamic language in mainstream IT settings over the next 36 months. Methinks that the initial hype surrounding Rails has already started to subside, and Rails will continue as a good way of building standard web applications, especially for solo developers and small teams.

It just won’t end up being the way to build web applications.

Don’t get me wrong, the Rails folks have done a lot to validate the use of dynamic languages in mainstream settings, and the Ruby language itself has a bunch of cool and features. You will find no negativity here. But I think that Rails’ great and durable contribution to the field of software engineering is not its libraries and syntax, but rather its philosophy of “Convention over Configuration”. That philosophy transcends technology stacks, and has already found its way into a slew of web application frameworks on a number of platforms.

But when the technology rubber meets the Enterprise IT road, I believe that Python (and its community/ecosystem) is a better and more accessible choice than Ruby for most companies looking to adopt a new application development platform, current conference attendance statistics nonwithstanding.

With PyGuys.com, we’re formalizing our Python advocacy in the branding of our consulting services. In hindsight, we should have done this 3 years ago. So to paraphrase Barbara Mandrell (and there aren’t too many times in life when one gets to say that), we were Python when Python wasn’t cool. (Note: link contains audio)

This posting provides some background for our new Python Center of Excellence (PyCOE) consulting offering.

We are slowly entering the post-honeymoon era of Ruby on Rails (RoR). For a while, the IT press was head-over-heels in anointing RoR as the heir apparent to J2EE/.NET development in the Enterprise. Now, there is a much more balanced pro/con view of RoR, especially among the Alpha Geeks.

While the press talks about IT fashion du jour, I think that IT and business folks alike are secretly pining for the next “Visual Basic”, where an easy-to-learn syntax coupled with powerful libraries can be used by normal people to solve real problems quickly in a variety of contexts.

Python fits that bill quite nicely, and is attractive to both hardcore programmers and the “IT is my day job” crowds alike.

Other languages are arguably more [INSERT ADJECTIVE HERE] than Python, but few combine its expressiveness and ease of learning.

Unfortunately for the typical IT manager, it’s hard to figure out what technology horse to bet on, since the popular bets are rarely the good ones. Taking the advice of the high-end IT sages will land you smack-dab in the middle of the pack at best. And following the advice of your sharpest go-to person may yield an approach that doesn’t scale across the talent continuum.

Stop me if you’ve heard this one before: Bob is the smartest guy in the IT department. Bob falls in love with Rails, and makes the pitch to his boss that they should do their new development in Ruby. Bob backs this up with circa-2006/2007 praise that Rails received from…everywhere. They get started and the boss quickly learns three things:

• Bob is happy working in Rails. Woohoo!
• The simple stuff gets done quickly in Rails. Woohoo!
• Everything else carries a pretty steep learning curve for everyone but Bob. D’oh!

Our PyCOE offering provides a contrasting model, one that states that You should not base technology selection solely upon the tastes and preferences of your sharpest employee.

Rather, PyCOE provides a nine-step process for helping companies adopt Python and incorporate the set of tools and approaches that are appropriate to their existing environment, making sure that the tool selection is consistent with the culture, charter, and legacy considerations of the organization, among other factors.

Paradoxically speaking, it’s exciting to help make dynamic language adoption become boring and predictable.

Introduction

As Christian mentioned the other day, we have two new service offerings in 2008, both centered around Python consulting:

• PyCOE.com is our new offering to help organizations build core competencies in Python in a deliberate and predictable fashion in a way that is fundamentally different from the way that firms tried to adopt J2EE and .NET in the past.

  In a nutshell, we help companies create an environment where the easy thing to do is also the right thing to do for their developers, analysts, and QA personnel. We focus on culture, standards, and infrastructure as we help our customers create a Python Center of Excellence (or COE, hence the PyCOE name) in their organization. We do this by approaching Python adoption as business process re-engineering for IT, rather than programmer training.

• PyGuys.com is our new Python consulting brand. We are focusing on small-to-midsize custom development projects, and SDLC process improvement consulting. We’ve been doing Python and Project Management consulting since the company started, but we felt that now was a good time to rebrand and grow our Python consulting business.

We also reworked the Botonomy.com company website, to reflect its role as a “holding company” for our branded service offerings.

Also, just before we launched our new websites, TIOBE declared Python as the programming language of 2007. Not a bad sign, if you ask me.

Here’s an Apple Developer Connection profile of Checkout, a Point-of-Sale application developed in Cocoa and Python, using PyObjC (Python<->Objective-C bridge) to glue the Python-based business logic to the UI and other framework-level Cocoa libraries.

On the value of Python and development process:

Bok explains how this combination contributed to the development efforts. “Using Python and the PyObjC bridge gives you the best of all worlds—Python and Cocoa,” Koen says. “Python is great for unit testing and agile development. Because Checkout manages critical business information, we swear by writing unit tests for everything that even comes close to any financial data.”

Koen continues, “And since Python is interpreted and not compiled, launching a debug session takes seconds. You can make a modification and see the effect of the change immediately.”

(From MacDevCenter.com)

This article in Guardian Unlimited provides a good introduction to SQLite and the story behind it.

Since SQLite is the database used by cool stuff like Google Gears, Apple’s Core Data, and (cough) ProjectPipe, it’s great to see it get positive coverage in a mainstream news outlet.

The article is also a veritable infomercial for automated testing:

Hipp attributes much of the reliability of SQLite to his use of tests. Less than half of the program code is delivered as the database engine. The larger part consists of thousands of automated tests, which exercise the code and check that the results are as expected. “That’s the real key to keeping it working well,” he says.

One of the goals for SQLite was to keep the database library very small (under 250KB). If Dr. Hipp et al. hadn’t spent all that time writing automated tests, I’m sure that SQLite would also have a very small user base as well