Tom Raftery recently wrote a piece calling for public cloud providers to be more open about the energy footprints of their services to allow for customer and consumer benchmarking. You might expect the likes of Amazon and Google would be open to publishing a footprint, but sadly that’s not the case yet. The Silicon Valley leviathans are doing some great work in terms of efficient IT – see Facebook’s OpenCompute initiative for example.

But its interesting that a company with a rather different heritage is banging the drum for sustainability metrics in the cloud. Step forward traditional outsourcing and systems integration firm CSC, and vp of cloud computing Siki Giunta.

“We really need to really understand a workload, how long it runs. We need to understand the rhythm of the business, and provision to that… At the moment metrics collection is all over the place.”

Giunta said that regulatory environments such as the UK Carbon Reduction Commitment would start to force enterprises to be more rigorous about energy monitoring and management. But what should you measure, in order to get a better handle on energy use?

Obviously we eventually need to instrument everything, for an internet of things that drives more sustainable outcomes, but Siki argues that a simpler metric would be a good place to start.

“In terms of servers a common area of metrics is RAM. It doesn’t matter how many VMs you what matters is RAM. But customers don’t know the RAM capacity…. of their workloads. they just provision to the spike.”

“Going forward we’ll see on spot memory… spot markets. At a couple of banks, like energy markets today- there is a spot rate. In IT RAM is the metric – like kilowatts.”

Its very early days, but its good to see Giunta leading the debate, talking to her customers, and folks like the CSC Advisory Council, about measuring server use, and moving towards more sustainable clouds.

Given that CSC bills for cloud on the basis of RAM you can see the attraction of a RAM-based energy measurement metric.

I also like the idea of a brute force metric so organisations can’t use complexity as an excuse not to report on energy use.

A core research thesis for me is that the best packager in any tech wave wins, and wins big. The number one position in a market, with all the network dominance that implies, goes to the best packager, not the best inventor. Packaging is where you cross the chasm- where geek stuff goes mainstream. Tim O’Reilly tracks the Alpha Geeks to understand how the world will change- but generally the mainstream chooses a different implementation that the geeks- Apple being an honourable exception to the rule (when application developers and the mainstream are pulling in the same direction then network affects really kick in).

Think of Compaq packaging Wintel. Or IBM packaging all of the inventions in mainframe computing as the System/360. Windows packaging the IP stack to take out Novell. SAP packaging ERP/MRP/etc for client/server in the shape of R/3. Or lets look again at Apple for a second: Nokia and even Microsoft had delivered all of the same phone technologies Apple later did, touchscreens and so on- but they packaged them far less effectively, focusing on functionality rather than user experience. On the desktop Apple packaged FreeBSD on Intel and made it beautiful.

Geeks love great packaging. So much so they fetishise the “unboxing” of new gear.

So what about open source, and the news this week that Red Hat is set to break the $1bn annual revenues barrier? Matt Asay argues that the company is one off. It seems to me that Red Hat, more than any other company in the space, simply realised the value of enterprise packaging – with all the documentation, certification, authorisation, training and platform testing that implies – across a wave of open source technology – namely Linux, and latterly Java application servers. Red Hat is a not a distribution for geeks – its a distribution for suits. While other commercial open source companies trumpet the fact the lead developer of an open source project works at their firm, Red Hat just gets on with business.

Last June my colleague Stephen said, in a post entitled The Economics of Open Source: Why the Billion Dollar Barrier is Irrelevant:

Open source as an application development model enjoys many advantages over proprietary, in-house development; distribution and usage among them. But revenue extraction has not traditionally been a strength, for obvious reasons. When payment is optional, as it is with most open source software, fewer users become commercial buyers.

Stephen was responding to Glyn Moody, who had simply asked: Why No Billion-Dollar Open Source Companies?

These days all software companies use open source methods- even Microsoft. Open source has won the battle as to what is the best development model for quality and agility. But packaging is where the value is. Red Hat won the shrinkwrap and enterprise wave of Linux, and deserves the money its making. It turns out though that the Cloud though seems to be the best commercial model for packaging open source technology, not least because it removed some of the restrictions of distribution-based open source licenses. It will be interesting to see how Red Hat sustains its momentum – after all Amazon is currently doing by far the best job of packaging Red Hat for the cloud – in the shape of Amazon Web Services. [update: realising I should spell this out- AWS runs on Red Hat Enterprise Linux, and uses its virtualisation engine.]

Packaging is how to make money from open source. But then its been the way to make money from every tech wave. Unix – a packaging exercise that Sun won until they took the eye of the Intel ball. What’s your favorite example? Packaging is innovation, everything else is devs writing code. Finally- a few words about broader economics. One of the great advantages of packaging is the margins it enables. Nestle packages water, and sells it at a huge profit. Bayer is the aspirin packager. Branding is packaging, and that’s where the money is.

If you’re interested in the packaging angle you might also enjoy this recent post Smarter Computing: An Agenda For (Re)Packaging IBM Software and Systems in the Age of The Cloud.

I also argue that Google’s Android is a packaging exercise – Towards a Permission-based Web. Wherefore Net Neutrality? Or: Maybe Open Source Wins After All

disclosure: IBM, Microsoft, Red Hat and SAP are clients.

EMC: Private/Public Cloud to Stabilise: 80/20 by 2015

I went to an EMC event back in January, and of course the topics of discussion were cloud, cloud and uh cloud – after all what’s the biggest driver of storage today? But on the subject of cloud one slide really stuck in my mind. Enough that I am posting on it now. I am a private cloud skeptic – indeed I posted the 15 Ways to Tell Its Not Cloud Computing back in March 2008- so the idea we’ll have settled into a simple Pareto distribution by 2015 seemed kind of crazy. Are we really going to settle into a simple 80/20 for Private/Public cloud within four years or so?

Now EMC made it very clear this was just an “illustrative” example, but its certainly a pointer to how EMC sees the market playing out. I am pretty sure Amazon would have a very different view: maybe 20% private cloud to 80% public by 2020? Actually I would love to know what Amazon’s planning assumptions fotr AWS. One problem is its so hard to measure the cloud market – everything gets labeled private cloud these days – its kind of a placeholder for Enterprise IT.

Anyway – the point of this post is that really I’d like to know what you think. How do you see this stuff shaking out?

OK – lets try again now shall we?

Evidently I spent a couple of weeks obsessing about something I misinterpreted. However I still think the model is interesting- notably for the fact EMC doesn’t expect to see much if any growth in SaaS, as share of overall enterprise workloads, between 2015 and 2020. Why would SaaS stabilise rather the grow after 2015? I mean- I would expect to see a lot of, for example, Microsoft Exchange to head into the cloud. And then there’s more Salesforce, more RightNow, more the list is endless. Of course Infrastructure as a Service and Platform as a Service will continue to grow – and I woiuld expect them to grow faster than EMC expects. All in all I really wanted your thoughts on this. What do you think of EMC’s planning assumptions?

I must stress these models are illustrative.

disclosure: EMC is not a client, but VMware is.

Yay!

I wrote a post a while ago – Reading is Writing: Illuminating The Digital Manuscript – that argued the role of annotation can be as important as primary content when it comes to reading, learning, and knowledge. Celebrate marginalia!

I believe this is the future of digital publishing. Learn from open source. The idea of content lock down just makes no sense. Paper books don’t have DRM. You can share them, write on them, cut bits out for your scrapbook and so on. But imagine if you could do all that digitally…

Why shouldn’t books be a little more like Wikipedia and a bit less like a copy-protected CD?

It might seem like the editable, annotatable, shareable book is a pirate’s charter, but publishers have little choice but to adapt.

It seems the adaptation is happening right now. This morning I came across this Steven B Johnson piece in the New York Times about the latest Amazon Kindle functionality, with its new “popular highlights” function. Users will be able to annotate the lines in a book they find most interesting. The idea of the book group just took a hit of crack. The world can now share thoughts on a manuscript. The opportunities for a social networking play should not be underestimated. If Amazon becomes a de facto standard location to annotate the written word, imagine the implications for universities, businesses and everybody. I see this as a big deal. Big enough that I now want a Kindle…  Where’s that Kindle for Android app, anyway?

sd

Amazon CloudFront is a new distributed caching mechanism, designed to get data closer to the user. Amazon says:

It integrates with other Amazon Web Services to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no commitments.

Which is nice and all- but I need to be hearing it from developers. Which is why this caught my eye the other day…

geek footnote: the bigass images on dopplr’s new city pages are served from Amazon’s Cloudfront CDN. And it was really easy.

02:51 PM November 27, 2008 from twitterrific

Needless to say I twittered the comment, and discovered that others in the network are having similar experiences. See davejohnson. Of course these are just two data points, but the fact is my dial is turned very clearly to Matt Biddulph. He is someone I have the greatest respect for, and his technical chops are unimpeachable. I shared an office with Matt and learned years worth of web development technique in months.

Its also very easy using twitter search to come up with further agreement. see @jalegre:

Very happy about Amazon CloudFront, about 100-120 msec for static files like images, js, etc… in Spain (going to France). Faster than S3

Its not rocket science its findability. Meanwhile from a marketing perspective Amazon service naming is impeccable. The company hasn’t put a foot wrong since it started marketing its excess capacity to developers, and naming is very important in framing a market. On Twitter I said:

in other news – cloudfront- what a brilliant product name. descriptive, evocative, right on the money. others could learn from amazon naming

I was reading The Wisdom of Clouds blog today and it struck me how fast we’re moving. Webside thinking on the cloud is ratcheting up quickly, hubs being linked by spokes, spokes to rims, rims to rubber meeting road.

I like the way James Urquhart ties his thinking about Microsoft’s Oslo domain specific language modeling technology story into a broader analysis of Microsoft’s Azure announcements from last week’s PDC.

Oslo goes beyond the Java/C# debates. It creates a platform in which complex systems can be described by a series of fairly simple models, assembled together as needed to meet the task at hand. Text tools for the low level stuff, visual tools for the high level assembly, etc. It really was an eye opening view of what Microsoft has been working on now for a few years.

Now take a look at Azure, and the “All-Enterprise” infrastructure that will be available there. Identity services (OpenID, no less), an Enterprise Service Bus, relational and unstructured database services–the list goes on. If you take the time to learn .NET, you can get an amazing experience where the development tools just flow into the deployment models, which just flow into the operational advantages, whether on-premises or in the cloud.

James worries that Amazon will crowd out (cloud out) other innovators and “mom and pop shops”- for example in the provisioning, monitoring and management spaces. Amazon as Walmart? Well, Amazon is already the premier retailer on the web isn’t it? You’d expect it to innovate and tightly manage the supply chain aspects of the cloud operation just as WalMart has in the world of goods warehouses, bags and bricks.

Clearly a lot is going to depend on standards for cloud computing: standards for on-ramps, standards for off-ramps, standards for virtualised data shipping containers. Walmart of course was able to assume a dominance because of supply chain standards rather than in spite of them. Standards and free markets lead to dominant suppliers, often two or three – its the rules of statistical physics at work. They can also create opportunities for entirely new players to emerge. Nick Thinks Tim is wrong. I think we’ll see what I call an hourglass economy, with huge players dominating the top, and small companies filling out the long tail. In the middle- consolidation. Hugh as so often lit a short fuse which fired a bunch of people’s synapses.

RedMonk’s very own Stephen O’Grady has been working the cloud a lot lately (well he is a developer advocate, after all) and has a nice Q&A to Microsoft’s launch last week – FORZA AZURE.

Q: For those readers that may not have read the news in detail, can you summarize the nature of the offering?
A: Certainly. Azure is a base platform with several available service components layered on top of it, that can be consumed in an ad hoc fashion. Specifically these service components are: Live Services (incl. “Mesh”), .NET Services (ACL, workflow, service bus, etc), SQL Services (a query engine), Office Sharepoint Services, and Microsoft Dynamic CRM Services. Last, there is a layer of directly available, discrete applications rather than building blocks (Sharepoint Online, Exchange Online, etc). These are built upon Azure, a unique and not generally available flavor of Windows that was designed for the custom datacenters that Microsoft has constructed.

The occasionally awkward naming aside, these services offer developers and ISVs the fundamental building blocks necessary for constructing applications.

Q: Ok, so there are a lot of cloud offerings: how does this compare to the current slate?
A: It is highly differentiated from Amazon’s EC2/S3 paradigm, which preserves in the cloud traditional notions of machine instances and so on. In terms of the development style it compels, it is most similar to Force.com or Google’s App Engine, both of which eschew the instance in favor of a fabric. Rather than plan for capacity using machine instance metrics, fabric cloud developers simply deploy to the platform, which – in return for the user surrendering substantial control – assumes the burden of scaling.

As was the case with Google App Engine, there will undoubtedly be much debate on whether this is a good or a bad thing for developers: while the removal of scaling concerns eases their responsibilities slightly, it also introduces a learned helplessness.

But irrespective of the debate, the model that Azure follows is one increasingly popular in the cloud.

Stephen points to the suboptimal tooling experiences of current cloud offerings.

Put bluntly, it could be said that today’s cloud platforms are succeeding in spite of their tooling, rather than because of them.

Against this backdrop, then, it’s nothing but logical that Microsoft would leverage its development strength and position – Visual Studio having set for years, in my opinion, the development tooling benchmark. Still, their execution in Azure – at least from the briefing and demo that I’ve received, is impressive

As I have previously argued – in business generally and technology specifically the best packager wins. Which brings us back to standards, de facto and de jure. RedMonk thrives on understanding patterns of technology standardisation and helping our clients to understand them. I don’t expect the cloud to be any different, and its becoming clearer who we’ll be learning most from. The social networks behind cloud computing are going to be an important element of the story ahead. Microsoft just threw down the gauntlet to be on of the two big ecosystems left standing as all this shakes out.

I didn’t want to blog too early about Azure because frankly I came away pretty impressed. Microsoft putting its considerable weight as an educator behind REST could be very very significant indeed for corporate developers, and is one reason I disagree with Stephen somewhat that “the platform [Azure] is a natural fit for Microsoft developers, and a somewhat unnatural one for everyone else.”

I see it as very likely we’ll see REST calls to Azure URIs from all manner of alternative environments. Note that Cardspace and Microsoft Live IDs will support OpenID for example. I don’t remember Dopplr moaning about the Live Identity API – on the contrary the implementation helped Matt overcome the Password Antipattern. You see Dopplr, its made of messages.

Microsoft look to be in pole position right now for enterprisey clouds, while Amazon rules the roost with web developers. Google and IBM – your move. Oracle says Cloud is Fashion and I expect Larry will be wearing the latest hemline in early 2009. SAP is in deep retrenchment mode, and as yet doesn’t have a volume, developer-led story for its own Business ByDesign platform. Zoho continues to make cloud-based apps that sing. I understand Dell is supplying Microsoft’s data center build out, which has to have ticked off HP, which has yet to to demonstrate thought leadership here.

But its Saturday. I should be spending time with my family.

To further the discussion why not come to CloudCamp in London on the 13th November.

disclosure: IBM, Microsoft, and SAP are all clients.