James Governor's Monkchips

Amazon CloudFront: Simple Caching and Naming

2008-11-28T11:49:53-06:00

Amazon CloudFront is a new distributed caching mechanism, designed to get data closer to the user. Amazon says:

It integrates with other Amazon Web Services to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no commitments.

Which is nice and all- but I need to be hearing it from developers. Which is why this caught my eye the other day…

geek footnote: the bigass images on dopplr’s new city pages are served from Amazon’s Cloudfront CDN. And it was really easy.

02:51 PM November 27, 2008 from twitterrific

Needless to say I twittered the comment, and discovered that others in the network are having similar experiences. See davejohnson. Of course these are just two data points, but the fact is my dial is turned very clearly to Matt Biddulph. He is someone I have the greatest respect for, and his technical chops are unimpeachable. I shared an office with Matt and learned years worth of web development technique in months.

Its also very easy using twitter search to come up with further agreement. see @jalegre:

Very happy about Amazon CloudFront, about 100-120 msec for static files like images, js, etc… in Spain (going to France). Faster than S3

Its not rocket science its findability. Meanwhile from a marketing perspective Amazon service naming is impeccable. The company hasn’t put a foot wrong since it started marketing its excess capacity to developers, and naming is very important in framing a market. On Twitter I said:

in other news - cloudfront- what a brilliant product name. descriptive, evocative, right on the money. others could learn from amazon naming

Share This

Is capacity planning dead or set for a revival?

2008-11-28T10:43:56-06:00

Capacity planning is something I broadly associate with the mainframe market. Its something data centers used to do before the advent of commodity computing - when chips became cheap and storage even cheaper. Web companies are notorious for their “throw servers at the problem, don’t worry its scale out” approaches. But virtualisation, and dare I say it, the Cloud, are bringing issues of strategic infrastructure planning back to the fore. I recently recommitted to blogging, and said I would try and bring some of my twitter conversation back to my blog, making it less ephemeral and perhaps even more timeless. The most simple method I can think of is to search and paste, like so. I got some really solid responses. The cool thing is now you can go talk to these people yourself.

jiludvik: @monkchips Capacity planning will be less of a worry for end users and more so for cloud providers. Capacity does cost money,even at scale

3 days ago · Reply · View Tweet

jesserobbins: @monkchips Capacity Planning = Tactical Advantage, Strategic Ability. John Allspaw’s explains why & *how* http://tr.im/1g79

3 days ago · Reply · View Tweet

jevdemon: @monkchips capacity planning will be dead when people stop building and deploying systems. In other words, never.

3 days ago · Reply · View Tweet

seabird20: @monkchips Capacity lanning is still a core discipline and will remain so. Everything that has cost must have appropriate planning

3 days ago · Reply · View Tweet

monkchips: @dannybradbury great point. thinking about anti-capacity planning. what’s the minimum resource to declare system will still run effectively

3 days ago · Reply · View Tweet · Show Conversation

rodet: @monkchips Capacity Planning is automating, at least on the mainframe: http://tinyurl.com/4zk8ov (expand)

3 days ago · Reply · View Tweet

ikarzali: @monkchips, that said, some rudimentary capacity planning will always be required (do I need 1 machine, 10, 100, 1000).

3 days ago · Reply · View Tweet · Show Conversation

arnehulstein: @monkchips capacity planning has never been gone in the industrial sector. They had to plan for shortages previously and will switch again

3 days ago · Reply · View Tweet

monkchips: is Capacity Planning dead or set for a revival? Your thoughts?

3 days ago · Reply · View Tweet

Share This

What Should Sun Do?

2008-11-26T13:19:12-06:00

Stephen wrote an awesome long form post on the subject, so I don’t need do. He was responding to Tim. Here is what I would do.

At best go private at worst go back to SUNW as a stock ticker.

Focus absolutely all of innovation efforts on a new core value proposition - The Storage Is The Computer. The Network used to be the computer, but now its the storage. Everything runs on everything. What matters is how fast you get to the bits and how flexibly you can work with them. That is why FOX and wikimedia have recently made major purchases of Sun gear. Data is like wine, Apps are like fish. I was very impressed to see this new Sun data erasure service - I can imagine some financial services companies are desperate to remove any data they aren’t legally obliged to hold at this point…

Put Marten Mickos in charge of the new app platform group, with Anil Gadre as consigliere. Sun acquired MySQL for a reason- and Mickos was a big part of it. Anil’s background for the last many years has been marketing. But APM needs to be about sales execution and hardcore operational rigour.

Go balls out for WebSphere, WebSphere and Oracle database offload. Budgets are getting slashed next year. Currently the enterprise really doesn’t have many options in that regard. Glassfish works. MySQL is a great bucket of bits. Partner closely with GreenPlum and Terracotta here. Does MySQL compete with Oracle as an OLTP engine? No. Are thousands of Oracle sticking data in Oracle as a matter of course when they could just be using a bucket of bits. Absolutely. And don’t even get me started on companies thinking the only possible app platform is a JEE server from IBM or Oracle. If its not EJB run it somewhere else. Use Tomcat or whatever- or maybe Glassfish

Don’t worry about pissing off Oracle- it leads with Linux and has done for years.

If you love something, set it free. I wll just quote Tim here:

As for Sun’s role as Steward of Java, and in particular the Java Community Process, let it go already. Java has mostly won and is mostly the establishment, and the community is smart and conservative enough to keep anyone from doing what Microsoft tried last millennium, or in any other way to subvert Java’s interoperability. In 2008, the JCP is costing Sun opportunities and friends and gaining us very little that I can see.

So I’d like Sun to set the JCP free, turn it over to the community, and when we develop some cool Java-based technology in-house, take it to market, try to make some money with it, and after it’s caught on and the bugs are shaken out, consider whether or not it ought to be taken to the JCP.

I disagree somewhat with Tim about the sales force. It should be selling more effectively than it is. Then again it could be the sales force is addicted to the margins of the old legacy products (the very problem which led StorageTek into Sun’s arms).

Get behind Sun Ray. Market the hell out of it. It works. Not as a Microsoft replacement, but as part of a secure enterprise infrastructure, delivering Microsoft Office as a managed service. The pull through there is with Identity, another Sun software bright spot.

Bill Joy was wrong. Innovation doesn’t happen elsewhere. It happens at Sun. The portfolio is in better shape than ever. The retooling of a tired mess into a clean and solid portfolio is complete. I was therefore surprised at this story by Dawn Kawamoto on CNET last week. Quoting an unnamed investment bank source that Sun’s “lack of product innovation is hurting them”. Then quoting an unnamed private equity player… Let me just say - it is really not such a journalistic challenge to get on the record negative comments about Sun right now.

Say what you like about Jonathan - the portfolio overhaul has been a success. There is the small matter of being able to collaborate effectively with Intel and Microsoft. But it all comes down to making money and selling stuff, not building slick products.

[update: I forgot to say. Get behind Hudson is a big way. Developers really like it. It just works. It makes managing your builds a relatively painless experience. It is even, dare I say it, "pretty" - at least that was the adjective Dan McWeeney from Adobe used for it when he demoed it to me]. Oh yeah - and the server tweets too, with a check box, which just has to be a good thing…]

disclosure: IBM is a client. Sun is a client. Terracotta is a client. GreenPlum has been a client and will be again. Jonathan Schwartz is a friend of mine.

Share This

Be Kind, Retweet… On the timeless way of building networks

2008-11-25T06:13:11-06:00

Jeremiah has a nice post about the retweet phenomenon in twitter (much like via: in delicious, another convention/function established by the community rather the platform provider).

As a result, the most powerful activity within Twitter is to watch the “Retweet” phenomenon. A retweet is when one individual copies a tweet from someone in their network and shares it with their network. It’s perhaps the highest degree of content approval, it means that the content was so valuable and important that they were willing to share it with their network –causing it to spread from one community to the next –retweets are the core essence of the viral aspect of content spreading.

I am not sure retweet is the most powerful activity on twitter, although Jeremiah and Shel both believe so, and would probably have argued the toss had I not seen a marked uptick in my followers this weekend. Tim O’Reilly has been retweeting me a lot lately, thanks to the Twhirl button. Tim says:

As one of the most prolific retweeters around, I want to give a shout out to twhirl for encouraging retweet behavior by giving a simple button to do it. I don’t think I would ever have adopted retweeting with such gusto without that easy feature.

This example illustrates just how important architectural choices are in creating viral applications.

What’s so wonderful about twitter is that its minimal interface has led to an explosion of user and third-party application innovation. It’s a bit like what Alexander Pope said about writing in rhymed couplets: the limits of the form made his creativity shoot out, as water from a fountain.

Ironically given this is really a post about link economies Jeremiah’s blog doesn’t appear to support permalinks for comments, which i must admit surprises me a bit. But back to twitter and retweets.

I twittered:

why don’t the oil companies bail the auto manufacturers out? they are a crucial distribution channel, and help keep the price of gas up

And Tim followed up.

Clever: @monkchips: “why don’t the oil companies bail the auto manufacturers out?”

It was later in the day when I searched Twitter for @monkchips replies and mentions that I saw writ large that the right tweet really could cascade through a network, potentially picking up new followers as it went. So that’s a page a half of people, engaged with my idea through Tim.

Retweeting is indeed a powerful mechanism, though it would be good to see the convention more standardised - the right tool probably needs to emerge to help formalise the idiom. Jeremiah indirectly points to a lack of standardisation like so:

In fact, in a very primitive way, you can see those that are repeating the content of others, for example Tim O’Reilly’s content. See this search query showing “Retweet @timorelly“, or “RT @timoreilly” (an abbreviated version).

Sometimes people use “retweeting @timoreilly. Another problem with retweets is that they are often much longer than the original (pesky long twitter names!), thus requiring edits.

I will continue to tweet, and retweet, and thanks to all those who retweet me.

For me the most interesting angle about retweeting is the point in the intro. Its really a language convention that the community created. Bottom up phenomenon are always interesting, and tend to happen in what I jokingly call Poor Internet Applications. Small Pieces, Loosely joined foster innovation and allow users to make paths and establish conventions rather than having them forced upon us. This is the Timeless Way of Building Social Networks. Sloppy is good. Don’t tell me its “just a feature” - so is a comma, or a vowel, but that doesn’t make them any less useful in the language.

The still above is of course from the awesome Be Kind Rewind, Michael Gondry’s meditation on the nature of creativity in the age of digital
reproduction.

Share This

Three Better Ways To Tell Its not Cloud Computing?

2008-11-25T04:52:39-06:00

First we had 15 Ways, then we had 16 Corrections, then 15 Ways I am Wrong, and now we have 3 Ways. What would Martin Luther have done?

Staffing Software Talk takes a very hard-boiled and boiled down approach:

For those of you not already plugged into this latest addition to tech jargon, you can read more about cloud computing here. But actually I wouldn’t waste your time. If you’re over 50 just think “service bureau”. 30 to 49, think “Application Service”, and 20 to 30 “SAAS (Software As A Service)”. If under 20 then cloud is everything you need in your online life - amazon, ebay, facebook, myspace, gmail.

Gregg certainly has a point that there is “nothing new” about the core concepts behind cloud, although I would argue its reimplementation of these ideas which gives them their current strength. I also think its notable that Gregg’s definition points to an age-related segmentation. I have to say that Gregg’s leap from there to 3 Ways To Tell Its Not A Cloud didn’t really chime with my understanding.

1. If it leaves you with unintegrated systems, it’s not cloud computing.
2. If it leaves you with non-revenue generating IT and data entry staff, it’s not cloud computing.
3. If it doesn’t eliminate your paper processes, it’s not cloud computing.

1. I disagree because of David Weinberger’s core Web design pattern of Small Pieces, Loosely Joined. Often integration is left to the user, which is no bad thing. Cloud componentry should be loosely-coupled, taking advantage of service orientation. Of course integration-as-packaging has value, but we must be careful not to simply recreate monoliths in the cloud.

2. Not all powerful and useful cloud services will be “revenue generating”. Take data archiving for regulatory purposes for example, which is a necessity but doesn’t drive revenue. Data Backup is another extremely useful cloud service. It may not drive revenues but its vital to a business.

3. I like the promise behind this definition, especially given the dog’s breakfast state of so many of our paper-based processes, but it ends up feeling more like Office 2.0 than cloud to me. Clearly IT should help to remove laborious paper-based processes, but arguably that’s true of any application, cloud-based or not.

Gregg, Jeff and Eric have all helped to advance my thinking on the cloud subject, as have Hugh, Nick, Tim, Tim and Stephen. Its time (if I can find some) for a significant update.

picture courtesy of Wikipedia via the US Library of Congress

Share This

IBM Executives for a Smart Planet

2008-11-21T11:00:58-06:00

I was going to write something about IBM’s Smart Planet positioning after a breakfast on the subject at the company’s Software Group summit this week, but it strikes me there is a more important point to make. Smart Planet is going to need Smart People, and IBM has plenty of those.

With respect to IBM I have to say it was GREAT to be there this week. Everywhere I looked there were very senior executives that deeply believe in the importance of global sustainability. This isn’t CSR, or even “shareholder value”. These are smart people, some with children, that care and want to make a real difference.

One dude is totally awesome - John Soyring - he was an environmentalist back in the first wave in the early 70s. In fact he decided not to retire from IBM because Green is too important, and he sees the opportunity/feels the responsibility to drive substantive change in that area. He is working on IBM go to markets in the Energy and Utilities market that will deliver efficiency, Demand Response and peak shaving.

I also spent some time talking to Doug Heintzman of Lotus, a really good guy that I have profiled before. He showed off some extremely cool new editor technology thinking (anyone for office suites with audio editing capabilities?) in his “day job”, but after the reception on Wednesday evening we sat down and talked about how, or should I say if, we’re going to make the disruptive leap from a carbon economy to something else. Is this really as good as it gets? Is it all downhill from here? Do we face managed decline or catastrophic change? Or can America pull off a renewal that will make the Apollo Project look like a cakewalk rather than a spacewalk? Doug is Canadian, but as he said, who else has the resources, especially in pure science and research, to really deliver us from carbon? America.

I have been hanging out with some web types thinking about driving hacker led sustainability. But we can’t succeed without a top down push from the Big Cos. As I said on an email to the group today:

“Quick update - it seems the top down folks are actually quite serious about smart grids. I know Google is working with GE. But I have also been impressed with what’s going on at IBM, SAP and Oracle around Advanced Meter Infrastructures. This is quite real, and the utilities don’t seem to be fighting it.”

If you’re interested in the future of Energy Networks and the Read/Write Grid Neutral electricity check out Greenmonk.net. Tom Raftery is doing a great job filling out the white space.

Job one in creating a Smart Planet is sustainability. We all need to work our asses off. It makes me feel a whole lot better knowing IBM is on the case, and I want to thank Sandy Carter and John Kennedy for “giving it a name”, something IBMers can rally around. The passion is there. This isn’t CSR. This isn’t just making a buck. Its way more important than that.

Share This

CloudCamp London 2: On Standards. Special Guest Post.

2008-11-21T10:06:32-06:00

Last Thursday I was supposed to be moderating a discussion about standards at the second Cloud Camp London. I had to pull out at the last minute, because of some prior family commitments I should never have considered breaking. To cut a long story short Benjamin Ellis stepped in and saved my bacon. Not only did he moderate the session with Matthias from Zimory, but then he summarised the event for me in the post below. I love the people of the web, who really live the Contribution Society. Benjamin is a top top bloke! He is also an awesome photographer!

BE:

Thursday night saw the second CloudCamp in London. It was a sell-out gig, with the Crypt packed to capacity. The fact around 250 people came out on a cold wet London night to discuss Cloud Computing is a testimony to the growing fascination with all things cloud. One way or another, it is a phenomenon that will disrupt the status quo.

The evening kicked of with a set of brutally speedy presentations. 5 minutes per presenter. Their velocity and compactness makes them hard to summarize, but let me try:

Simon Wardley, now with Canonical, was first. Always an interesting presenter (I might have been biased by his love of ducks and the fact I had his power supply). The fact that I could use his power supply on my laptop is a testament to the importance of portability and the power of re-use, which was the theme of Simon’s presentation. It continued his XaaS theme from the last CloudCamp, making the case for the importance for standards. Simon also argued for the importance of transparency in cloud services (cf. recent happenings in the financial world as the result of opacity). Who owns and operates the equipment isn’t always obvious in cloud services. An immaculately timed 4 minute 59 second presentation.

Joe Baguley, CTO Europe of Quest Software, was next up, asking if the cloud is really green. Here we are, putting 20kw of power demand into a rack, then trying to keep it at 22′c. Less than 0.3% of the electricity used makes it through to the processor. Rather than looking at how much processing each watt gives, Joe argued that the new question people will ask is: for each service is use, who much energy is consumed? Server power consumption doesn’t scale linearly with use. An idle server still consumes huge amounts of power. Even so, virtualization doesn’t give the power savings many expect. Even worse, while VDI (desktop virtualization) is trendy, it moves even more processing into the datacentre and uses lots of RAM, and therefore even more energy. Then you throw away the old desktops… definitely not very green. Joe’s advice? Go green: take a hybrid approach, re-use and rethink. Oh, and make sure you get the (power) bills. (4 minutes 50 seconds).

Paul Watson, of Newcastle University, has been doing some work on behalf of Arjuna. Throw away the cloud infrastructure, or get cloud from existing infrastructure? Paul suggested that you should create a private cloud, and spread the load. Put service agreements on sharing resource in place, and then share computing resources between different departments to make efficient use of (unused) computing power. Then link that to public cloud services to cope with peak demand. Paul’s model suggests the idea of many federated clouds (both public and private) - a kind of cloud of clouds. Interesting.

Duncan Johnston-Watt, Enigmatec CTO, performed a canned demo (using Elastic Fox), showing how the cloud might be used for disaster recovery. Apart from his very correct use of the work “momentary”, the demo was a little pedestrian and long.

Thankfully Cisco’s Phil Dean hopped on stage to tell us what CIOs want from the cloud. A reminder from Phil that CIOs are seeking to be business leaders. Accord to him CIOs like Cloud’s offer of consistency for all users, service simplicity (cost and 24×7 operation) and service orientation (for business agility). They don’t like the loss of control, risk management issues (business continuity and security) or migration and hybrid operation. I found myself thinking that perhaps CIOs don’t understand Cloud Computing yet, and are still wrapped around the axle trying to understand Web 2.0. Anyway, cloud needs a business focus says Phil.

Zimory’s Philipp Huber used his commanding stage presence to talk about what pervasive cloud computing might look like. He wheeled out comparisons with the energy market. Why does the energy market run so smoothly? Multi-tier supply - energy products, distributors, resellers - and 100+ years to get where they are. Both private and commercial demand, with well educated customers, and well established infrastructure standards (fuses, plugs… ). SLAs and quality standards, operating in an open market. How does cloud computing compare? New cloud providers emerging daily. Increasing demand, but driven by early adopters. The early majority still has major concerns about security and stability. Standards are emerging, but still not providing a seamless experience. Full interoperability is still “in the clouds” - fundamentally it works against the cloud producers, since it reduces stickiness. Billing models need to be transparent between clouds too, and easy to understand - again, something that might not be immediately attractive to providers.

Rhys Jones (from RBS) laid out an IT department perspective in “Clouds are cool (so why aren’t we using them yet)”. Power is nothing without control was his cautious note, before laying out some key benefits from his perspective: Switching from capex to opex is attractive. Cloud can do things that couldn’t be done before (due to resource constraints). Cloud gives ability to scale down (relevant currently). Someone else can do the optimisation, leaving you free for higher value activities. Rhys doesn’t see the cloud as just another form of box rental. Because of the cost structure, it enables rapid, low-risk trials - that opens up new possibilities for innovation. He noted that moving to the cloud is a cultural shift, and changing culture is hard. In corporate IT, demand outstrips supply, and the demand has always been upfront.

Wayne Horkan, Sun Microsystems CTO (UK and Ireland), talked about the global cloud infrastructure build out. A little unsurprisingly, he sees cloud computing as becoming dominant. He cited bandwidth figures from Amazon showing S3 using more bandwidth than Amazon web sales - I’m not sure that’s a proof point, but interesting none-the-less. Wayne said, “This is the Klondike gold rush…” - the building wave is starting to feel that way. Wayne drew out a layered model from network up to operations…. “What’s the next stack?” open source he says. Wayne ran out of time and was gracefully ushered from the stage by a gracious host.

Neil Hutson, senior director at Microsoft, outlined their vision. Drawing a linear platform evolution from mobile, to client, to server, to cloud, Neil outlined the changing economics of software towards a consumption based model. He pointed out that Microsoft already run their own data centres, and have done for a long while, supporting their Live offerings. He outlined the Microsoft Azure announcement, and the themes contained in it: standards and simplicity, and cited support for HTTP, REST, SOAP…. The Azure announcement has been covered just about everywhere, so nothing new to add. Only time will tell where it goes.

After a quick break, the gathered masses split into three groups. I facilitated the “standards and interoperability” open space with Matthias Kohl of Zimory. It was a highly interactive 45 minutes, touching on a broad range of issues. Although the discussion kicked off focusing on potential standards for system images, with a view to enabling portability, it was very clear that portability requires significantly more than that. The variations in storage architectures, database technology and support for network features all contribute to blocking migration between services. It is unlikely that a single body will be able to standardize all of these different aspects, and even if they could, standards bodies have traditionally struggled to keep up in rapidly developing spaces.

Just the ability to move the code doesn’t enable migration between different providers. Organizations are also reliant on operational procedures and interfaces. There are a huge number of dependancies. The group talked through some real world examples, during which Amazon got a good deal of air time. Do they have the momentum to become a de-facto standard, with a Amazon clones springing up around the place?

A standard image format might provide a base level of standardization, but there is a risk that the industry then gets caught up in a ‘lowest common denominator’ model that throttles much of the unique innovation that the scale and speed of cloud computing allows. There was a consensus for a pragmatic approach: a layering of APIs, standardizing a layer at a time.

Regardless of the different view, it is clear that without standards of one kind or another (de-facto or from a recognised body), there won’t be a market, and without a market, the cloud is unlikely to thrive. The competition isn’t as much between cloud providers, as it is between cloud providers and internal IT organizations. Cloud providers need to keep that firmly in mind.

The hardest thing about moderating was stopping the conversation when we ran out of time. Even warm pizza wasn’t enough to stop the conversations that went on late into the evening.

Share This

Everything Runs On Everything: End of the beginning

2008-11-20T09:34:13-06:00

One of the great advantages of the absurdly high powered machines we’re all running these days is that virtualisation is more than viable even for performance-hungry applications. Meanwhile Open source continues to crack open the hard nut called proprietary. Two recent pieces of news struck me and I thought worth capturing.

First off IBM acquired Transitive. This is major news. With Transitive’s technology POWER-based systems can run x86 applications. IBM is on a major tear around x86 consolidation- for better management, energy consumption and so on- and this acquisition should significantly accelerate its efforts to sell customers on the notion of data center consolidation.

Over lunch yesterday IBM Software Group general manager Steve Mills said something extremely smart, and absolutely relevant to the deal.

“Tech used to be about displacing labour. Today tech is about displacing tech.”

Of course consolidating servers also potentially allows you to cut staffing levels… which in the current financial environment is a more pressing concern than ever. We used to say Do More With Less. Now we’re going to live it. What’s the end game- Windows servers running on IBM mainframes is one possibility, though I can imagine negotiations between Microsoft and IBM about production support for customers are likely to be rather interesting.

Byron Dennis sees things rather differently, talking of “choking off user choice”.

The other bit of progress that struck me pretty squarely lately as part of the same broad trend, but here driven by open source rather than virtualisation is the Moonlight project, spanning Microsoft Silverlight 2 and Linux with Novell as the intermediary. Here’s Miguel. This is not a totally isolated effort- Microsoft is also working with a French software company to see Silverlight supported through Eclipse. The point here is that there is now developer tool and runtime support for Silverlight in open source environments. Also bear in mind that “the full Silverlight Toolkit is available under the Microsoft Public License, an OSI-Approved license that allows full reuse of the code.”

We’re increasingly living in a world where people say things like:

“The above program was built on MacOS, the result copied to Linux and then executed using the LinuxPlayer. This is still very basic, the port is yet far from done.”

[italics mine]

What effect will Open Silverlight have? Likely more open Flash. As Chris Messina aka @factoryjoe put it yesterday on twitter:

“It would not surprise me one bit if Adobe open sourced Flash to co-opt Silverlight. And to make Flash content web-lucrative with Google.”

Bear in mind that Netflix is now supporting video on demand for Macs using Silverlight, so this is not all BS. The pressure on Adobe is real.

A big question after Microsoft’s recent PDC was whether you could deploy Linux to the Azure cloud. Given the powerful virtualisation technology in Windows Server that’s a business, rather than a technical decision for Microsoft.

Am I saying that proprietary technology is dead? Certainly not - many of the smartest developers on the planet are currently running headlong into the embrace of Apple environment optimisation, for example. But open source and virtualisation are dramatically changing the economics of enterprise computing. Write Once Run Anywhere- we didn’t get there with Java, but old mainframe technology is now changing the game.

Share This

15 Ways I Am Wrong About Enterprise Cloud Computing

2008-11-12T09:50:24-06:00

My post 15 Ways to Tell Its Not Cloud Computing has had a lot of play - 24 trackbacks and counting…

A lot of the commentary so far has generally approved of the angle I took, though clearly there are strong dissenting voices. Before I proceed to one of them I just want to point out that 15 Ways was always intended to take a strong position, though hopefully with some humour. That is, the post was somewhat rhetorical. Of course not all enterprises are going to suddenly drop enterprise IT and do everything over at Amazon Web Services. That would be a. absurd and b. impossible.

But on the flipside I see a lot of over-complication from major players - around something that originally simply spoke to provisioning architecture for developers without the headaches. Cloud started as a description of a web phenomenon, not an enterprise, phenomenon. Can enterprises learn from Web simplicity? Of course they can, and should. Can web companies learn from enterprise middleware scalability patterns? Yup.

However- whatever else the cloud is, it should not just be an excuse to try and rewarm technology that has already failed in the marketplace. It should not be an excuse to sell a new complexity.

And so to pushback. I am really happy that I got such a considered and thoughtful response from Eric Novikoff of ENKI Consulting, who does some work with 3Tera. You might want to read my original piece first, but I will leave everything inline, my original 15 points in bold.

I thoroughly disagree with this “backwards” definition of cloud computing. James Staten of Forrester defines Cloud Computing as “A pool of highly scalable, abstracted infrastructure, capable of hosting end-customer applications, that is billed by consumption.” This is a definition that is amenable and extensible to the enterprise. This list of “nots” is not.

> If you peel back the label and its says “Grid” or “OGSA” underneath… its not a cloud.

Cloud computing as defined by Staten can be delivered from a variety of architectures, including Grids or SalesForce’s Big Iron. That’s the point of the cloud: abstracted infrastructure.

> If you need to send a 40 page requirements document to the vendor then… it is not cloud.

More and more cloud vendors are offering solutions rather than cpu cycles. CPU cycles are great for programmers, but businesses want to solve business problems. Without a definition of what the customer’s problem is, and an honest and transparent reply from the cloud vendor, you are running on hopes and dreams instead of a known value delivery system. The result is that the companies most successful with Amazon have dedicated staff and management to ensure a successful cloud deployment and to make sure Amazon doesn’t change something underneath that breaks their app. Where is the savings in that?

> If you can’t buy it on your personal credit card… it is not a cloud

Businesses don’t like to pay for expenses on personal (or business) credit cards. In fact, they prefer a budgetable monthly spend which of course contradicts with consumption billing. Our customers *don’t want* to pay by credit card, by and large. Only the web2.0-in-a-garage startups are interested in credit card payment: possibly because they’re financing their business that way.

> If they are trying to sell you hardware… its not a cloud.

This seems like it should be true. However, corporate customers are reticent to send internal data out into a public cloud. Why wouldn’t they buy hardware from a cloud vendor to get similar advantages, delivered internally?

> If there is no API… its not a cloud.

You won’t get much disagreement from me on that, but most of our customers aren’t interested in an API, so they probably would disagree. They just want to deploy and go, not write their own cloud operating system.

> If you need to rearchitect your systems for it… Its not a cloud.

That would be nice, but I haven’t seen one cloud that doesn’t require some rearchitecting. Wholesale rearchitecting is often required to get around Amazon’s peccadilloes, and even the AppLogic system we deploy on, despite it’s virtual datacenter analogy, still has some characteristics that require minor architecture changes for some applications. Whether or not you have to rearchitect depends more on how much you encoded your expectations of the hardware environment into your code than it does the particular cloud you deploy to.

> If it takes more than ten minutes to provision… its not a cloud. If you can’t deprovision in less than ten minutes… its not a cloud.

Where are you measuring the time period from? We can bring up a virtual private data center in the cloud in 2 minutes. But our customers often take days, weeks, or months to figure out how they want to provision from the moment they decide to. For businesses, the time from decision to deployment is a better measure than time to provision.

> If you know where the machines are… its not a cloud.

This simply points up disagreements about the definition. Most customers who demand high performance (say, running Oracle) and certification (say, HIPAA) both want and need to know where the machines are. They are unlikely to deploy to clouds where the architecture is so opaque that they can’t meet their requirements.

> If there is a consultant in the room… its not a cloud.
This is false on two counts. First, those companies who have successfully deployed their entire operations to the Amazon cloud have dedicated staff to manage Amazon deployment. Call them “internal consultants” if you will - people whose job is to manage/consult on the cloud. Second, a large number of reseller/consultants have sprung up to facilitate the use of Amazon. Amazon is becoming something like a physical server: you need someone to run it for you. These consulting/reselling companies do that. They may not be “in the room” however, which is one of the benefits of Cloud Computing: it creates an internet-enabled ecosystem of knowledge workers you can use with your application without providing a cube for them.

> If you need to specify the number of machines you want upfront… its not a cloud.

This is an assumption that Amazon has catered to. If you do the math, you’ll see that Amazon bought it’s cloud business by selling under its costs. Especially in the early days, keeping huge reserve capacity up and running was a cost that was not passed on to the users. Many of today’s cloud providers manage their inventory of physical hardware to reduce their costs, by offering discounts on usage to customers based on contracts. This will (and does) lower the prices for customers over the “big daddy in the sky with infinite computing power” cloud model. You’re going to see more of it. If cloud computing is a commodity with perfect competition, like air travel, you’re going to see vendors deploying intense strategies to manage their unused capacity.

> If you need to install software to use it… its not a cloud.

Cloud computing and open source seem to go hand in hand. My experience is that each of my customers wants a different version of their operating system, database, etc. Cloud vendors typically provide a library of code images, but they cannot custom-configure to meet an individuals’ needs. The trick is not avoiding installation, but making it easy.

> If you own all the hardware… its not a cloud.

There definitely are cloud providers out there who have their customers lease the underlying hardware, which defeats the concept of usage-based billing. However, your statement would get puzzled looks from many large enterprises who run their own clouds.

Its funny I get such strong reactions from consultants like Jeff and Eric. I just can’t work out why that is

So what about these “puzzled looks” from folks that Eric says have built their own behind the firewall cloud. An enterprise guy is welcome to call the infrastructure they’ve been building out a cloud. Everything is a cloud these days, right (one of the reasons I wrote 15 ways of course)? An end to end VMWare shop is going to be in pretty good shape for rapid provisioning, for example - which is why VMware is certainly in the running as a CloudOS player.

When I say that IBM lacks a cloud strategy what I mean is that its cloud strategy is currently too diffuse, too complex, and not closely enough managed. Where are the simple on ramps? We currently have such a bad case of definition sprawl that some segmentation is required.

I like Tim O’Reilly’s recent three leg definition - Utility computing, Platform as a Service and Cloud-based end-user applications. But we need more segmentation on the Utility side, to take account of “enterprisey” concerns and mores. To get a little bit goldilocks about all of this the 451 has done an awesome job of cloud segmentation with a report I… can’t find on the internet, mind the firewall guys -but its actually too granular as a I remember it.

Its not my intention to put forward my own segmentation at this point but I do want to point to a recurring theme from Eric, about my missing the point - we could call this the Hidden Costs of Amazon Web Services argument. Or AWS TCO Smackdown or something. I will refer at this point to another post by Novikoff, where he talks about his experiences as an SDForum Cloud Event.

Each of the companies using EC2 had developed intense internal expertise on Amazon’s service. That’s right, you read correctly: on Amazon’s service. Not on Cloud Computing in general (though there are those who argue that EC2 and Cloud Computing are the same thing) but rather on the intricate technical and organizational details of dealing with Amazon and its service as well as optimizing their use of it.

It wasn’t a surprise to me that these companies, dependent on Amazon for their success, would get to know the service well. What did surprise me was the lengths to which they went to mitigate the risks. Generally, if you have doubts or fears about something, you want to learn all you can about it. These companies showed incredible technical depth at understanding EC2’s characteristics and peccadillos. They asserted that it was critical to their success. But this seems to me to be in direct conflict with the ideal of Cloud Computing, which is the democratization of information technology; in other words, lowering the barriers of entry. I think this is an admission that Cloud Computing technology in any form is still not polished enough for completely casual use. At ENKI, we realized this early on and decided to strive to be the “onramp to the Cloud” by providing all the services necessary to use our cloud without having to know anything about Cloud Computing.

At Redmonk we tend to think low barriers to entry don’t require consultants to navigate - but that’s just us. As is clearly the case right now the turbulence surrounding the cloud conversation is a barrier to entry, which is where people like Eric and RedMonk come in.

The truth is I am probably half right, and half wrong. I leave the reader to decide to turn the dial to parse Eric’s comments and my own. What do you think?

Share This

IBM Joins Obama’s Coalition for a Smart Planet: Change!

2008-11-06T18:10:15-06:00

A few days ago Sandy Carter, IBM Software Group’s hard-charging, always on, marketing supremo and good friend pinged me to talk about the company’s latest Big Idea - the Smart Planet. Well CEO Sam Palmisano kicked it off this morning, with a speech at the Council of Foreign Relations.

Andy Piper explains the core idea really well here:

Sam talks about how the world has become instrumented, more interconnected, and devices more intelligent. And he talks about how the current world crises – ecological, financial, and others – represent an opportunity for change. The next step for the globally integrated economy is a globally integrated and intelligent economy and society.

You should read Luis too, with his roundup of links to IBMers blogging on the initiative. Cool. Sometimes IBM’s obsessive desire to work on the toughest problems seems wrong-headed, and leads to complexity. But this time around I feel things could be different. Why? Because IBM is choosing to work on exactly the right problems- that problems that Obama has also pledged to make his own. The kind of problems that Greenmonk is trying to help tackle. The kind of problems that involve eating your broccoli (thanks again vendorprisey).

That’s right - i am not sure IBM realised it when it set out on this Smart Planet mission but it has clearly captured the Zeitgeist better than any other company in the world right now. Its surprising that Steve Lohr’s New York Times piece didn’t make the Obama connection (at least not explicitly).

I.B.M.’s chief executive, Samuel J. Palmisano, is proposing a technology-fueled economic recovery plan that calls for public and private investment in more efficient systems for utility grids, traffic management, food distribution, water conservation and health care.

So let’s look at Obama’s agenda on the AWESOME change.gov site.

Revitalizing the Economy

Ending the War in Iraq

Providing Health Care for All

Protecting America

Renewing American Global Leadership

While IBM isn’t talking about American hegemony, or the Iraq situation - its clearly focusing on the other three areas. The language might as well come from Obama’s speeches, right down to the call for change.

People want change

There are two things recent events have shown us these past weeks: how small and interconnected our world is. And that change is unavoidable.

Once our economies have stabilized, we will need to heed the call for change and address the systemic inefficiencies in our world’s infrastructures—the processes that underpin the way billions of people work and live—and make our planet smarter.

Consider:

How ineffective our power grids are: 67% of energy is lost due to inefficiencies.

How sick our healthcare system is: costs have pushed nearly 100 million people below the poverty line.

How dirty our water is: one in five people do not have access to drinkable water.

How wasteful our food chains are: U.S. consumers throw away $48 billion worth of food a year.

Healthcare, Energy, Trying to Revitalise the economy. Surely its no accident that Palmisano evoked Franklin Delano Roosevelt, as does Obama.

In an interview, Mr. Palmisano compared today’s economic challenge, in broad strokes, with that faced by the United States as it struggled to emerge from the Depression or after World War II. In the 1930s, he said, the New Deal programs, among other things, brought electrical service to much of country — not only to rural homes, but also to factories, which no longer needed to build their own power plants, as many had previously.

After World War II, Mr. Palmisano said, the government’s construction of a national highway system helped create larger markets for goods.

“We’re at a similar stage now in that these are difficult economic times,” he said. “The right way through it is not to hunker down, but to step up and invest and improve our competitiveness.”

We need to invest to save the planet. Its a time for Heroes (more Zeitgeist surfing).

When Greenmonk first started talking to Smart Grids we looked like outliers. Now our ideas are mainstream. Google and IBM are now strongly behind the idea.

Obama should give Mr Palmisano a call. We just voted for change. We are hopefully leaving some of the idiocy behind us. We voted for a smart planet. IBM is going to help deliver it.

One of the points I make in my sustainability stump pitch is that, paraphrasing Michele Obama,

“For the first time in my adult life I am truly proud to be in the IT industry.”

IBM likes to avoid talking politics, but the subtext is clear. The company is ready to help Obama with a massive program of renewal. The spirit of FDR is with us. The challenges have never been greater.

Go Obama. Go IBM.

disclosure: I voted for change. IBM is a client. Sandy is a friend.

my flight was just called. See you on the flipside.

Share This