SPDY could gain acceptence very quickly - with some product innovation

Google have announced some early findings about their research into a faster protocol to reduce latency times due to good old fashioned HTTP. HTTP was designed as a really simple protocol to delivery (primarily) text content over the Internet and thus was born the Web.

One of the problems with HTTP is that it only really allows a single request to be serviced at any one time. The reason this doesn't APPEAR to be the case is because modern browsers create multiple connection threads that connect independently to the server and it gives the appearance of things downloading in parallel. It's a neat hack and works because we have good network speeds and mast processors to manage all this multi-tasking. Go back to a Pentium II with Netscape 2 and you'll watch the glacial procession of elements loading in from the top and goes down the page.

The Google project page talks a lot about why HTTP pipelining doesn't work and some of the technical architecture behind SPDY which I won't cover here other than to say that it's great we are seeing this type of innovation at the protocol level. What's most interesting for me however is how we get it in production.

There is a lot of nay-saying going on around this suggesting that because of the size of the Web you'll never get people to shift to a new protocol HTTP:// won, let's all leave it at that because there are too many web servers and web browsers to convert. This is what I want to address in this post.

Yes - there are fair too many legacy browsers to deal with to make this transition happen. Look how many IE 6 browsers are still in use, but we'd also have to shift all the Mozilla users, Chrome users (easy because of forced update) and Safari users as well. Not to mention all those pesky mobile devices that are springing up.

Dealing with the web servers is a much more straightforward issue. There really aren't that many in the scheme of things. Indeed much of our existing infrastructure runs multiple servers, Apache alongside a lightweight server like nginx and this is increasingly common.

As such there's nothing stopping me dropping in a SPDY server alongside my existing infrastructure for those users that can directly access it (Chrome 4, Firefox 5, Safari 6 and IE 10 for example).

But let's not stop there. A network admin could create a software appliance at the Firewall or Internet Gateway level for the corporate network that took HTTP requests, turns them into SPDY requests and then proxies these back. Now I have doubly fast Internet connectivity without upgrading my connection. For the price of a box that is well worth it.

For home users we could do the same thing. This protocol is software - it runs on TOP of TCP so because of that a Firmware upgrade of your average Netgear or Linksys home router could get you the same benefits as those above. ISPs could force this remotely on certain systems (Cable for example) or provide info on how to do it such as through a web, phone or personal service.

So for all the nay-sayers out there - this is a MASSIVE opportunity to speed up the web and people need to think outside the browser sometimes. QoS was delivered at the router level based on intelligent packet analysis - that speeds up network traffic massively but it's a software change not a hardware one.

I don't think it will be long until we see Netgear and Linsys start promoting this like they did with the WiFi standards and force adoption because it makes a great marketing case to do so.

I'll be trying this out at the rawest state to see if we can make it work and if I can, watch how fast our servers and network gateway get upgraded before I embark on upgrading client machines.

AdMob purchase by google paves way for interesting developer funding

It's just been announced that Google is set to buy AdMob for $750M in an all-stock deal. This is the third biggest purchase Google has ever made (the only two bigger are YouTube and DoubleClick).

AdMob started in 2006 so they have capitalised very well for a 3 year old business. Indeed they've been cash positive for a while now so this is a great acquisition by Google. The full gory details of the deal can be found here and a press site by google here

We know this is all aligned to Google's interest and in particular their big appetite presently for anything Mobile. However this also opens up some enormous opportunities for developers.

This acquisition brings with it some great opportunities for in-application display advertising that is delivered contextually but also based on Google AdWords auctioning technology. Along side this I can then use the same advertising account to drive ads on my mobile website that compliments my application and then use standard ads on my main website that provides additional information / community support etc.

All of a sudden a possible revenue opportunity opens up that was kind of there previously but wasn't very smart. Over the last 18 months in particular we've been watching the rise of free-ad-supported applications as well as paid-no-ad versions of the same application. I would expect to see a lot more of the ad-supported apps once this deal goes through.

The reason for this is twofold:

1. As a developer I can manage all of my advertising spaces with one vendor. I don't really want to have to deal with all these businesses I just want to get some beer money for my app that I'm spending my non-work hours producing.

2. With contextual ad serving, I can make certain elements of data within the application available and use that to generate calls to the Ad Server - much the same way AdWords works with a web page or in Gmail. This means the ads that are served will be more relevant to the content which should lead to higher Click Through which then leads to potentially more revenue for me (see note above about beer money)

This makes a lot of sense for an advertiser as well. Certain applications have huge amounts of uptake - twitterific on iPhone or Twidroid on Android for example. Imagine having contextual ads served based on the content of your twitter stream. Twitter might resist it but it could make some serious cash for the app developers.

Overall I think this will really blow the top of mobile advertising. Advertisers who have been a little shy in the mobile space will be comforted by the fact it's Google doing it. App and mobile site developers stand to gain some good funding from it and it be relevant for their audiences and as the world goes increasingly smartphone mobile mad over the next 18 months this will be worth serious $Billions in the next 5 years or so.

Cross posted to Citrus Agency Blog

Crown Oaks Day Racing Challenge

Last night I was writing code to play around with an idea I had rather than studying the Form Guide. See today I am off to the races (Crown Oaks day at Flemington) with some clients - hence why I should have been studying the form guide and not playing around with Erlang.

So I've decided to try an experiment:

Can the wisdom of the Twitter crowd outperform both blind luck and the bookies favourites with regards to return on bets during the racing.

Now we all know blind luck should lose. Betting on a winner at random is probably not going to get a single hit but against the favourite should prove interesting.

What interests me most is that betting on a race is actually contributing to an Information Market. Now theoretically information is held by all the various agents (people betting) and each one doesn't have a full picture but together the market becomes efficient and pushes down the odds of certain horses winning and then having long shots.

In smaller races where you have not so many people betting this works and either the favourite or a horse will relatively low odds will win. At large race meetings this doesn't work because a lot of people bet randomly (based on name, birthday number etc) and because of this it creates a lot of noise in the market so it breaks down.

So here's the challenge.

I'll start this from Race 3 at Crown Oaks Day today.

Each of the races is displayed below with a link to the field list. I'll be making a bet based on complete randomness (random number of the horse) and following the bookies' favourite. I'll then take the majority from messages to my twitter account (@ajfisher) for that race and place a bet following that. Simply send me a tweet "@ajfisher Race: NUMBER Horse: Name or Number"

Race 3
Race 4
race 5
Race 6
Race 7

Also you can use the tag #crowd-oaks if you want.

So can a smaller crowd provide more wisdom and outperform the bookies and complete blind luck on a big race day. Let's find out. It'll be fun either way.

The only reason why Linux isn't ready for prime time desktop

Okay, so this title's probably a bit misleading as there are probably a few reasons but as far as I'm concerned there's only one thing stopping my final transition to desktop Linux for complete every day usage.

Presenting

In my job I do a lot of presenting. I give major milestone presentations on projects, I present to the business on things that are going on, I present in pitches where we are attempting to win new business and recently I've started presenting at conferences.

I would not use my Linux desktop (and I have combinations of Ubuntu and KUbuntu 9.04, CentOS) to present with at all - even if someone paid me.

Before I say why I'll also lay out my Linux credentials. I use RHEL, Ubuntu and Centos EVERY day. All of my home computers are Linux based, I have a Linux PDA, I prefer my Ubuntu desktop for work and I administer numerous Linux (CentOS and RHEL) servers - via command line - all of the time. I've used it for over a decade and am more than happy with it and more than happy to hack on it to get stuff working.

However, there comes a point where I am not going to entrust a complete presentation that our business or my reputation relies upon to Linux's extremely flaky graphics system.

Yes, I know laptop Linux is problematic (but if the rest of the desktop is stable why not my second video out?)

Yes, I know that graphics card support (particularly from ATI) is very closed so there's lots of reverse engineering going on (but again if I can have one video out working why not two?).

I'm not sure why this is the case - I think it's a combination of X.org config and poor tools for configuring multiple screens with different resolutions but it definitely needs a lot of work to go ready for prime time.

I was at a conference this week and I had built my entire presentation in my Gnome desktop using FLOSS tools like Open Office Impress, had a great looking presentation and was legitimately keen on presenting using either my Ubuntu or Centos desktop. After hours of mucking around however I didn't feel supremely confident in just walking up to the podium, plugging in my laptop and "It Just Works"^TM. It's just too hit and miss.

I don't generally experience this with Linux in general and Ubuntu specifically although I am aware of other people saying it. For me 99% of the time it does actually just work.

So I defaulted back to my dual-boot Windows partition and presented from that instead. This was the partition that I had considered nuking because I hadn't used it in about 6 months. In this instance though I didn't have any other choice - and sure enough it did just plug in and go.

I still presented from Open Office Impress though (which is a fantastic bit of software I might add!) and I think I was the only one at WDS09 that presented with it (and I'm sure no one could tell I wasn't using PowerPoint or Keynote).

Desktop experience is exactly that - an experience and our experience, particularly when we are doing something social with a computer can affect our mental state quite substantially.

If I'd have taken the decision to present using Ubuntu I would have felt worried about whether my laptop would work and I would have been nervous and probably would have delivered a terrible presentation. In contrast because I knew I wasn't going to have any support issues I felt confident, in control and delivered what I hope was a good presentation to the audience.

Ubuntu are trying to address many of these issues with the Paper Cuts project but that's really aimed at business. Apple have addressed similar issues (hardware compatibility) by having a presenter's kit (which you buy) which provides all kinds of adapters to go from Mac to just about every video input type. Microsoft addressed this years ago from Windows 2000 with a great set of dual head tools that made it simple and a standardised way for vendors to incorporate them and it is extremely rare for it to fail.

Business use is one of the areas that Linux (and especially Ubuntu) has got a real opportunity to shif users across as there are so many other business benefits but users want a single consistent desktop so they aren't going to build on one desktop and present on another - it's too inconsistent.

For me this issue on presenting and graphics support isn't so much a paper cut as it is a gaping flesh wound and it really needs to be addressed.

Web Directions South 2009 - Cloud Sourcing the Business

I recently gave a presentation at Web Directions South which was fantastic (the conference I'm referring to here!). The session was on cloud computing and I hope everyone got something out of it. I've put the presentation itself over at slideshare

Cloud Sourcing the Business

View more presentations from Andrew Fisher.

My speaking notes are below as the presentation won't make much sense without it as it's mostly images or single statements.

Slide: Introduction

Cloud computing is a fast moving part of the IT landscape right now and, media hype aside, it does actually represent a fundamental shift in the way we produce and consume IT services.

[CLICK]

Slide: Cloud computing simplified

Partly because of where Cloud Computing resides on the hype curve but also because of it's pace of change, understanding the landscape can sometimes be confusing and it's probably best explained by this slide.

Today I'm going to talk about what cloud computing actually is in practice, how cloud sourcing can be used in your business as well as it's future direction and some of the traps you need to be aware of.

[CLICK]

Slide: What is cloud computing?

Cloud computing encompasses a huge array of services, technologies and techniques. Marketers would have you believe everything plugged into the Internet is a cloud. To understand the distinct areas of cloud computing let's try a little experiment.

Hands up if you are right now using some kind of Cloud Computing Service in your business?

Okay so if you are using some kind of infrastructure service such as S3, EC2, vCloud or RackCloud put your hand up.

If you're using something like Google App Engine or Force.com actively right now, stick your hands up.

Okay, so what about those people using Gmail, Google Docs, Office Live, Hotmail, Twitter or Facebook or something similar to that?

So nearly all of you are using cloud services right now without perhaps even realising it. So my work today is done as you're all experts...

As you can see, cloud computing covers a lot of different services and disciplines. Partly because of press coverage the things you expect to be Cloud Services such as infrastructure are there but so too are things you don't think of such as google maps or Gmail.

The networking company F5 commissioned some research not long ago with many CTO and CIOs of large US businesses on what they considered to be THE definition of cloud computing. They trialled 6 front runners for what cloud computing is.

None stood out from the rest as a sufficient or complete definition.

Googling Cloud computing will give you a heap of results and nothing really convincing either.

[CLICK]

Slide: Infrastructrure

Hardware guys generally believe if there's no infrastructure then it can't be a cloud.

Slide: Web 2.0

Software guys like me generally take the line that it's all about Web 2.0 style applications.

[CLICK]

Slide: NIST Def

The National Institute of Standards and Technology in the US have come up with this working definition. The full thing runs to a couple of pages, but here's the guts of it,.

Cloud computing is a pay-per-use model for enabling available, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and so it goes on...

Obviously this is a definition by committee in order to be able to get as wide a consensus as possible

NISTs definition is great for someone who is firmly entrenched in the cloud community whether as a vendor, seasoned consumer or consultant but is a bit impenetrable for everyone else coming in.

So for the purposes of today let's go with something a little more straightforward.

[CLICK]

Slide: Cloud Computing Definition

Cloud computing is: A Network based Service, that is available on demand such that the end consumer considers it not their problem

I think it sums up the spirit at least of what NIST is trying to say.

Let's dive into this a little.

[PAUSE]

It's a NETWORK BASED SERVICE. Cloud computing provides a service over a network of some kind.

[PAUSE]

Clouds are provided ON DEMAND. It is imperative that you can turn the service on and off as well as scale it up or down as needed. Cloud Computing is all about using a commodity so I should be able to stop using it if I need to or only use it part of the time.

[PAUSE]

Finally Cloud Services are Not My Problem. Or yours. Offering the service is going to be making someone lose sleep somewhere but it shifts the provisioning, management and maintenance issues away from the consumer towards the provider.

It's like flicking on a light switch. To make my lights go on I don't need to understand how the power gets across the cables or indeed connect the wires up myself each time I want to make it bright. It's not my problem – someone else looks after this for me. It's like Gas, Water or Electricity.

[PAUSE]

This definition is simple but allows us to explore what cloud look like and some of the ways they can work for you right now.

[CLICK]

Slide: Flowers?

Before we get into the actual types of clouds and what they can do specifically, it's worth touching on where these things live.

Clouds can be Public or Private. It doesn't matter whether you are available to everyone on the internet or just one little part of it. Service clouds just made available to your business are just as important as the massive public ones that are made available to everyone.

[CLICK]

Slide: Public clouds

Public clouds are those located out on the Internet and provided for a fee to parties that wish to consume them.

[CLICK]

Public Clouds are the ones that are being talked about most out in market such as those provided by Amazon, or Google.

I would use a public cloud because I either don't want or don't have the expertise to run one privately. Likewise as a business I can look at a public service and think that will do the job perfectly and I just pay to use it - why bother going and building my own?

[CLICK]

Slide: Private Clouds

Private clouds are typically owned or leased by a single organisation for their purposes only.

[CLICK]

Private clouds are experiencing very large amounts of growth as many very large organisations such as banks and governments start flexing up their IT to provide more services that can be combined and consumed on demand by internal divisions.

[CLICK]

Slide: Hybrid Clouds

Much like a hybrid car where you combine something like eco-friendliness with performance, it's also possible to have Hybrid Clouds

[CLICK]

As you'd expect Hybrid clouds are where businesses are using a combination of public and private components. This provides a huge amount of flexibility to the business as decisions can be taken around which services you are happy to have public and those you prefer to keep private.

[PAUSE]

So now we know where these clouds are located, it's time to look at what they can do.

[CLICK]

Slide: what can cloud computing do?

So now we know where these clouds are located it's time to look at what they can do.

[CLICK]

Slide: Service

Remember in our definition I talked about how Cloud Computing was some kind of service delivered over the network? Well unimaginatively every time you're looking at something cloud related you'll always see it called Something as a Service. The word Service in Cloud land is almost interchangeable with commodity and Cloud Computing is really all about taking some piece of technology and providing it in the most commoditised manner possible.

[CLICK]

Slide: Cloud Computing Landscape

The current Cloud Computing Landscape is made up of four major service areas- Infrastructure, Platforms, Software and Data.

Infrastructure and Software as Services are currently the most well known. These have the highest uptake from consumers at the moment – Amazon's Infrastructure Services alone generates over $250M in revenue a year. However there are more services you can use as seen by platforms and Data so let's take a tour around the Cloud Computing Stack.

[CLICK]

Slide: Infrastructure as a Service

This is most commonly used form of Cloud Computing currently. It's commodity hardware or infrastructure that is available for consumers of storage or processing.

[CLICK]

Slide: Server room

With infrastructure you can go and put your own servers in your own room and manage them yourself or else you can go and use some else's who have built it on a drastically different scale.

Usually these are paid for based on some kind of metering such as GB of storage or hours of processing time.

The consumer usually doesn't need to manage the the underlying infrastructure, they just put the pieces together and configure it as they like. Infrastructure Clouds have really taken off over the last 18 months or so because of the investment businesses such as Amazon, Go Grid and RackSpace are making in their User Interfaces making it quick and easy to provision storage or full servers or appliances.

Slide: Aws

As you can see, being able to manage your infrastructure services

[CLICK]

Slide: Go Grid

Is as simple as clicking a mouse.

[CLICK]

SLIDE: Using Infrastructure Services.

So how can I use this? With Infrastructure Clouds I can go and create everything from a single virtual server to host some site or project all the way through to creating a virtual data centre – in minutes.

Because I am a massive cloud geek I just decided one evening to create and run a Neural Network using 300 machines in Amazon's EC2 cloud – just because I could. It cost me about $50 and worked perfectly.

Many Infrastructure vendors also provide storage clouds where you can upload files and pay for the storage and traffic.

In practice, these services are great for trial projects or for when you need massive scalability very quickly.

SLIDE: Melbourne cup

Last year for the Melbourne Cup we deployed a Hybrid Cloud to help with web site demand during the Melbourne Cup carnival. Most of the year, the Victoria Racing Club's website is busy but manageable with pretty average infrastructure.

The Melbourne Cup is Australia's biggest sporting, cultural and social event with hundreds of millions of viewers world wide.

Over the Melbourne Cup Carnival, the site visits ramps up considerably with over a thousandfold increase in traffic, most of which is over only a couple of days and 90% is during about 6 hours on Melbourne Cup Day.

Using Amazon's S3 service as a Content Delivery Network, we hosted all our heavy assets such as images, video and flash files. Amazon's EC2 service was used in order to deploy servers on demand as they were needed for extra capacity.

Alongside this we then had a private cloud at Web Central that could be configured how we needed it based on the types of traffic we experienced.

The solution proved successful with nearly a million visitors on Cup Day and at peak over 200,000 files being transferred a minute.

On top of this though, it was also extremely cost efficient because VRC don't have servers sitting around doing nothing for 360 days a year as all of that extra capacity was switched on just prior to Melbourne Cup and switched off 2 days after.

Using Infrastructure Services in these circumstances helps you maintain a great user experience and also makes your costs much more manageable and reduces your provisioning time.

A lot businesses are using this right now to help with managing their heavy files. Look where Twitter hosts it's images...

[CLICK]

SLIDE: S3 Amazon Screen shot

That's right – Amazon S3

[PAUSE]

Given the sheer numbers of people visiting twitter now this makes complete sense for them to be able to manage the costs of image hosting and delivery.

At last count they had over 60 Billion items in there which is growing at a rate of about 10 Billion every few months so there's plenty of people using this and similar services out there. The infrastructure services are getting better all the time as competition is driving innovation.

[CLICK]

SLIDE: Platform as a Service

Platform clouds will be immensely popular over the next couple of years. They are 1 level up from Infrastructure services and these clouds take away the “machine” part of the cloud giving you a platform fully configured and ready to deploy your application.

[CLICK]

Slide: Diving platform

This means developers get the on-demand parts of Infrastructure clouds without having to concern themselves with managing a virtual data centre or configuring machines.

Certain assumptions are made at the platform level – for example Google App Engine restricts you to certain languages you can use. However in making these assumptions, it means developer has a defined jumping off point. Google App Engine and Force.com are seeing an explosion of developers who are embracing the platform and coming up with new ways to use it.

[CLICK]

SLIDE: Logo Slide.

At the moment platform clouds are in their infancy however they are growing rapidly. The two most well known and heavily used right now are Google App Engine and Force.com which is the platform underpinning Salesforce.com

The single biggest benefit of Platform Clouds is simply Speed to Market. The development process is drastically shortened because many behind the scenes decisions have been taken for you. As such the testing time is shortened because if you're using components of the platform they are already tested so you only need to test your code not the Platform's. Additionally you also don't need to go away and build an entire system stack – databases, servers, APIs etc – it's all there waiting for you to come along and do something with it. Ongoing it means a lot less maintenance as well.

Developers are looking at Platform Clouds in order to spend more time working on innovations and less time on replication. As a developer, why would I want to spend time writing yet another application that puts some customer information into a database off the back of a web form and then write some little app that allows the customer service team to make sense of it. It's this behaviour that Platform Clouds are addressing and the side effect is that not only do I get the benefits of having a complete system to work with that's developing new features all the time, I get to produce it quickly.

Nucleus Research looked at 17 projects done using Force.com earlier in the year and found that speed of delivery had up to a 10 times increase. The average across all projects was about a 5 times increase in speed.

Imagine what you could achieve right now if the projects your business is doing could be in market in one fifth of the time it's currently taking you? Just consider that for a second...

Developers are citing similar benefits of App Engine as well as some of the other platforms out there though little official research exists just yet.

As these platform clouds really mature over the next year or so the pressure will be on for businesses to start using them a lot more heavily as it will be uncompetitive to build equivalent systems from scratch.

[CLICK]

SLIDE: Software as a Service

Salesforce.com has been the Pin Up of SaaS for a very long time – in many ways they defined what it was and probably started a lot of the Cloud Computing discussion as they showed very early on it was possible to commoditise a service and deliver it over the Internet.

The reason Software clouds exist is because in the traditional Software industry there is a dirty little secret – maintenance costs more than innovation. Most Software Vendors have a small team coming up with new ideas and developing software then a massive one looking after the support and maintenance of these new ideas. Software Vendors have product lifecycles where they support certain versions of a product until they can justify not doing it any more and then hopefully force the customer to upgrade. During this lifecycle various customers could be on many different release versions and all of this adds to the cost of support.

The heart of Software as a Service is that there is only ever one version of a product, delivered over the Internet and all customers use the same version until the next upgrade. Immediately much of that time used for maintenance and support is available for sales, marketing and innovation.

Is it any wonder that Salesforce is now on it's way to dominating the global CRM space.

[CLICK]

Slide: SaaS Collage

Most software is now moving in this direction and is a big part of what enables Web 2.0 and is transforming the way developers deliver and maintain software for their user base. Look at software like Google Docs – who'd have thought even a few years back it was even remotely possible to use a spreadheet like Excel in a web browser.

Consumers of Software Clouds are generally pretty ignorant of all the underlying platform and infrastructure and are only interested in configuring the application itself.

[CLICK]

SLIDE: Gmail screenshot.

There are many applications of Software Clouds, far too many to go through today and my advice for any business procuring any new piece of software for the organisation is to make sure you seriously consider a Software Service in your selection process.

As an example let's look at Gmail. I'm guessing a lot a people here are using GMAIL for your personal email however Google has a fully supported Enterprise edition that works for your domain. You can still use outlook if you want or use the web client or mobile client if you prefer.

Email for a business is as important as cashflow however for many businesses the management and in particular cost of management of email is significant. On Premises email is something you have to maintain – backups, mailbox cleanouts, archiving, system upgrades etc are all the standard tasks done by an IT manager whether employed or contracted. Even with all this time spent, email still occasionally goes down.

Taylor Woodrow, a large construction business in the UK had much this scenario. They are a large business with about 1,800 employees and they were managing all their email in house using their IT team.

Last year Taylor Woodrow switched over to Enterprise Gmail, moving away from their on Premises corporate email they'd had for years. Whilst there are still costs with Gmail such as licenses to pay – Gmail costs $50 per user per year - and you still need someone to administer the accounts the cost savings for the business were massive. Over the year they saved about $2 Million compared to their previous mail software.

Given that large organisations have a scale of efficiency and they can get great cost savings per user, the benefits to smaller businesses are proportionately greater.

For an Australian business that has between 20 and 50 employees it's estimated that many businesses will save on average about $100,000 a year in total costs due to reduced licenses, man power, backups, hardware requirements etc.

That cost saving could be ploughed back into marketing, sales or other IT projects such as building your own Software Cloud.

Again, this is just one Software Service example but it shows how it's possible to start realising cost savings and get a better solution. Whilst I'm sure everyone here is broadly happy with their email system, are you now or will you ever be the best email providers? Probably not. Gmail can do it better and cheaper than you can. This is the crux of most Software Services – they can be provided, maintained and constantly upgraded to a much higher standard and for less cost than you can do it for and that's why we seeing them grow so rapidly.

[CLICK]

SLIDE: Data as a Service

Data as a Service is the newest part of the cloud computing landscape and I see this as having the most opportunity to shift how we create and consume data and is the one you can get involved in right now..

[CLICK]

SLIDE: NY Times Visualisation

Data is being provided everywhere and we are just now starting to see the interesting ways that people can use it. Data as a Service provides the consumer with the ability to retrieve and publish data – whether their own or that which is publicly provided - via the service.

The consumer gives no consideration to the underlying software, platform or infrastructure, only whether the data is available or now.

Data clouds are being used to expose data out to end consumers such as via Retail or Search APIs so those consumers can take them, remix them and provide them outwards again.

Data as a Service is built for mash ups – think about all of those sites with mashups of people's tweets plotted on google maps

[CLICK]

Slide: Zappos

or this great Zappo's site that shows what products people are looking at on a map showing where they are located.

[PAUSE]

SLIDE: DaaS Applications.

Data Clouds are so new that there aren't that many examples to show you yet though over the next 18 months we'll see an explosion in the data that's being made available.

Organisations that are starting to do this are people like the US Government with their Open Government or Government 2.0 initiative. Lower level governments are starting to make their data available such as the San Francisco city government who are making available data sets for things like crime data, highway information, and lists of public trees of all things.

Whilst these data clouds are very new we are seeing some interesting business models develop out of them. A US company called Jigsaw has recently become self funding. Jigsaw specialises in managing personal data by crowd sourcing contact information from participants and then they use that data to sell to marketers who want to cleanse their own subscription or marketing lists or who want to acquire new customers with very specific requirements.

Other businesses are looking at creating industry based data clouds to manage source information such as where livestock have been reared, transported and eventually slaughtered before making their way to market thus providing a traceable and authentic source for produce certification.

[CLICK]

Slide: Layar

The other area where Data Services are starting to appear is in the burgeoning field of Augmented Reality. Consumer ready AR is really being driven by and will continue to be driven by Data Clouds as huge data sets are made available to consumers via their mobile phones or PDAs.

Layar for Android and soon for iPhone is one such app that allows data sets to be viewed on your mobile device. Perhaps obviously at the moment this is skewed towards things like tweets, photos from Flickr or Wikipeadia entries that are near to you but this will change as organisations start to make their data available.

The examples on screen now show you where your local 7-elevens are and some points of interest in Sydney.

This space is very much in its infancy right now but it's definitely an area you can get involved in massively – if you're sitting on data make services available to get at it and watch the things people start doing that you never considered.

[CLICK]

SLIDE: Cloud stack display again

We've taken a tour around the cloud computing landscape and you can see there are a lot of areas that you can use. There are also some extremely niche areas I haven't talked about such as Desktop as a Service where your computing desktop is all cloud sourced however we'll leave them for today.

One really interesting thing that I've noticed about Cloud Computing is that as you go up the Cloud Stack consumers tend to USE more of those services at the bottom to PROVIDE more of the those services at the top. There's this knock on effect of one type of cloud computing providing the means for another type to occur and it's this effect that is driving much of the growth of the newer cloud types.

Now we've seen what's available let's take a look at the future to see where cloud computing is headed.

[CLICK]

SLIDE: Cloud Computing Future.

Cloud Computing is rapidly changing shape right now.

In 5 or 10 years the landscape will be as different again as it was 5 years ago.

Industries and consumers will be changed by how they interact with Cloud Services.

The fundamental paradigm shift that is occurring will allow IT to go back to being an enabler of business. Taken altogether, cloud computing becomes IT as a Service and businesses that embrace this philosophy will be able to do some great things.

Here's some examples.

[CLICK]

Slide: Manufacturing

Manufacturing will be changed by almost infinite processing power that can be provided on demand coupled with advanced computational techniques such as Genetic Programming to explore new product designs and get competitive advantage.

[CLICK]

Slide: Health Services

In health, for a long time governments in particular have been trying to make health related IT services better, providing a seamless service from one end of the health system to the other. Hybrid Cloud Services will be at the heart of our new health systems where any health professional will have proper access to medical records so they can treat patients much better.

[CLICK]

Slide: Start Ups

Start up businesses with great ideas will be able to be in market quickly, driving innovation and competing immediately with their bigger competitors, amplifying their disruptive potential within their chosen industries.

[CLICK]

Slide: Science & Research

All scientific centres will have access to supercomputer grade processing and storage which will drive new discoveries rather than just those with extreme budgets. Presently a lot of cutting edge science and research is done by centres that have almost unlimited funds. Now those with much more modest budgets will be able to use On Demand and Cost Effective cloud services to help their research.

[CLICK]

Slide: New media

All online sites and campaigns will be able to deal with large scale traffic and maintain high levels of availability, not just those with large budgets. Developers will be able to take advantage of this scalability and push the boundaries of the content they can deliver and the applications they can create.

[CLICK]

Slide: New Markets

New markets will open by providing guaranteed access to data for consumers such as to their music and videos no matter where they are in the world – this is at the heart of services such as Hulu.

Opportunities for businesses guaranteeing management and security of users personal data or industry data in so called “Data Banks” will start to present themselves. How much would you pay to have on demand access to all of your personal information all in one place that is searchable and secure?

[CLICK]

Slide: Geography

As a side effect of all these Cloud Services becoming available, businesses will no longer suffer from geographic location. Businesses in Australasia, South America and Africa will compete directly against Europe and North America. Competition will be based on knowledge and innovation not on available capital.

As you can see, all types of organisation stand to benefit from Cloud Sourcing.

[CLICK]

Slide: Warning sign

Whilst I've painted a picture of almost infinite computing resources being available, great bits of software being produced and a veritable mountain of data that's about to come online it's not all rosy.

With the pace of change of the technology outstripping everyone's ability to react quickly both within Government and Business, Cloud Computing is currently in a similar position to the Wild Wild Web of the early 90s.

Clarity is the most important requirement for any organisation producing or consuming cloud computing services.

Here's a short guide to some of the things your business needs to address before you start Cloud Sourcing.

[CLICK]

Slide: Disaster Recover and Risk Management

Disaster Recovery & Risk Management are no less important than they are in your business right now! Just because clouds are someone else's problem doesn't mean the risk management and recovery issues AREN'T your responsibility.

If you want 100% availability you have to work for it whether you are using Cloud Services or not. 100% availability is only possible by combining services whether at your DC or via the cloud.

As an example, GMAIL had a recent outage and caused a Chicken Little style panic that “The Cloud was falling”. Taken in context however, even with the number of people affected Gmail is still more reliable than corporate non-Cloud email which fails quite often.

Disaster and Risk management are no less important just because you are using Gmail or some other cloud email service. You need to have a policy to deal with outages just as you would do normally.

[CLICK]

Slide: SLAs

SLAs at the moment are the worst part of the Cloud Computing industry. Some vendors don't have any commitment, some do and there's a lot of variation between. Before committing, you need to assess the Service Level Agreement and decide if it's right for your business and right for the type service you're buying and what you're going to use it for. Some applications don't need highly available Service Levels but others do.

Slide: Data policies

Understanding the operational risk with regards to data is hugely important in a Cloud Computing environment as your data is probably going to be more public than you are used to.

Existing data policies may need to be updated due to the additional risk of being in a Cloud Environments. Many Vendors are actually better at managing this than their clients as they are doing it day in and day out.

Slide: Question mark

Before selecting a vendor it's important to ask the following questions and understand the responses and what they mean for your business

[CLICK]

1. How safe is your data?

Who has access to the physical machines?

Can the DC be cross contaminated? In large data centres it is possible for your data to be available even after you've switched off your service. Can this become available to someone else down the line?

Can you access your data to move it if you want to leave? Lock in and standards around this are a massive issue that is still not completely addressed

[CLICK]

2. How safe is your uptime?

What happens if a major hub is unavailable because of some Internet issue?

Who do you call when it all hits the fan at the Vendor end? How is their support provided? How do you support your consumers?

Who is monitoring your services?

[CLICK]

3. Are there any border concerns?

Laws are different between Australia and places such as the US and Europe.

Crossing international boundaries may bring with it additional legal requirements or may mean your services or data are subject to different laws than you are used to. Legal counsel here is imperative.

[CLICK]

Slide: Traffic light

Don't let me put you off. Forewarned is fore armed and none of the issues are insurmountable with the right policies put in place by your business.

If governments are using or considering Cloud Services then you are almost guaranteed that the issues with SLAs, Uptime and Data concerns are being addressed right now. Few businesses are as stringent in their requirements as multinationals and government agencies.

Also, bear in mind that many of these issues should be taken into account for any of your current IT services anyway so knowledge and policies you have in place now are still relevant for cloud service provision.

The important thing to remember is that you need to assess risk against the possible opportunity and make an informed business decision before embarking on any IT project.

[CLICK]

Slide: Seismic shift.

Cloud Computing is THE single most disruptive technology available to businesses right now.

We are in the middle of a fundamental change in the way IT works for the business.

This is being driven by the convergence of extremely cheap telecoms alongside sophisticated software and commodity hardware has led to an explosion of available computing power and storage. For the first time in our computing history we are approaching the point where actual deployable computing resources are going to outstrip our ability to use them. This creates a vast pool of cheap resource that will be used in innovative and disruptive ways.

If you are not using cloud computing actively within your organisation soon you WILL be left behind whilst your competition streaks away.

[CLICK]

Slide: Ticking Clock

Competitive market pressures are making this transformation occur so rapidly you have only a handful of years left to make this shift.

[CLICK]

Slide: Revolution

Cloud computing is as much of a revolution as the Internet was to business in the 90s but will have occurred in less than half the time.

Why? Because Cloud Computing enables not only the commoditisation of technology services but the democratisation of it as well.

In the same way that cheap PCs enabled an information explosion and reformed the way we communicate, Cloud Computing changes the way technology is used by everyone. Large or SMALL.

When the cloud computing revolution is complete there will be virtually NO distinction between the large and small business with regards to the scale and quality of the technology they can deploy to market.

[CLICK]

Slide: Zero

The cost to deploy these technologies will be almost ZERO

[CLICK]

Slide: Time to market

The time to be in market will be days and weeks not months or years.

[CLICK]

Slide: Infinite availability

The flexibility and scalability of the services available will be virtually INFINITE.

[CLICK]

Slide: What can I do now?

How can you start with Cloud Computing now?

If you're a business get your CRM onto something like Salesforce, get your email to Office Live or Gmail.

If you're a developer, get to grips with something like Google App Engine and deploy your next app there so you can build it faster.

If you're about to launch a campaign, use some infrastructure services to support it from the outset using something like S3, EC2

Are you sitting on a pile of data? Make it available through a data API. Chances are if it's about products you'll start selling more of them.

Here's some tips on how to sell cloud computing into your business.

Slide: Cost focussed business

If your business is cost focussed – and let's face it, most are at the moment – Cloud Services are Operating Expenditure rather than CAPEX. There's no depreciation, financing or generally up front costs. You pay for what you use and nothing more.

Cloud services are generally cheaper than maintaining Infrastructure, Platforms and Software of your own.

Providing Data as a Service might lead you to discover additional revenue streams or result in more sales of your core product.

[CLICK]

Slide: Process focussed business

If you are a process oriented business then you need to look at starting with with small, low risk trials and projects.

Look at primarily Software services for a starting point.

Make sure you do your homework around SLAs & legals. Most Cloud vendors realise there are issues and are attempting to fix them.

You can also consider setting up your own private cloud. VMWare, Melbourne IT, GoGrid and Rackspace all do this and I'm sure would be very happy to help out.

[CLICK]

Slide: Innovation focused businesses

If you are an innovation business the key thing to stress is that you can be in market faster by using Software Services or developing on Platform Clouds. Your focus needs to be on how you can innovate using these tools and how they stop you from having to replicate and go over ground that has been covered many times before.

For innovation businesses you can trial things out with much less commitment. If the results are poor, switch off the service and the costs stop. If you have a hit you can scale up quickly to accommodate your new market.

Innovation businesses will typically combine many kinds of services together in novel ways. Experience across all parts of the Cloud landscape will provide significant competitive advantage in market.

[CLICK]

Slide: Last Slide

Hopefully this presentation has helped provide some insight into what Cloud Computing is, what it can do and what you need to be aware of before jumping in. Cloud computing IS a fundamental shift in the way business works.

Cloud computing is not about a single technology or a single service. All of the elements come together differently for each and every business. Cloud Computing is primarily a shift in the way of thinking about IT and how IT can enable the business to radically transform the way it operates.

Those businesses who embrace Cloud Computing will have a significant business advantage, limited only by their ideas and their willingness to execute on them. How are you going to Cloud Source your business.

This article was cross posted to the Citrus Agency Blog

An unofficial endorsement of the Android platform?

As TechCrunch reported: Pigs Fly as Facebook and Google work together on an Android App - there's been a few indicators that this might be happening, particularly with some random mentions here and there on Twitter but no one was really expecting anything to occur given the competitiveness between the two businesses.

What's most interesting about this (particularly from my standpoint as an Android user) is that it will be the only other official mobile client besides iPhone; which really endorses the Android platform as the second runner to iPhone. And in acknowledging that, it also indicates that Facebook are considering that Android will have substantial traction in the coming year - not least when you consider there are two dozen Android based phones slated to hit the market in the rest of the year which could make a serious dent in iPhone's penetration.

iPhone launching with the Facebook client has been largely cited as one of the big levers in it's sales. Smart phones have been around for a decade and there have been sleek devices previously (Nokia 7710 for example was just one big touch screen 3 years before the iPhone launched) but the mind of the consumer wasn't fired by the opportunities it could provide to them. You only had to look at the marketing by the telco's around the iPhone launch to see that the Facebook client was the Killer App for the smart phone in terms of hooking people in. It gave them a very tangible benefit to owning what would have been the most expensive handset they'd have bought to date - "I can keep in touch with my friends besides calling them..."

With an official Facebook client for Android, the same endorsement has been conferred and one of the key marketing differentiators has been removed. I'm tipping late 2010 to be an interesting time as Apple and Google really go toe to toe and start slugging it out - which will be fantastic for innovation in this space.

Case Study: Django + Agile = Sportsgirl redevelopment

I've decided to write this one up because there isn't much on large scale and high speed Django development as yet and this is all still fresh in my head so it's worth getting down on paper (or screen as it were).

The agency I work for, Citrus, works with Sportsgirl, an iconic Australian Fashion Retailer and we were commissioned to help them build a community component to their site to help create a social shopping experience. The store was already there and was built as a bespoke Flash / .NET application and we had the opportunity to sit this on a different box in the data centre anyway. We thought this would be a fantastic opportunity to use Django and is exactly what it's designed for.

Architecturally we are using a LAMP stack using RHEL 5, Apache 2, mySQL (yes I know but it's to do with hosting) and obviously Django. Process wise we actually use an agency version of Agile that allows a collaborative effort between Designers, Application Developers and User Interface Developers.

Overall we were on a fixed deadline that meant the production phase was less than 8 weeks from sign off to go live including production of the site, interface and design then lock downs for content population and testing.

To make this work, everything was based around the platform - we chose as a base Django 1.0 and then layered into it a stripped down version of Pinax (we currently use v0.5.1 - the current official release, with updated apps) that has user profiles, avatar and gravatar functionality, photologue photo / image management, blog, pyBB forums, user voting and commenting.

With an established platform all three teams could start working concurrently much more effectively. This is one of the biggest benefits of Django and working to a framework and platforms like it because code can be prototyped so fast to a development build that everyone can see what they have to play with - Thanks to @jtauber and team at Pinax for that as well.

From there it was a case of lots of designing, interface creation, development and review to get it into it's final state ready for testing.

During this time we also worked on the flash home page produced by our flash master complete with nice collision detection, and full modularity so maintenance on this is all about creative not about development every time there's a refresh (very often on this brand). We'll cover this in more detail at some point.

The final phase saw deployment to the live environment which we did in Amazon EC2 for launch. We did this primarily for scalability reasons as the launch was going to be pretty large and promoted both on and off line.

As part of our final testing we also performed a lot of optimisation, this was based around optimising queries Django was making to the DB on both ends and we also then rolled out our delivery optimisations.

The first part of this was to implement memcached which is simply one of the best pieces of software presently available for data driven applications. On launch day we had a cache hit rate of over 80% which meant only 20% of all possible queries were going through to the database. With a couple of hundred thousand people visting the site during the launch phase this was instrumental in keeping particularly RAM usage low on the DB server as well as removing any bottlenecks to the Database due to latency.

We used nginx alongside apache to deliver all the static files on the site (not least because the imagery is so hi-res it was killing Apache to serve it!!). I'd often wondered how well this would work with a reasonably trafficked site but I wasn't disappointed. nginx dropped the load off the apache server which struggles for both CPU and memory (even with static files served outside of Django) from peaks on pre-live at 90% CPU and 70% available RAM + SWAP to 25% peaks on CPU and 30% RAM which is what Django was using to deliver pages with Apache's overhead.

The site went live on July 8, 2009 coinciding with a very large in store, off line and online campaign that drove quite a bit of traffic to the site. The server functioned exactly as required and with the optimisations peaked at only about 60% utilisation.

Overall this was a great project to work on not least because of the Agile process coupled with a technical foundation that allowed us to work even more collaboratively. 8 weeks for a major site launch is hard work for everyone at all levels no matter what their involvement. A great team helps with this but having the benefit of fantastic Open Source platforms to get our clients into market makes this even more achievable. Even less than 2 years ago I'm not sure I'd have attempted what the team achieved.