Technology Treason

2010: The year the Netbook turns into the Web-book

2010-01-04T10:37:00.005+11:00

2010 is set to be a bumper year for Consumer Electronics. With people spending less outside the home they are focusing more inside and as just about everyone now has some monstrous TV it's the little things that count.

2009 was arguably the year of the Netbook. After Asus launched with their eeePC in 2008 a land-rush occurred last year with virtually every Notebook manufacturer providing an offering. HP's Mini and Acer's Aspire One ranges both did very well as did Asus with their eeePC.

At the end of 2009 however it became virtually impossible to get a Netbook that is truly still a Netbook. Acer, Dell, Asus and HP all fell back into their same tired old routine - bigger, faster, more capacity! And the Netbook experience suffered.

I've made this point before but the power of the Netbook is in the Network - not in how big a hard drive it has. Why do I need a 160GB hard drive when I have Terabytes of NAS and Gigabytes of cloud storage? I don't need 5 USB ports and I certainly don't need Windows.

Having said all of this the Netbook category has gone ballistic - having doubled from 16 to 33 Million units sold in 2009 - sales are worth about $USD11Bn globally (Display Search research for 2009)

My money is on the next generation - so called Web Books, Slates or Tablets. These devices are being actively invested in by a number of investors and represent a merging of several types of computing behaviour.

Architecturally, most are small form factor (10 inch or less), are either a tablet or have a folding range far outside the normal Notebook range (can be flipped over on itself entirely - so is just a screen), they are generally touch screen capable with many being multi-touch and the big one - most are not running windows (generally running flavours of Linux).

Behaviourally, the Web Book is designed to be a piece of Consumer Electronics. It's not a desktop replacement, it's not an office machine. It's a device that is a general purpose computer but built to use in the home as such it plays on the following:

It's relatively small and definitely light.
The processor is powerful but not an energy guzzler (Intel Atom's do brilliantly here as do ARMs)
The display is gorgeous and has high viewing angles so multiple people can see it
It uses wifi and may not even have Ethernet connectivity
Solid State disks are a must but are low capacity (you don't need more than 16GB in a machine that is connected to a network) thus saving on energy
battery life is a must - the longer the better thus every component is energy efficient
Ideally the screen is touch capable and ideally multi-touch (thus eliminating the need for a keyboard)

The device is permanently connected to the network and thus the Internet. It's there to connect with people, view photos, play your tunes, watch movies and read web pages. It's not there to write documents, do full scale design or programming (though people will use it to do this in a limited, fast fashion).

I've been excited about Internet tables, Slates, Web Books - call them as you will since Nokia released the N710 and with Apple, HTC (Google), Litl and others all about to play in this space in a big way over the next few months, there will be a lot of people asking for a Web Book or Internet Slate in their Christmas Stocking next year.

Expect to see masses of innovation in this space as companies that have not been too caught up in the Netbook scene enter the fray for the first time and start showing off some new ideas. Litl does this with their awesome Easel Frame style web book and both HTC and Apple will doo some great stuff on the user experience end of things too.

2010 will definitely be the year that the Internet goes increasingly mobile both inside and outside the house but the experience of it literally becomes more tactile and less bound to the keyboard.

Why I'm interested in AWS Spot Prices for EC2

2009-12-22T17:46:00.002+11:00

There's been a lot of chatter going on around the intertubes over the last couple of weeks since Amazon Web Services released their Spot Instances pricing model for EC2.

In a nutshell - AWS have created a compute market. Instead of charging the same price to every person for the same product they have basically created a market where people can buy compute time at less than a price they are willing to spend based on the current demand.

There's been some conversation about the fact everyone should just put the current demand price in as their maximum and this would game the system (the comments here for example) however this misses the point slightly. The Clouderati often talk about Utility Computing or Commodification as one aspect of Cloud Computing and what AWS have done is the logical conclusion of that - they have created a true market for the provision of computing time based on supply and demand.

Now what's interesting with the ideas some commentators have come up with regarding gaming is it assumes everyone's working the interest of everyone else. That isn't the case. Yes I know I could get some resource cheaply if I keep my bid low and am willing to wait for a period of time however I have clients and they have deadlines. That big compute job crunching all the marketing data needs sorting out this afternoon - so I'm going to put a high bid in for 50 nodes NOW! The market will accommodate that and those with low bids will be knocked off. Thus the market constantly corrects to the requirements of demand.

But it's the flipside of this which makes me really interested in EC2 Spot Instances. I can have a battery of servers doing work at little to no cost if I build my system correctly.

The critical element to this is I need to address availability correctly - that is I need to ensure that my entire system doesn't go down because I've been priced out of the market.

This is a really rough idea at this point but I'd love feedback around it - it's obviously based around some kind of online application that requires multiple nodes.

I have an instance which is the master. All parts of the stack could be retreated back to this server if it's needed.
I have Cloudwatch or some other monitoring system assessing the performance of my nodes so I can see when I have spare capacity or when I'm under utilised.
The master server has a series of heuristics looking at the current work loads and the current costs that each server is incurring versus the work it is carrying out. Thus low utilisation and low cost is okay but low utilisation and high cost would cause alarms to go off.
The heuristic set up makes reference to the demand pricing level and strives to always keep each instance below that price.
As the spot prices go up and over the demand price I immediately terminate expensive spot instances and start replacing them with lower price demand ones. As the price comes back under then I can replace demand prices with spot prices
The master server then creates instances as required to fulfill the work units that are required and link them into the system.
Each node is able to be switched off mid unit so the entire network is self-healing

So the only thing that would be required to get this up and running now is having a reliable system for creating nodes and getting them working into the network as quickly as possible and producing the heuristic system to monitor and create and destroy instances based on some rules that would create some intelligence around pricing.

Not least the system would need to determine whether a mix of different types of instances would be appropriate if there are large distinctions between their current spot price for given work units. For example:

If we were serving a bunch of web pages using some heavy duty memcached system then RAM is the most important commodity. Say I have an instance of 1.7GB RAM at 3C/hr and another instance of 7.5GB RAM at 15C/hour then my intelligence system needs to understand the component (Memcached) just needs buckets of RAM and that getting 5 instances at 3C/hour is better value than 1 at 15.

Importantly it can then ramp up towards that number based on what is actually required rather than doing the whole lot and then under-utilising.

So I think we're quite a way away from this type of system but my opinion is that this isn't out of the realms of possibility and importantly the market Amazon has created has allowed (I could almost say "is going to force") these types of architectural considerations to start being made.

Interestingly all of a sudden decisions I am going to make around infrastructure is going to be much more value based. It's not about ROI - it's about value and am I getting the best value from my infrastructure. IT teams that get this are going to make an absolute killing with the type of services they can offer and the prices they'll be able to do it for.

Am I off my rocker? I'd love to explore this idea further.

Prediction: 2010 will be the year Apple and Google have a cage fight

2009-12-21T10:46:00.005+11:00

The pre- match slanging is pretty much over and the location of the fight has been chosen. 2010 is going to be the year Apple and Google finally stop dancing around and actually get in the ring. Unlike a nice clean refereed boxing match (Apple V Microsoft) this is going to be a dirty underground cage fight complete with barbed-wire wrapped gloves - expect to see a lot of blood on the floor - and fanbois rucking in the concourses.

The ground is, of course, Mobile and the massive dominance both organisations have taken in this space over the last 12 months. Mobile is still a fast growing area of communications but smartphones is where it's at. There's no question Apple ignited the world's imagination of what is possible in the mobile space and capitalising on the fact that the fashionability of a phone is important in a way that RIM and Microsoft just didn't get.

Google have taken that to a whole different level with Android which just "gets" what it is to be a data capable and Internet connected phone. Couple this with some fashionability and the stage is set for an almighty fight.

Looking through the AdMob report for November, it's astonishing to see how fast Android has grown in the last 2 months (doubled on traffic requests through their network) but more importantly was the launch of the Motorola Droid and the whole Droid Does campaign. The Droid is one of the fastest selling phones of all time almost hitting iPhone 3Gs sales levels (which was working from an installed base upgraded) and is now accounting for about a quarter of Android device share - only behind the G1 which has been out for 18 months - expect to see that change over December.

Now Motorola have entered the fray and with Samsung and Sony Ericsson both scheduling major launches into Q1 2010 the mobile landscape is going to get increasingly messy as the iPhone isn't the only great phone out there. Indeed I think Sony is going to do a Motorola with the Xperia X10 as it is simply stunning and is a big name in the mobile space - especially in Europe. HTC have had a great lead but 2010 will see Motorola and Sony return to some dominance here - and they can fight Apple in the Fashionability stakes.

The biggest challenge for Apple is how to combat Google on the phone itself. Outside of iTunes, Apple has little in the way of first party apps for the iPhone and whilst it has a huge developer network it is definitely alienating them through it's App Store management nightmares. Many developers are developing for both iPhone and Android devices - especially those using Web technologies for building and apps like Phone Gap to cross-package.

A lot of what makes the iPhone really useful are Google applications (native Gmail, Maps and most importantly Search!) - Apple has no way to combat this. Are they going to deny Gmail or Search like they did with Google Voice?

Apps that are available on both platforms and services that are available "in the cloud" eg Maps, Comparison Shopping etc dilutes Apple's position as it's only point of differentiation becomes fashionability and both Sony Ericsson and Motorola have competed for over a decade against Nokia by building highly fashionable phones.

I'm not sure this fight will be a death match but all the signs are there for a battle of epic proportions. Both are likely to be extremely battered by the time they come out the other side and would be wise to hold a little bit in reserve in case Nokia's Maemo platform takes off the way they are expecting it to - at that point things could get really messy.

SPDY could gain acceptence very quickly - with some product innovation

2009-11-13T12:12:00.002+11:00

Google have announced some early findings about their research into a faster protocol to reduce latency times due to good old fashioned HTTP. HTTP was designed as a really simple protocol to delivery (primarily) text content over the Internet and thus was born the Web.

One of the problems with HTTP is that it only really allows a single request to be serviced at any one time. The reason this doesn't APPEAR to be the case is because modern browsers create multiple connection threads that connect independently to the server and it gives the appearance of things downloading in parallel. It's a neat hack and works because we have good network speeds and mast processors to manage all this multi-tasking. Go back to a Pentium II with Netscape 2 and you'll watch the glacial procession of elements loading in from the top and goes down the page.

The Google project page talks a lot about why HTTP pipelining doesn't work and some of the technical architecture behind SPDY which I won't cover here other than to say that it's great we are seeing this type of innovation at the protocol level. What's most interesting for me however is how we get it in production.

There is a lot of nay-saying going on around this suggesting that because of the size of the Web you'll never get people to shift to a new protocol HTTP:// won, let's all leave it at that because there are too many web servers and web browsers to convert. This is what I want to address in this post.

Yes - there are fair too many legacy browsers to deal with to make this transition happen. Look how many IE 6 browsers are still in use, but we'd also have to shift all the Mozilla users, Chrome users (easy because of forced update) and Safari users as well. Not to mention all those pesky mobile devices that are springing up.

Dealing with the web servers is a much more straightforward issue. There really aren't that many in the scheme of things. Indeed much of our existing infrastructure runs multiple servers, Apache alongside a lightweight server like nginx and this is increasingly common.

As such there's nothing stopping me dropping in a SPDY server alongside my existing infrastructure for those users that can directly access it (Chrome 4, Firefox 5, Safari 6 and IE 10 for example).

But let's not stop there. A network admin could create a software appliance at the Firewall or Internet Gateway level for the corporate network that took HTTP requests, turns them into SPDY requests and then proxies these back. Now I have doubly fast Internet connectivity without upgrading my connection. For the price of a box that is well worth it.

For home users we could do the same thing. This protocol is software - it runs on TOP of TCP so because of that a Firmware upgrade of your average Netgear or Linksys home router could get you the same benefits as those above. ISPs could force this remotely on certain systems (Cable for example) or provide info on how to do it such as through a web, phone or personal service.

So for all the nay-sayers out there - this is a MASSIVE opportunity to speed up the web and people need to think outside the browser sometimes. QoS was delivered at the router level based on intelligent packet analysis - that speeds up network traffic massively but it's a software change not a hardware one.

I don't think it will be long until we see Netgear and Linsys start promoting this like they did with the WiFi standards and force adoption because it makes a great marketing case to do so.

I'll be trying this out at the rawest state to see if we can make it work and if I can, watch how fast our servers and network gateway get upgraded before I embark on upgrading client machines.

AdMob purchase by google paves way for interesting developer funding

2009-11-10T10:14:00.005+11:00

It's just been announced that Google is set to buy AdMob for $750M in an all-stock deal. This is the third biggest purchase Google has ever made (the only two bigger are YouTube and DoubleClick).

AdMob started in 2006 so they have capitalised very well for a 3 year old business. Indeed they've been cash positive for a while now so this is a great acquisition by Google. The full gory details of the deal can be found here and a press site by google here

We know this is all aligned to Google's interest and in particular their big appetite presently for anything Mobile. However this also opens up some enormous opportunities for developers.

This acquisition brings with it some great opportunities for in-application display advertising that is delivered contextually but also based on Google AdWords auctioning technology. Along side this I can then use the same advertising account to drive ads on my mobile website that compliments my application and then use standard ads on my main website that provides additional information / community support etc.

All of a sudden a possible revenue opportunity opens up that was kind of there previously but wasn't very smart. Over the last 18 months in particular we've been watching the rise of free-ad-supported applications as well as paid-no-ad versions of the same application. I would expect to see a lot more of the ad-supported apps once this deal goes through.

The reason for this is twofold:

1. As a developer I can manage all of my advertising spaces with one vendor. I don't really want to have to deal with all these businesses I just want to get some beer money for my app that I'm spending my non-work hours producing.

2. With contextual ad serving, I can make certain elements of data within the application available and use that to generate calls to the Ad Server - much the same way AdWords works with a web page or in Gmail. This means the ads that are served will be more relevant to the content which should lead to higher Click Through which then leads to potentially more revenue for me (see note above about beer money)

This makes a lot of sense for an advertiser as well. Certain applications have huge amounts of uptake - twitterific on iPhone or Twidroid on Android for example. Imagine having contextual ads served based on the content of your twitter stream. Twitter might resist it but it could make some serious cash for the app developers.

Overall I think this will really blow the top of mobile advertising. Advertisers who have been a little shy in the mobile space will be comforted by the fact it's Google doing it. App and mobile site developers stand to gain some good funding from it and it be relevant for their audiences and as the world goes increasingly smartphone mobile mad over the next 18 months this will be worth serious $Billions in the next 5 years or so.

Cross posted to Citrus Agency Blog

Crown Oaks Day Racing Challenge

2009-11-05T07:09:00.005+11:00

Last night I was writing code to play around with an idea I had rather than studying the Form Guide. See today I am off to the races (Crown Oaks day at Flemington) with some clients - hence why I should have been studying the form guide and not playing around with Erlang.

So I've decided to try an experiment:

Can the wisdom of the Twitter crowd outperform both blind luck and the bookies favourites with regards to return on bets during the racing.

Now we all know blind luck should lose. Betting on a winner at random is probably not going to get a single hit but against the favourite should prove interesting.

What interests me most is that betting on a race is actually contributing to an Information Market. Now theoretically information is held by all the various agents (people betting) and each one doesn't have a full picture but together the market becomes efficient and pushes down the odds of certain horses winning and then having long shots.

In smaller races where you have not so many people betting this works and either the favourite or a horse will relatively low odds will win. At large race meetings this doesn't work because a lot of people bet randomly (based on name, birthday number etc) and because of this it creates a lot of noise in the market so it breaks down.

So here's the challenge.

I'll start this from Race 3 at Crown Oaks Day today.

Each of the races is displayed below with a link to the field list. I'll be making a bet based on complete randomness (random number of the horse) and following the bookies' favourite. I'll then take the majority from messages to my twitter account (@ajfisher) for that race and place a bet following that. Simply send me a tweet "@ajfisher Race: NUMBER Horse: Name or Number"

Race 3
Race 4
race 5
Race 6
Race 7

Also you can use the tag #crowd-oaks if you want.

So can a smaller crowd provide more wisdom and outperform the bookies and complete blind luck on a big race day. Let's find out. It'll be fun either way.

The only reason why Linux isn't ready for prime time desktop

2009-10-11T12:58:00.004+11:00

Okay, so this title's probably a bit misleading as there are probably a few reasons but as far as I'm concerned there's only one thing stopping my final transition to desktop Linux for complete every day usage.

Presenting

In my job I do a lot of presenting. I give major milestone presentations on projects, I present to the business on things that are going on, I present in pitches where we are attempting to win new business and recently I've started presenting at conferences.

I would not use my Linux desktop (and I have combinations of Ubuntu and KUbuntu 9.04, CentOS) to present with at all - even if someone paid me.

Before I say why I'll also lay out my Linux credentials. I use RHEL, Ubuntu and Centos EVERY day. All of my home computers are Linux based, I have a Linux PDA, I prefer my Ubuntu desktop for work and I administer numerous Linux (CentOS and RHEL) servers - via command line - all of the time. I've used it for over a decade and am more than happy with it and more than happy to hack on it to get stuff working.

However, there comes a point where I am not going to entrust a complete presentation that our business or my reputation relies upon to Linux's extremely flaky graphics system.

Yes, I know laptop Linux is problematic (but if the rest of the desktop is stable why not my second video out?)

Yes, I know that graphics card support (particularly from ATI) is very closed so there's lots of reverse engineering going on (but again if I can have one video out working why not two?).

I'm not sure why this is the case - I think it's a combination of X.org config and poor tools for configuring multiple screens with different resolutions but it definitely needs a lot of work to go ready for prime time.

I was at a conference this week and I had built my entire presentation in my Gnome desktop using FLOSS tools like Open Office Impress, had a great looking presentation and was legitimately keen on presenting using either my Ubuntu or Centos desktop. After hours of mucking around however I didn't feel supremely confident in just walking up to the podium, plugging in my laptop and "It Just Works"^TM. It's just too hit and miss.

I don't generally experience this with Linux in general and Ubuntu specifically although I am aware of other people saying it. For me 99% of the time it does actually just work.

So I defaulted back to my dual-boot Windows partition and presented from that instead. This was the partition that I had considered nuking because I hadn't used it in about 6 months. In this instance though I didn't have any other choice - and sure enough it did just plug in and go.

I still presented from Open Office Impress though (which is a fantastic bit of software I might add!) and I think I was the only one at WDS09 that presented with it (and I'm sure no one could tell I wasn't using PowerPoint or Keynote).

Desktop experience is exactly that - an experience and our experience, particularly when we are doing something social with a computer can affect our mental state quite substantially.

If I'd have taken the decision to present using Ubuntu I would have felt worried about whether my laptop would work and I would have been nervous and probably would have delivered a terrible presentation. In contrast because I knew I wasn't going to have any support issues I felt confident, in control and delivered what I hope was a good presentation to the audience.

Ubuntu are trying to address many of these issues with the Paper Cuts project but that's really aimed at business. Apple have addressed similar issues (hardware compatibility) by having a presenter's kit (which you buy) which provides all kinds of adapters to go from Mac to just about every video input type. Microsoft addressed this years ago from Windows 2000 with a great set of dual head tools that made it simple and a standardised way for vendors to incorporate them and it is extremely rare for it to fail.

Business use is one of the areas that Linux (and especially Ubuntu) has got a real opportunity to shif users across as there are so many other business benefits but users want a single consistent desktop so they aren't going to build on one desktop and present on another - it's too inconsistent.

For me this issue on presenting and graphics support isn't so much a paper cut as it is a gaping flesh wound and it really needs to be addressed.

Web Directions South 2009 - Cloud Sourcing the Business

2009-10-09T13:26:00.008+11:00

I recently gave a presentation at Web Directions South which was fantastic (the conference I'm referring to here!). The session was on cloud computing and I hope everyone got something out of it. I've put the presentation itself over at slideshare

Cloud Sourcing the Business

View more presentations from Andrew Fisher.

My speaking notes are below as the presentation won't make much sense without it as it's mostly images or single statements.

Slide: Introduction

Cloud computing is a fast moving part of the IT landscape right now and, media hype aside, it does actually represent a fundamental shift in the way we produce and consume IT services.

[CLICK]

Slide: Cloud computing simplified

Partly because of where Cloud Computing resides on the hype curve but also because of it's pace of change, understanding the landscape can sometimes be confusing and it's probably best explained by this slide.

Today I'm going to talk about what cloud computing actually is in practice, how cloud sourcing can be used in your business as well as it's future direction and some of the traps you need to be aware of.

[CLICK]

Slide: What is cloud computing?

Cloud computing encompasses a huge array of services, technologies and techniques. Marketers would have you believe everything plugged into the Internet is a cloud. To understand the distinct areas of cloud computing let's try a little experiment.

Hands up if you are right now using some kind of Cloud Computing Service in your business?

Okay so if you are using some kind of infrastructure service such as S3, EC2, vCloud or RackCloud put your hand up.

If you're using something like Google App Engine or Force.com actively right now, stick your hands up.

Okay, so what about those people using Gmail, Google Docs, Office Live, Hotmail, Twitter or Facebook or something similar to that?

So nearly all of you are using cloud services right now without perhaps even realising it. So my work today is done as you're all experts...

As you can see, cloud computing covers a lot of different services and disciplines. Partly because of press coverage the things you expect to be Cloud Services such as infrastructure are there but so too are things you don't think of such as google maps or Gmail.

The networking company F5 commissioned some research not long ago with many CTO and CIOs of large US businesses on what they considered to be THE definition of cloud computing. They trialled 6 front runners for what cloud computing is.

None stood out from the rest as a sufficient or complete definition.

Googling Cloud computing will give you a heap of results and nothing really convincing either.

[CLICK]

Slide: Infrastructrure

Hardware guys generally believe if there's no infrastructure then it can't be a cloud.

Slide: Web 2.0

Software guys like me generally take the line that it's all about Web 2.0 style applications.

[CLICK]

Slide: NIST Def

The National Institute of Standards and Technology in the US have come up with this working definition. The full thing runs to a couple of pages, but here's the guts of it,.

Cloud computing is a pay-per-use model for enabling available, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and so it goes on...

Obviously this is a definition by committee in order to be able to get as wide a consensus as possible

NISTs definition is great for someone who is firmly entrenched in the cloud community whether as a vendor, seasoned consumer or consultant but is a bit impenetrable for everyone else coming in.

So for the purposes of today let's go with something a little more straightforward.

[CLICK]

Slide: Cloud Computing Definition

Cloud computing is: A Network based Service, that is available on demand such that the end consumer considers it not their problem

I think it sums up the spirit at least of what NIST is trying to say.

Let's dive into this a little.

[PAUSE]

It's a NETWORK BASED SERVICE. Cloud computing provides a service over a network of some kind.

[PAUSE]

Clouds are provided ON DEMAND. It is imperative that you can turn the service on and off as well as scale it up or down as needed. Cloud Computing is all about using a commodity so I should be able to stop using it if I need to or only use it part of the time.

[PAUSE]

Finally Cloud Services are Not My Problem. Or yours. Offering the service is going to be making someone lose sleep somewhere but it shifts the provisioning, management and maintenance issues away from the consumer towards the provider.

It's like flicking on a light switch. To make my lights go on I don't need to understand how the power gets across the cables or indeed connect the wires up myself each time I want to make it bright. It's not my problem – someone else looks after this for me. It's like Gas, Water or Electricity.

[PAUSE]

This definition is simple but allows us to explore what cloud look like and some of the ways they can work for you right now.

[CLICK]

Slide: Flowers?

Before we get into the actual types of clouds and what they can do specifically, it's worth touching on where these things live.

Clouds can be Public or Private. It doesn't matter whether you are available to everyone on the internet or just one little part of it. Service clouds just made available to your business are just as important as the massive public ones that are made available to everyone.

[CLICK]

Slide: Public clouds

Public clouds are those located out on the Internet and provided for a fee to parties that wish to consume them.

[CLICK]

Public Clouds are the ones that are being talked about most out in market such as those provided by Amazon, or Google.

I would use a public cloud because I either don't want or don't have the expertise to run one privately. Likewise as a business I can look at a public service and think that will do the job perfectly and I just pay to use it - why bother going and building my own?

[CLICK]

Slide: Private Clouds

Private clouds are typically owned or leased by a single organisation for their purposes only.

[CLICK]

Private clouds are experiencing very large amounts of growth as many very large organisations such as banks and governments start flexing up their IT to provide more services that can be combined and consumed on demand by internal divisions.

[CLICK]

Slide: Hybrid Clouds

Much like a hybrid car where you combine something like eco-friendliness with performance, it's also possible to have Hybrid Clouds

[CLICK]

As you'd expect Hybrid clouds are where businesses are using a combination of public and private components. This provides a huge amount of flexibility to the business as decisions can be taken around which services you are happy to have public and those you prefer to keep private.

[PAUSE]

So now we know where these clouds are located, it's time to look at what they can do.

[CLICK]

Slide: what can cloud computing do?

So now we know where these clouds are located it's time to look at what they can do.

[CLICK]

Slide: Service

Remember in our definition I talked about how Cloud Computing was some kind of service delivered over the network? Well unimaginatively every time you're looking at something cloud related you'll always see it called Something as a Service. The word Service in Cloud land is almost interchangeable with commodity and Cloud Computing is really all about taking some piece of technology and providing it in the most commoditised manner possible.

[CLICK]

Slide: Cloud Computing Landscape

The current Cloud Computing Landscape is made up of four major service areas- Infrastructure, Platforms, Software and Data.

Infrastructure and Software as Services are currently the most well known. These have the highest uptake from consumers at the moment – Amazon's Infrastructure Services alone generates over $250M in revenue a year. However there are more services you can use as seen by platforms and Data so let's take a tour around the Cloud Computing Stack.

[CLICK]

Slide: Infrastructure as a Service

This is most commonly used form of Cloud Computing currently. It's commodity hardware or infrastructure that is available for consumers of storage or processing.

[CLICK]

Slide: Server room

With infrastructure you can go and put your own servers in your own room and manage them yourself or else you can go and use some else's who have built it on a drastically different scale.

Usually these are paid for based on some kind of metering such as GB of storage or hours of processing time.

The consumer usually doesn't need to manage the the underlying infrastructure, they just put the pieces together and configure it as they like. Infrastructure Clouds have really taken off over the last 18 months or so because of the investment businesses such as Amazon, Go Grid and RackSpace are making in their User Interfaces making it quick and easy to provision storage or full servers or appliances.

Slide: Aws

As you can see, being able to manage your infrastructure services

[CLICK]

Slide: Go Grid

Is as simple as clicking a mouse.

[CLICK]

SLIDE: Using Infrastructure Services.

So how can I use this? With Infrastructure Clouds I can go and create everything from a single virtual server to host some site or project all the way through to creating a virtual data centre – in minutes.

Because I am a massive cloud geek I just decided one evening to create and run a Neural Network using 300 machines in Amazon's EC2 cloud – just because I could. It cost me about $50 and worked perfectly.

Many Infrastructure vendors also provide storage clouds where you can upload files and pay for the storage and traffic.

In practice, these services are great for trial projects or for when you need massive scalability very quickly.

SLIDE: Melbourne cup

Last year for the Melbourne Cup we deployed a Hybrid Cloud to help with web site demand during the Melbourne Cup carnival. Most of the year, the Victoria Racing Club's website is busy but manageable with pretty average infrastructure.

The Melbourne Cup is Australia's biggest sporting, cultural and social event with hundreds of millions of viewers world wide.

Over the Melbourne Cup Carnival, the site visits ramps up considerably with over a thousandfold increase in traffic, most of which is over only a couple of days and 90% is during about 6 hours on Melbourne Cup Day.

Using Amazon's S3 service as a Content Delivery Network, we hosted all our heavy assets such as images, video and flash files. Amazon's EC2 service was used in order to deploy servers on demand as they were needed for extra capacity.

Alongside this we then had a private cloud at Web Central that could be configured how we needed it based on the types of traffic we experienced.

The solution proved successful with nearly a million visitors on Cup Day and at peak over 200,000 files being transferred a minute.

On top of this though, it was also extremely cost efficient because VRC don't have servers sitting around doing nothing for 360 days a year as all of that extra capacity was switched on just prior to Melbourne Cup and switched off 2 days after.

Using Infrastructure Services in these circumstances helps you maintain a great user experience and also makes your costs much more manageable and reduces your provisioning time.

A lot businesses are using this right now to help with managing their heavy files. Look where Twitter hosts it's images...

[CLICK]

SLIDE: S3 Amazon Screen shot

That's right – Amazon S3

[PAUSE]

Given the sheer numbers of people visiting twitter now this makes complete sense for them to be able to manage the costs of image hosting and delivery.

At last count they had over 60 Billion items in there which is growing at a rate of about 10 Billion every few months so there's plenty of people using this and similar services out there. The infrastructure services are getting better all the time as competition is driving innovation.

[CLICK]

SLIDE: Platform as a Service

Platform clouds will be immensely popular over the next couple of years. They are 1 level up from Infrastructure services and these clouds take away the “machine” part of the cloud giving you a platform fully configured and ready to deploy your application.

[CLICK]

Slide: Diving platform

This means developers get the on-demand parts of Infrastructure clouds without having to concern themselves with managing a virtual data centre or configuring machines.

Certain assumptions are made at the platform level – for example Google App Engine restricts you to certain languages you can use. However in making these assumptions, it means developer has a defined jumping off point. Google App Engine and Force.com are seeing an explosion of developers who are embracing the platform and coming up with new ways to use it.

[CLICK]

SLIDE: Logo Slide.

At the moment platform clouds are in their infancy however they are growing rapidly. The two most well known and heavily used right now are Google App Engine and Force.com which is the platform underpinning Salesforce.com

The single biggest benefit of Platform Clouds is simply Speed to Market. The development process is drastically shortened because many behind the scenes decisions have been taken for you. As such the testing time is shortened because if you're using components of the platform they are already tested so you only need to test your code not the Platform's. Additionally you also don't need to go away and build an entire system stack – databases, servers, APIs etc – it's all there waiting for you to come along and do something with it. Ongoing it means a lot less maintenance as well.

Developers are looking at Platform Clouds in order to spend more time working on innovations and less time on replication. As a developer, why would I want to spend time writing yet another application that puts some customer information into a database off the back of a web form and then write some little app that allows the customer service team to make sense of it. It's this behaviour that Platform Clouds are addressing and the side effect is that not only do I get the benefits of having a complete system to work with that's developing new features all the time, I get to produce it quickly.

Nucleus Research looked at 17 projects done using Force.com earlier in the year and found that speed of delivery had up to a 10 times increase. The average across all projects was about a 5 times increase in speed.

Imagine what you could achieve right now if the projects your business is doing could be in market in one fifth of the time it's currently taking you? Just consider that for a second...

Developers are citing similar benefits of App Engine as well as some of the other platforms out there though little official research exists just yet.

As these platform clouds really mature over the next year or so the pressure will be on for businesses to start using them a lot more heavily as it will be uncompetitive to build equivalent systems from scratch.

[CLICK]

SLIDE: Software as a Service

Salesforce.com has been the Pin Up of SaaS for a very long time – in many ways they defined what it was and probably started a lot of the Cloud Computing discussion as they showed very early on it was possible to commoditise a service and deliver it over the Internet.

The reason Software clouds exist is because in the traditional Software industry there is a dirty little secret – maintenance costs more than innovation. Most Software Vendors have a small team coming up with new ideas and developing software then a massive one looking after the support and maintenance of these new ideas. Software Vendors have product lifecycles where they support certain versions of a product until they can justify not doing it any more and then hopefully force the customer to upgrade. During this lifecycle various customers could be on many different release versions and all of this adds to the cost of support.

The heart of Software as a Service is that there is only ever one version of a product, delivered over the Internet and all customers use the same version until the next upgrade. Immediately much of that time used for maintenance and support is available for sales, marketing and innovation.

Is it any wonder that Salesforce is now on it's way to dominating the global CRM space.

[CLICK]

Slide: SaaS Collage

Most software is now moving in this direction and is a big part of what enables Web 2.0 and is transforming the way developers deliver and maintain software for their user base. Look at software like Google Docs – who'd have thought even a few years back it was even remotely possible to use a spreadheet like Excel in a web browser.

Consumers of Software Clouds are generally pretty ignorant of all the underlying platform and infrastructure and are only interested in configuring the application itself.

[CLICK]

SLIDE: Gmail screenshot.

There are many applications of Software Clouds, far too many to go through today and my advice for any business procuring any new piece of software for the organisation is to make sure you seriously consider a Software Service in your selection process.

As an example let's look at Gmail. I'm guessing a lot a people here are using GMAIL for your personal email however Google has a fully supported Enterprise edition that works for your domain. You can still use outlook if you want or use the web client or mobile client if you prefer.

Email for a business is as important as cashflow however for many businesses the management and in particular cost of management of email is significant. On Premises email is something you have to maintain – backups, mailbox cleanouts, archiving, system upgrades etc are all the standard tasks done by an IT manager whether employed or contracted. Even with all this time spent, email still occasionally goes down.

Taylor Woodrow, a large construction business in the UK had much this scenario. They are a large business with about 1,800 employees and they were managing all their email in house using their IT team.

Last year Taylor Woodrow switched over to Enterprise Gmail, moving away from their on Premises corporate email they'd had for years. Whilst there are still costs with Gmail such as licenses to pay – Gmail costs $50 per user per year - and you still need someone to administer the accounts the cost savings for the business were massive. Over the year they saved about $2 Million compared to their previous mail software.

Given that large organisations have a scale of efficiency and they can get great cost savings per user, the benefits to smaller businesses are proportionately greater.

For an Australian business that has between 20 and 50 employees it's estimated that many businesses will save on average about $100,000 a year in total costs due to reduced licenses, man power, backups, hardware requirements etc.

That cost saving could be ploughed back into marketing, sales or other IT projects such as building your own Software Cloud.

Again, this is just one Software Service example but it shows how it's possible to start realising cost savings and get a better solution. Whilst I'm sure everyone here is broadly happy with their email system, are you now or will you ever be the best email providers? Probably not. Gmail can do it better and cheaper than you can. This is the crux of most Software Services – they can be provided, maintained and constantly upgraded to a much higher standard and for less cost than you can do it for and that's why we seeing them grow so rapidly.

[CLICK]

SLIDE: Data as a Service

Data as a Service is the newest part of the cloud computing landscape and I see this as having the most opportunity to shift how we create and consume data and is the one you can get involved in right now..

[CLICK]

SLIDE: NY Times Visualisation

Data is being provided everywhere and we are just now starting to see the interesting ways that people can use it. Data as a Service provides the consumer with the ability to retrieve and publish data – whether their own or that which is publicly provided - via the service.

The consumer gives no consideration to the underlying software, platform or infrastructure, only whether the data is available or now.

Data clouds are being used to expose data out to end consumers such as via Retail or Search APIs so those consumers can take them, remix them and provide them outwards again.

Data as a Service is built for mash ups – think about all of those sites with mashups of people's tweets plotted on google maps

[CLICK]

Slide: Zappos

or this great Zappo's site that shows what products people are looking at on a map showing where they are located.

[PAUSE]

SLIDE: DaaS Applications.

Data Clouds are so new that there aren't that many examples to show you yet though over the next 18 months we'll see an explosion in the data that's being made available.

Organisations that are starting to do this are people like the US Government with their Open Government or Government 2.0 initiative. Lower level governments are starting to make their data available such as the San Francisco city government who are making available data sets for things like crime data, highway information, and lists of public trees of all things.

Whilst these data clouds are very new we are seeing some interesting business models develop out of them. A US company called Jigsaw has recently become self funding. Jigsaw specialises in managing personal data by crowd sourcing contact information from participants and then they use that data to sell to marketers who want to cleanse their own subscription or marketing lists or who want to acquire new customers with very specific requirements.

Other businesses are looking at creating industry based data clouds to manage source information such as where livestock have been reared, transported and eventually slaughtered before making their way to market thus providing a traceable and authentic source for produce certification.

[CLICK]

Slide: Layar

The other area where Data Services are starting to appear is in the burgeoning field of Augmented Reality. Consumer ready AR is really being driven by and will continue to be driven by Data Clouds as huge data sets are made available to consumers via their mobile phones or PDAs.

Layar for Android and soon for iPhone is one such app that allows data sets to be viewed on your mobile device. Perhaps obviously at the moment this is skewed towards things like tweets, photos from Flickr or Wikipeadia entries that are near to you but this will change as organisations start to make their data available.

The examples on screen now show you where your local 7-elevens are and some points of interest in Sydney.

This space is very much in its infancy right now but it's definitely an area you can get involved in massively – if you're sitting on data make services available to get at it and watch the things people start doing that you never considered.

[CLICK]

SLIDE: Cloud stack display again

We've taken a tour around the cloud computing landscape and you can see there are a lot of areas that you can use. There are also some extremely niche areas I haven't talked about such as Desktop as a Service where your computing desktop is all cloud sourced however we'll leave them for today.

One really interesting thing that I've noticed about Cloud Computing is that as you go up the Cloud Stack consumers tend to USE more of those services at the bottom to PROVIDE more of the those services at the top. There's this knock on effect of one type of cloud computing providing the means for another type to occur and it's this effect that is driving much of the growth of the newer cloud types.

Now we've seen what's available let's take a look at the future to see where cloud computing is headed.

[CLICK]

SLIDE: Cloud Computing Future.

Cloud Computing is rapidly changing shape right now.

In 5 or 10 years the landscape will be as different again as it was 5 years ago.

Industries and consumers will be changed by how they interact with Cloud Services.

The fundamental paradigm shift that is occurring will allow IT to go back to being an enabler of business. Taken altogether, cloud computing becomes IT as a Service and businesses that embrace this philosophy will be able to do some great things.

Here's some examples.

[CLICK]

Slide: Manufacturing

Manufacturing will be changed by almost infinite processing power that can be provided on demand coupled with advanced computational techniques such as Genetic Programming to explore new product designs and get competitive advantage.

[CLICK]

Slide: Health Services

In health, for a long time governments in particular have been trying to make health related IT services better, providing a seamless service from one end of the health system to the other. Hybrid Cloud Services will be at the heart of our new health systems where any health professional will have proper access to medical records so they can treat patients much better.

[CLICK]

Slide: Start Ups

Start up businesses with great ideas will be able to be in market quickly, driving innovation and competing immediately with their bigger competitors, amplifying their disruptive potential within their chosen industries.

[CLICK]

Slide: Science & Research

All scientific centres will have access to supercomputer grade processing and storage which will drive new discoveries rather than just those with extreme budgets. Presently a lot of cutting edge science and research is done by centres that have almost unlimited funds. Now those with much more modest budgets will be able to use On Demand and Cost Effective cloud services to help their research.

[CLICK]

Slide: New media

All online sites and campaigns will be able to deal with large scale traffic and maintain high levels of availability, not just those with large budgets. Developers will be able to take advantage of this scalability and push the boundaries of the content they can deliver and the applications they can create.

[CLICK]

Slide: New Markets

New markets will open by providing guaranteed access to data for consumers such as to their music and videos no matter where they are in the world – this is at the heart of services such as Hulu.

Opportunities for businesses guaranteeing management and security of users personal data or industry data in so called “Data Banks” will start to present themselves. How much would you pay to have on demand access to all of your personal information all in one place that is searchable and secure?

[CLICK]

Slide: Geography

As a side effect of all these Cloud Services becoming available, businesses will no longer suffer from geographic location. Businesses in Australasia, South America and Africa will compete directly against Europe and North America. Competition will be based on knowledge and innovation not on available capital.

As you can see, all types of organisation stand to benefit from Cloud Sourcing.

[CLICK]

Slide: Warning sign

Whilst I've painted a picture of almost infinite computing resources being available, great bits of software being produced and a veritable mountain of data that's about to come online it's not all rosy.

With the pace of change of the technology outstripping everyone's ability to react quickly both within Government and Business, Cloud Computing is currently in a similar position to the Wild Wild Web of the early 90s.

Clarity is the most important requirement for any organisation producing or consuming cloud computing services.

Here's a short guide to some of the things your business needs to address before you start Cloud Sourcing.

[CLICK]

Slide: Disaster Recover and Risk Management

Disaster Recovery & Risk Management are no less important than they are in your business right now! Just because clouds are someone else's problem doesn't mean the risk management and recovery issues AREN'T your responsibility.

If you want 100% availability you have to work for it whether you are using Cloud Services or not. 100% availability is only possible by combining services whether at your DC or via the cloud.

As an example, GMAIL had a recent outage and caused a Chicken Little style panic that “The Cloud was falling”. Taken in context however, even with the number of people affected Gmail is still more reliable than corporate non-Cloud email which fails quite often.

Disaster and Risk management are no less important just because you are using Gmail or some other cloud email service. You need to have a policy to deal with outages just as you would do normally.

[CLICK]

Slide: SLAs

SLAs at the moment are the worst part of the Cloud Computing industry. Some vendors don't have any commitment, some do and there's a lot of variation between. Before committing, you need to assess the Service Level Agreement and decide if it's right for your business and right for the type service you're buying and what you're going to use it for. Some applications don't need highly available Service Levels but others do.

Slide: Data policies

Understanding the operational risk with regards to data is hugely important in a Cloud Computing environment as your data is probably going to be more public than you are used to.

Existing data policies may need to be updated due to the additional risk of being in a Cloud Environments. Many Vendors are actually better at managing this than their clients as they are doing it day in and day out.

Slide: Question mark

Before selecting a vendor it's important to ask the following questions and understand the responses and what they mean for your business

[CLICK]

1. How safe is your data?

Who has access to the physical machines?

Can the DC be cross contaminated? In large data centres it is possible for your data to be available even after you've switched off your service. Can this become available to someone else down the line?

Can you access your data to move it if you want to leave? Lock in and standards around this are a massive issue that is still not completely addressed

[CLICK]

2. How safe is your uptime?

What happens if a major hub is unavailable because of some Internet issue?

Who do you call when it all hits the fan at the Vendor end? How is their support provided? How do you support your consumers?

Who is monitoring your services?

[CLICK]

3. Are there any border concerns?

Laws are different between Australia and places such as the US and Europe.

Crossing international boundaries may bring with it additional legal requirements or may mean your services or data are subject to different laws than you are used to. Legal counsel here is imperative.

[CLICK]

Slide: Traffic light

Don't let me put you off. Forewarned is fore armed and none of the issues are insurmountable with the right policies put in place by your business.

If governments are using or considering Cloud Services then you are almost guaranteed that the issues with SLAs, Uptime and Data concerns are being addressed right now. Few businesses are as stringent in their requirements as multinationals and government agencies.

Also, bear in mind that many of these issues should be taken into account for any of your current IT services anyway so knowledge and policies you have in place now are still relevant for cloud service provision.

The important thing to remember is that you need to assess risk against the possible opportunity and make an informed business decision before embarking on any IT project.

[CLICK]

Slide: Seismic shift.

Cloud Computing is THE single most disruptive technology available to businesses right now.

We are in the middle of a fundamental change in the way IT works for the business.

This is being driven by the convergence of extremely cheap telecoms alongside sophisticated software and commodity hardware has led to an explosion of available computing power and storage. For the first time in our computing history we are approaching the point where actual deployable computing resources are going to outstrip our ability to use them. This creates a vast pool of cheap resource that will be used in innovative and disruptive ways.

If you are not using cloud computing actively within your organisation soon you WILL be left behind whilst your competition streaks away.

[CLICK]

Slide: Ticking Clock

Competitive market pressures are making this transformation occur so rapidly you have only a handful of years left to make this shift.

[CLICK]

Slide: Revolution

Cloud computing is as much of a revolution as the Internet was to business in the 90s but will have occurred in less than half the time.

Why? Because Cloud Computing enables not only the commoditisation of technology services but the democratisation of it as well.

In the same way that cheap PCs enabled an information explosion and reformed the way we communicate, Cloud Computing changes the way technology is used by everyone. Large or SMALL.

When the cloud computing revolution is complete there will be virtually NO distinction between the large and small business with regards to the scale and quality of the technology they can deploy to market.

[CLICK]

Slide: Zero

The cost to deploy these technologies will be almost ZERO

[CLICK]

Slide: Time to market

The time to be in market will be days and weeks not months or years.

[CLICK]

Slide: Infinite availability

The flexibility and scalability of the services available will be virtually INFINITE.

[CLICK]

Slide: What can I do now?

How can you start with Cloud Computing now?

If you're a business get your CRM onto something like Salesforce, get your email to Office Live or Gmail.

If you're a developer, get to grips with something like Google App Engine and deploy your next app there so you can build it faster.

If you're about to launch a campaign, use some infrastructure services to support it from the outset using something like S3, EC2

Are you sitting on a pile of data? Make it available through a data API. Chances are if it's about products you'll start selling more of them.

Here's some tips on how to sell cloud computing into your business.

Slide: Cost focussed business

If your business is cost focussed – and let's face it, most are at the moment – Cloud Services are Operating Expenditure rather than CAPEX. There's no depreciation, financing or generally up front costs. You pay for what you use and nothing more.

Cloud services are generally cheaper than maintaining Infrastructure, Platforms and Software of your own.

Providing Data as a Service might lead you to discover additional revenue streams or result in more sales of your core product.

[CLICK]

Slide: Process focussed business

If you are a process oriented business then you need to look at starting with with small, low risk trials and projects.

Look at primarily Software services for a starting point.

Make sure you do your homework around SLAs & legals. Most Cloud vendors realise there are issues and are attempting to fix them.

You can also consider setting up your own private cloud. VMWare, Melbourne IT, GoGrid and Rackspace all do this and I'm sure would be very happy to help out.

[CLICK]

Slide: Innovation focused businesses

If you are an innovation business the key thing to stress is that you can be in market faster by using Software Services or developing on Platform Clouds. Your focus needs to be on how you can innovate using these tools and how they stop you from having to replicate and go over ground that has been covered many times before.

For innovation businesses you can trial things out with much less commitment. If the results are poor, switch off the service and the costs stop. If you have a hit you can scale up quickly to accommodate your new market.

Innovation businesses will typically combine many kinds of services together in novel ways. Experience across all parts of the Cloud landscape will provide significant competitive advantage in market.

[CLICK]

Slide: Last Slide

Hopefully this presentation has helped provide some insight into what Cloud Computing is, what it can do and what you need to be aware of before jumping in. Cloud computing IS a fundamental shift in the way business works.

Cloud computing is not about a single technology or a single service. All of the elements come together differently for each and every business. Cloud Computing is primarily a shift in the way of thinking about IT and how IT can enable the business to radically transform the way it operates.

Those businesses who embrace Cloud Computing will have a significant business advantage, limited only by their ideas and their willingness to execute on them. How are you going to Cloud Source your business.

This article was cross posted to the Citrus Agency Blog

An unofficial endorsement of the Android platform?

2009-07-29T12:08:00.006+10:00

As TechCrunch reported: Pigs Fly as Facebook and Google work together on an Android App - there's been a few indicators that this might be happening, particularly with some random mentions here and there on Twitter but no one was really expecting anything to occur given the competitiveness between the two businesses.

What's most interesting about this (particularly from my standpoint as an Android user) is that it will be the only other official mobile client besides iPhone; which really endorses the Android platform as the second runner to iPhone. And in acknowledging that, it also indicates that Facebook are considering that Android will have substantial traction in the coming year - not least when you consider there are two dozen Android based phones slated to hit the market in the rest of the year which could make a serious dent in iPhone's penetration.

iPhone launching with the Facebook client has been largely cited as one of the big levers in it's sales. Smart phones have been around for a decade and there have been sleek devices previously (Nokia 7710 for example was just one big touch screen 3 years before the iPhone launched) but the mind of the consumer wasn't fired by the opportunities it could provide to them. You only had to look at the marketing by the telco's around the iPhone launch to see that the Facebook client was the Killer App for the smart phone in terms of hooking people in. It gave them a very tangible benefit to owning what would have been the most expensive handset they'd have bought to date - "I can keep in touch with my friends besides calling them..."

With an official Facebook client for Android, the same endorsement has been conferred and one of the key marketing differentiators has been removed. I'm tipping late 2010 to be an interesting time as Apple and Google really go toe to toe and start slugging it out - which will be fantastic for innovation in this space.

Case Study: Django + Agile = Sportsgirl redevelopment

2009-07-26T21:46:00.007+10:00

I've decided to write this one up because there isn't much on large scale and high speed Django development as yet and this is all still fresh in my head so it's worth getting down on paper (or screen as it were).

The agency I work for, Citrus, works with Sportsgirl, an iconic Australian Fashion Retailer and we were commissioned to help them build a community component to their site to help create a social shopping experience. The store was already there and was built as a bespoke Flash / .NET application and we had the opportunity to sit this on a different box in the data centre anyway. We thought this would be a fantastic opportunity to use Django and is exactly what it's designed for.

Architecturally we are using a LAMP stack using RHEL 5, Apache 2, mySQL (yes I know but it's to do with hosting) and obviously Django. Process wise we actually use an agency version of Agile that allows a collaborative effort between Designers, Application Developers and User Interface Developers.

Overall we were on a fixed deadline that meant the production phase was less than 8 weeks from sign off to go live including production of the site, interface and design then lock downs for content population and testing.

To make this work, everything was based around the platform - we chose as a base Django 1.0 and then layered into it a stripped down version of Pinax (we currently use v0.5.1 - the current official release, with updated apps) that has user profiles, avatar and gravatar functionality, photologue photo / image management, blog, pyBB forums, user voting and commenting.

With an established platform all three teams could start working concurrently much more effectively. This is one of the biggest benefits of Django and working to a framework and platforms like it because code can be prototyped so fast to a development build that everyone can see what they have to play with - Thanks to @jtauber and team at Pinax for that as well.

From there it was a case of lots of designing, interface creation, development and review to get it into it's final state ready for testing.

During this time we also worked on the flash home page produced by our flash master complete with nice collision detection, and full modularity so maintenance on this is all about creative not about development every time there's a refresh (very often on this brand). We'll cover this in more detail at some point.

The final phase saw deployment to the live environment which we did in Amazon EC2 for launch. We did this primarily for scalability reasons as the launch was going to be pretty large and promoted both on and off line.

As part of our final testing we also performed a lot of optimisation, this was based around optimising queries Django was making to the DB on both ends and we also then rolled out our delivery optimisations.

The first part of this was to implement memcached which is simply one of the best pieces of software presently available for data driven applications. On launch day we had a cache hit rate of over 80% which meant only 20% of all possible queries were going through to the database. With a couple of hundred thousand people visting the site during the launch phase this was instrumental in keeping particularly RAM usage low on the DB server as well as removing any bottlenecks to the Database due to latency.

We used nginx alongside apache to deliver all the static files on the site (not least because the imagery is so hi-res it was killing Apache to serve it!!). I'd often wondered how well this would work with a reasonably trafficked site but I wasn't disappointed. nginx dropped the load off the apache server which struggles for both CPU and memory (even with static files served outside of Django) from peaks on pre-live at 90% CPU and 70% available RAM + SWAP to 25% peaks on CPU and 30% RAM which is what Django was using to deliver pages with Apache's overhead.

The site went live on July 8, 2009 coinciding with a very large in store, off line and online campaign that drove quite a bit of traffic to the site. The server functioned exactly as required and with the optimisations peaked at only about 60% utilisation.

Overall this was a great project to work on not least because of the Agile process coupled with a technical foundation that allowed us to work even more collaboratively. 8 weeks for a major site launch is hard work for everyone at all levels no matter what their involvement. A great team helps with this but having the benefit of fantastic Open Source platforms to get our clients into market makes this even more achievable. Even less than 2 years ago I'm not sure I'd have attempted what the team achieved.

The Golden Age of mobile? Soon maybe...

2009-07-22T09:24:00.001+10:00

Some would say that it's already been in the heady days of GSM Data and WAP, some would say it stalled when European clients pulled all funding from mobile internet apps in the post-dot-com-crash GPRS days, some would say that with the advent of the iPhone we're there in all it's shiny-coverflow-enabled-finger-waggling-goodness.

It seems like every second person is now weilding some kind of internet enabled device and in Europe and the US the penetration is even higher than Oz although we are racing for a frontline position showing that reasonable access is more important than either coverage or cost.

Half way through 2009 it's interesting to look at some of the predictions for this year - particularly where mobile is concerned and take a quick stock.

The big 3 (Apple, MS, RIM) of last year are now well and truly the big 6 with highly competitive offerings from Android who we all knew had aspirations, the re-emergence of Palm with a life-recharging elixr known as Pre and of course Nokia firmly touting its Maemo platform that's been in development for many years and is arguably the most stable and feature rich of all.

Costs for data access across the globe are plummeting with Vodafone in the UK offering the first truly unlimited data packages on phones, showing we live in a commodity market that is almost free. EU laws limiting the charges for call and data roaming will see uptake rise as people start using their phones across countries as well.

Applications obviously make up a huge part of what our mobile experiences are like and I think if anything 2009 will go down in history as the year of the widget or micro app. Whilst iPhone still only supports Objective-C and Cocoa and their iron control is starting to hinder their progress on this front there are enough people keen to try and make a buck that the ecosystem around applications is phenomenal with over 50,000 available at last count.

Nokia, MS, RIM and Palm all have app stores however these are fledgling compared to Apple's and of all the other players Android is the only one that can be considered a contender with approaching 20,000 apps available, the vast majority of which are free. Android has a very hands off approach to this so its interesting to see what makes it through compared to Apple's more militant approach. Being Java based is also helping Android be the largest growing development community too as it's super quick to get up and running.

So where will we be in another 6 months? Will we look back and think 2009 is where it all started?

I think it's a little premature. We are really at the start right now. Much of what we are doing on phones right now isn't much more than we were doing 5-6 years ago just with a bigger screen and prettier graphics.

My money's on 2010 when we see a real rise of Augmented reality applications hit the phones. This is the area that will truly show what carrying the entire Internet around in your pocket can do and has been the spur for this part of computer science this year where it had languished for over a decade.

When my phone can alert me when my friends are nearby, interact with environmental sensors, buzz me when a store within 500m is having a sale on an item I'd previously shown interest in, automatically adjust its settings dependent on where I am and the privacy level I want to adopt and filter all of the information on the Internet into a 3 inch screen in a way that is contextual and meaningful then I think we'll consider the Golden Age has started.

It's been a while...

2009-07-01T09:20:00.000+10:00

So what with a relocation from London to Melbourne to take on the Technical Directorship of one of Australia's leading digital agencies and all that entails, keeping this blog up to date has been kind of problematic, especially with the lazy-man's alternative being Twitter and a corporate blog that I contribute to as well.

I've reassessed this blog though and it's going to be taking a slight departure but hopefully should be all the more useful because of that.

EEEPC's power is in the network not the machine

2008-05-10T05:08:00.000+10:00

Over here at Technology Treason we lurve ASUS' EEEPC. We liked the idea when they first came out and specifically trawled around Hong Kong computer markets to find one not long after they were launched. It's not the Apple Air or iPhone kind of aesthetic lust, we're talking about true "in sickness and in health" type love when it comes to the EEEPC.

Indeed for someone to now take this device off me it really would have to be from my cold, dead, rigamortis set fingers - and then only with a saw.

Go online and look at reviews. They fall into two camps - those who think it's great as a second machine that just happens to do a lot of funky things (see latest Linux Format June edition for a classic example) or those that just don't "get it" any wonder why the hell anyone would want a tiny-weenie machine when you can get a low spec dell for a few hundred quid now.

There is third camp however who are starting to realise that a linux based UMPC is truly a brilliant bit of kit and it's really because of the network it sits on not the thing plugged into it.

I've had mine for about 5 months and realistically I've installed about half a dozen bits of software - 10 at a push. I can do docs, review spreadsheets, skype, web browse hell even play games if I want and when hooked to a network I can do all of these things with all of the files I could possibly want.

My machine comes home and it auto connects to my home network, synchs to my media server and can play all my media files out of the box. I can check my mail and actually read it without squinting without firing up the laptop. I can connect from home to work via a VPN and mod some files for a client without leaving the sofa or the garden and be doing wht I want before the laptop has finished booting to a desktop.

At work I can use it for presentations and taking notes on projects without printing stupid amounts of documentation and hefting my laptop along with me.

It's not a replacement computer - it's a tool. A finely shaped, infinitely configurable tool. All the things I want in my phone but will never get because of the lack of keyboard,mouse and processing power and without it being much bigger.

The thing is I'm a techie, if I'm talking a walk down the street phone and wallet are it. If I'm going somewhere then it's satchel with camera, book, PSP and now EEEPC in place of a laptop.

ASUS have released details recently of a new version designed to hit off the people who think the other is too small. I don't know myself. Small is beautiful and in this case perfectly formed.

Ubuntu 8.04 - truly desktop Linux

2008-05-02T19:51:00.002+10:00

I'm quite an Ubuntu fan, having followed the project since more or less it's original inception. Given the general lack of problems with it thus far you'll notice very few entries on this blog about it.

Indeed various clients of mine are running Ubuntu servers that are easily maintained, easily managed and just generally easy and have been for several years. It's not necessarily an industrial strength OS - see Fedora for example - but for quick deployment, great security and stability and a modifiable tool (thanks to its Debian base) that just gets the job done you don't really need to look much further.

But I think that's about to change. You see Ubuntu 8.04 (the latest version that also happens to be a Long Term Support version) has markedly shifted the goalposts of what I expect from a Linux distribution.

I've been wanting to be convinced to move to a complete Linux desktop for the better part of 10 years but there's always something holding it back - lack of support for a media type, lack of drivers for particular bit of hardware, issues to do with wireless, no power management for my laptop, I can't run some third party apps like Skype... but that has all now changed - and changed in a massive way.

As I'm want to do, every time a new version of Ubuntu comes out it is duly installed on my Acer Travelmate notebook - a very good test of whether an upstart OS "just works" or not. The machine is about 2 years old but it has some quirks such as it has an ATI 3D card embedded on it's motherboard that was difficult to get working properly even on XP, it also has an inbuilt webcam - again with proprietary Acer drivers and it has gigabit Ethernet. Oh and it's widescreen.

About 9 times in 10 I don't even get to a working desktop without some hackery of graphics drivers, x.org files and I've even had a couple of "bomb-proof" distros just not even boot up to a command line.

I've been around Linux for a long time and I know what to expect, I'm an enthusiast and advocate so none of this surprises me at all and I'm prepared to work through the issuses. Most of the time I get to a working desktop with some sort of graphical interface that is mostly not widescreen, with no 3d support, sometimes wireless and without the use of the webcam. Linux isn't aimed at desktop use - it's just a side effect of people using it for development who wanted some creature comforts whilst working - notably the Gnome and KDE bods.

Imagine my surprise when I booted Ubuntu 8.04 and I logged into a graphical desktop that detected wireless and gigabit ethernet, properly displayed my screen in widescreen mode, gave me the option to run my ATI drivers easily and then configured the 3D in a few seconds and on top of that gave me a working webcam that I'd never had running under Linux and you could tell from the whoops of joy that here was something worth formatting my hard drive for.

Every device I threw at it was auto detected and installed in moments, flash drives, USB devices, a weather station, even an old MP3 player than needed proprietary Sony software to synch on XP. All handled with aplomb and with scarcely a pause by the processor.

What the hell was going on? How did we go from solid and okay 7.10 to this awe inspiring 8.04 in just six months? Had Mark Shuttleworth finally given his soul to Beelzebub in exchange for the most promising distro to date?

Then the answer came to me in a word: Vista.

Vista - that problematic and misbegotten child of Redmond that has been causing havoc in the IT world for nearly a year now. I haven't installed it on a work machine, neither has any other techie I know that wants to "Get Things Done". My dad had it and tried it daily for 6 months - he's now back on XP. I know corporate users who've had it on new machines and reverted to XP in order to decrease the amount of support required for users.

Shuttleworth and his cabal of Elite Ubuntu coders have recognised a change is in the air - particularly in Europe that is Linux's stronghold - we have an opportunity to put Linux on the desktop of millions of users who might upgrade to Vista but are worried about its impact. Couple this with a slight economic downturn and people are worried their existing hardware just won't work with Vista thus leading to a higher upgrade cost.

This latest LTS version gives novice and power users alike the ability to do anything they want with their desktop and it just works. It gives corporate users the knowledge that they have the security of support for 5 years without the rug being pulled out from under them.

We bit the bullet this week and put all our support team onto this version exclusively - WinXP was nuked off their machines. The development team are all dual booting but the number of XP desktops seem to be fading from view at a very fast rate as the requirement to just "drop in" on Windows becomes less necessary.

We've been saying it for nearly a decade but "this year is the year for desktop Linux" and with 8.04 Ubuntu the excuses for moving OS can now be left at the door thanks.

Easy product or class rating system

2008-04-28T04:02:00.000+10:00

So you've got a lovely little ratings system going on your site. All of a sudden though you get slashdotted, dugg or just your marketing starts working and you have thousands of users all rating your products / services / systems / posts / videos etc and your pages start to creak.

"It's the shared web space you're on," say your techies, "it can't handle the users" and duly bounce you to a better hosting environment at triple the cost along with the migration charges.

From time to time I come across this problem when I've either picked up code from someone else or else a techie asks me how to optimise a page that's running really slowly. In this particular instance it was caused by a ratings system in the style of Amazon or YouTube - basically a user is displayed a product and then people rate it as to whether it's any good. The real problem came when they had a list of products, each of which had it's individual ratings displayed.

The cause of this very slow page however had nothing to do with shared hosting or otherwise or direct server load - it was all down to some naive coding executing what my old CS lecturer would call an O(n)² process.

What the coder had done was get a list of products, then for each product gone back to the database and got a list of all the rankings ever made and then averaged them out. Nice and simple but frightfully inefficient and that which caused the problem I've highlighted.

This isn't the first time I've seen this and I've been asked how to build them numerous times as well so here's a well optimised method of doing it in general terms.

Consider first that calculating the average when you insert into the database is going to be computationally less expensive than calculating it every time you perform a select when a user hits the page. This sounds obvious but it's stunning how often it's overlooked.

Make two extra fields for your product table, one called average and the other called user_count or something. On your insert of the rating into the ratings table, run a trigger or else add some code that will update the product table with the updated count and a new average calculated from the ratings info.

Now when you select the product data you pull down the average and user count as part of that select and they are just simple static fields, thus adding no more computational load than the original select or view does already.

This gives you a nice little rating system that's not heavy in terms of processor load. However we can improve things once step further if you aren't interested in the data.

The option I'm providing below is good if you are just after a running average and don't care about the individual ratings being kept. I did a project recently where we weren't worried about keeping individual ratings data because the site wasn't going to be up for very long and it didn't add anything to our system to have it.

This option uses a running weighted average in order to just update the data in the product table without requiring a ratings table at all.

Some useful background maths though:

If I have a set {3, 4, 4} and take it's average I need to add the numbers and divide by the number of entries. Thus this set's average is (3+4+4)/3 = 3.67

Now suppose I've precalculated this average as I've suggested above and stored it without the individual ratings, I now want to add another rating, 2 to the set.

Intuition says to do something like this: (2 + 3.67)/2 = 2.83 which is actually wrong. Looking at the set {3, 4, 4, 2} we can guestimate that the average is going to be somewhere more between 3 and 4 than it is 2 and 3 as we've calculated above.

Thankfully a technique from statistics gives us an option here which is to use a weighted average instead. This is useful for adding sets together that have different numbers of elements within them but maintain the averages by skewing the data using proportional averages (or a weighted average).

The general formula for this is:

Avg_w = (Avg₁ * (n₁ / (n₁+n₂))) + (Avg₂ * (n₂/ (n₁+n₂)))

Where:

Avg₁ is the average of the first set
Avg₂ is the average of the second set
n₁ is the number of elements in the first set
n₂ is the number of elements in the second set

In our example this simplifies even further because our second set is actuall only one item. So let's work this through:

Avg_w = (3.67 * (3/(3+1))) + (2 * (1/(3+1)))

= (3.67 * 3/4) + (2 * 1/4)

= 2.75 + 0.5

= 3.25

Which is the answer we're after for our average.

As we know all the base line average data in the product table and we know the value of the rating we're tracking, it's a very simple function to update this instead of doing another insert into a ratings table and we just keep on doing it for every rating that has been added.

Computationally this is a very inexpensive process and whilst I'm more than happy to be shown otherwise I think this is about as good as it gets in terms of optimisation.

The key thing is we've now reduced an O(n)² operation to O(n) which is a drastic improvement as n tends towards infinity.

Phorm over function?

2008-04-22T17:43:00.002+10:00

Phorm is, and will continue to be for some time I think a hugely divisive issue online. BBC have another story today about it, this time having spoken to the various security companies like F-Secure, McAffee etc about whether they will flag a message to the user about whether Phorm has been enabled or not.

Phorm management have come out saying "it's only a cookie", the same as many other sites use to provide tracking (such as Google Analytics), interactivity (such as shopping carts or ID maintenance on numerous retail sites), or a small amount of memory (configuration information for the BBC home page for example).

The difference, though, is that the information is being used differently because data is being shared.

This is what got the Information Commissioners Office's back up because sharing data between companies without users opting in is a breach of the Data Protection Act - "But not if it's anonymous data" say the legal eagles from Phorm - and technically they are correct. This is a case of adopting the letter of the law rather than the spirit of it.

Tim Berners-Lee came out saying he would move ISP if he found out they were using Phorm and whilst I admire his line I fear the vast majority of consumers won't care or rather just won't be bothered to switch - just see how many people actually switch bank or utilitiy companies.

For me this is a case of the slow erosion of privacy at the hands of our ISPs. In a massively competitive market where margins are being squeezed ever tighter, the sale of their user data to Phorm must have seemed like the proverbial golden goose.

It won't take long for someone to cotton onto the flip side of this and market aggressively on the privacy front. Talk Talk made huge inroads as an ISP on the back of their "The Internet should be free" campaign with regard to price (being bundled as it was with other services). Who will be the first to play the "Internet should be private" card and sign up to a deal not using Phorm or other tracking software?

In my cynical world view, I think the security firms have realised this and it is 99% of the reason for why they are looking at it all as the anti-spy, -mal and -virus software is worth billions.

In real terms Phorm isn't actually that clever a piece of technology - most of what has been achieved is in the brokering of deals between ISPs and content owners and then a bit of clever gluing in the middle.

In the end Phorm will either be a great white elephant and just slip off the radar the way many technologies and companies have done or else it may actually be a spur to drive privacy legislation forward in line with our digital behaviour - how long it will take to do this however is the question as government is typically a long way behind technology in terms of law-making.

Can Yahoo really get things so wrong?

2008-04-21T20:43:00.004+10:00

Update - The guys at Yahoo came to our rescue after tracing through the "network" somewhat to find someone that knows someone at Yahoo to help us out. Unfortunately their techies couldn't explain why we'd been bloack listed either but we are now officially on their whitelist so big thanks to the guys for helping us out.

Yahoo are one of the original dotcoms. They've been around for a long time so they should know their business. Imagine my surprise when one of my clients starts complaining that their confirmation emails to yahoo email accounts are permanently being binned as is everything else they send - including personal communications.

Like most mail providers, free or otherwise, Yahoo have a spam policy that will look at an inbound email and then drop it in your inbox or spam folder depending on how it is classified.

As with most techies I have about a dozen email addresses at various providers in order to test exactly these sorts of issues. Especially given that the goalposts are changing all the time.

Sure enough even a personally addressed confirmation email was killed as it came into my yahoo account. "Ah ha," said I, "they've been blacklisted". So off one goes and checks the various blacklisting sites and there's nothing there. Hmmm.

It transpires that yahoo have just taken it on themselves to block that domain. Weirdly though, a personally addressed mail to me from the client with only the word "test" in the subject line is still considered Spam yet an email from some random address that doesn't reply, containing several instances each of the words "penis", "cock", "viagra" and "cialis" made it through to my inbox completely unscathed. At this point the phrase about arses and elbows definitely comes to mind.

Trying to get Yahoo to do anything about this issue is similarly problematic as there are no feedback channels to deal with this problem at all.

So overall we've just had to advise people to not use Yahoo or to check their junk mail periodically and read the mail there.

Security 101 : The user should be able to authenticate

2008-03-16T04:21:00.002+11:00

Are you listening Barclays?

I like security - particularly data security and in very particular data security that protects my personal information (unlike a certain Uk government department a few months back).

However, I've been around this game long enough, worked for a bank long enough and built more web applications capturing user data for long enough that I know there is one fundamental truth when it comes to data security and that is: pragmatism.

When I was at Uni I was told, "The only secure system is one that has no network connection, no keyboard or mouse and most of all no users" (and I apologise Dr Fekete for bastardising your phrase but you can't have done a bad job for me to remember it 15 years later!).

However the flip side of all of this was that depending on the data being protected, the security protocol should be appropriate without undue burden placed upon the user. Which is why logging into flickr is trivial but logging into your bank should and is a more arduous affair.

Banks are very secure enviroments which is good because the last thing I want is some 13 year old script kiddie making off with the tens of pounds in my bank account. Having said that, the bank should never make it difficult for me to get to the tens of pounds in my account due to security reasons.

At the moment though banks are running very scared and they are nailing the customers because of it. On my recent trip to Australia I had my card stopped no less than three times because Barclays decided that the activity looked fradulent.

Initially I thought something serious had happened but a call to Barclays got them to right the problem which was part of their new security measures. The next time it happened was because Barclays decided that it was time for me to come home and that I shouldn't be using my card in a Fraud Capital of the world like Sydney. The third time it happened though it locked my account out entirely and I was told I would have to come into a branch with identification documents to sort it all out - except there aren't any in Australia and I was leaving the next day for Hong Kong. Luckily a very understanding parent lent some cash.

I applaud Barclays' sentiments - they really were trying to protect my account, however it would appear as though client / bank trust has disappeared and I can no longer say "I want access to my money globally" without alarm systems going off all over the place. If I was backpacking I'd have been in serious trouble as without a bailout I literally had about 10c in my pocket.

Upon return to the UK Barclays' statement was along the lines of "Sorry but we're dealing with a lot of fraud and it's better to be safe than sorry". Tell this to one of my employees who just had £3K wiped out of their account due to identity theft (spent on local UK products and didn't fire off a single warning) and they are being told they have to prove it wasn't them...

In a way I feel sorry for Barclays because they are damned one way or the other - on this issue though it should just be a case of phoning and doing a vocal authentication then saying "I'm abroad for 4 weeks allow any transactions from xyz country until I say otherwise". In this manner everything other than DDs occuring in my home country should be treated as fraudulent and everything authorised abroad should be fine...

Bring on the chip in my hand is what I say...

DVD Jon strikes again

2008-02-20T12:38:00.002+11:00

At Technology Treason we love DVD Jon or Jon Lech Johansen as he is more commonly known. This great Norwegian famously broke the DVD encryption put in place by the big firms with the release of some software primarily aimed at allowing DVDs to be played on computers and unlocking the regionality of DVDs and DVD players.

When he released DeCSS he ran afoul of the US DMCA and was almost charged, he was then indicted by Norwegian authorities acting on behalf of the US who actually did go to court twice to try and convict him of hacking. Both times they failed and decided not to go to the Supreme Court.

Imagine our complete amusement in the office when we find out he's now trying it on with Apple via iTunes.

iTunes is a love it or hate it product - if you are part of the Apple / Steve Jobs faithful it is obviously the greatest thing on earth, if you know nothing about technology it's a simple product that allows you to use one of those "fangled new digital music type thingies".

If you are a techie you see it as a proprietary lock in and try and avoid it like the plague. The main issue for most techies is you can't play your music on anything other than your PC / Mac that has iTunes installed and your iPod / iPhone / iTouch.

I've railed against lock in for time immemorial - just a quick count of my personal items puts the following music players at my disposal - mobile phone (x2 because my wife has one that can play music too), MP3 capable stereo, PC (x3 - my office, my home and laptop), PSP, Xbox, a real MP3 player and my Nokia Internet Tablet - 10 devices at my personal disposal that I want to play music from and indeed do play music from.

The thing is, I know how to do all of this so I just push the files around on memory cards or over my network (streaming from my media server for example) onto the various devices. For many people this isn't possible and Apple's enforcement of the iTunes lock ins firmly violate the right I have to play my music (or video) on whatever device I choose at whatever time I choose. I also vote with my wallet and don't buy tunes from Apple.

What DVD Jon has done with his software (available from DoubleTwist for free) is allow you to take files that are locked into iTunes and essentially it plays the file, re-encoding it into a format you can play on other devices (I haven't looked properly but presumably OGG or MP3).

Just to rub salt into the wound he's going to cause Apple and the US music industry he's decided to let you share your files with friends as well. One wonders how long it will be before a writ arrive from the RIAA and Apple... I'm sure they'll be racing to get in first.

So well done Jon - keep up the good work and keep fighting the good fight - media we have legitimately purchased is ours to use on any device we own for our personal use.

Eventually the media industry will wake up and realise where they've been going wrong. Perhaps if EMI had taken notice of the way the world was going they wouldn't have had to cull a couple of thousand staff.

Why industries can still be revolutionised on the web

2008-02-12T15:56:00.000+11:00

I'm a bit of a cynic really. Anyone that's trawled through the depths of this blog will know that I have a fairly acid tongue when it comes to technology. I am a walking example of the phrase "familiarity breeds contempt".

One of the projects I've been involved in rececntly has started generating press just by virtue of it being better than anything that has preceded it in this particular industry - I personally would have preferred them to be commenting about the content but any press is good press as they say.

By rights I should have a nice warm fuzzy feeling about having a site people talk about and it's always great to receive recognition for a job well done - especially for my more junior staff who have worked damned hard on the site - however it is disappointing that we still exist in an age online where just applying some good design, good information architecture and some well balanced technology is enough to turn an entire sector on it's head.

Apologists will hold up their hands and say "we're a young form of media - it's going to take time". I however am not in this camp - how much time do we need?

Personally I find it untenable that there are still sites being built using non-standards based HTML and CSS, that sites beyond a couple of holding pages are built using things like Dreamweaver and not content managed, that good structural web design is something that still amazes people rather than being the norm and that information architecture still hasn't found its way to the hearts of 95% of the digital agencies that service the web.

I am constantly lamenting the state of most industries' websites generally. Take a tour around the leisure industry and find a website for a hotel anywhere in the world. Look at most ecommerce sites for even big retailers and certainly go anywhere online in the government, volunteering or political sectors and you are sure to be assaulted by bad design, bad technology and most importantly bad information architecture.

Even five years ago there were excuses that bore merit - changing web standards and platforms, variation of internet connection speeds and different levels of web penetration in different markets. These excuses don't exist any more. And to be honest why was it when I was learning my craft as a developer all those years ago that I was told about things like usability, information design and later information architecture but the junior developers and designers now are not...

This is why there are still industries to revolutionise if you have the contacts, the desire or the contracts to do it. Here is my short list of the biggest problem industries:

1. Tourism and leisure - get some good design and photos, don't use bog stanard templates and for goodness sake stop sending my credit card details in unencrypted email.

2. Holiday / travel booking - get some fuzzy logic in your scripting. If I can't fly tomorrow but I can fly the next day tell me without making me guess. Also make it easy for me to bounce back and forth between different trips without having to start again. Remember all those lectures about how to maintain the state of a system in Computer Science... this is what they were for.

3. Retail - Keep your site updated with accurate stock levels. I also shouldn't have to go to the end of the check out process to find out what the shipping charges are. Do a detection on my regional settings or IP address and take a best guess and say it's a guess. 95% of the time you'll be right and I'll stop having to go back and forth.

4. Service Media - When will you learn that a flash site turns off most people as does a splash page. At least have an alternative HTML site so I can find your phone number / contact email or address. Also remember that table based design was around in 1997 - time to get with the times guys.

5. Volunteering / politics - Yes I know you are on a budget but just because someone you know or your favourite intern just happens to have a copy of dreamweaver doesn't make them a professional web designer or developer. More harm than good is done by casual development - find some budget, find someone aligned to your cause and they'll do it cheaper or for kudos value and develop a site worth looking at.

6. Government - Just because a turd is shiny doen't make it worth anything. Above all make sure someone in the procuring department knows the difference between HTML and CSS and you won't get shafted. Government expenditure online is extortionate for the value achieved. Given the amount of paperwork done for any bit of government work it is amazing that Information Architecture isn't put right to the centre of the brief... how many people using direct.gov.uk would that help?

So get stuck in and lets see some other industries and sectors turned on their head. It's about time the biggest information resource in history got a bit of a spit polish and had all the kinks straightened.

The state of Oz technology

2008-01-31T16:53:00.000+11:00

Well rarely does an entire country entice me to start ranting (and at this point I'll point out I am in fact Australian) but by crikey Australian technology hasn't really moved in the last 5 years.

Now I appreciate this is a sweeping statement and I'll point out that the technology I'm talking about primarily is media based - mobile / web / internet. I have also had the benefit of living in London for the better part of 10 years so I've been at the hub of what is going on.

What I don't understand is why is it that for a nation that was at the forefront of new media ten years ago are we now in a position where nothing has shifted for the last 5. SMS is still massively underutilised and the idea of an SMS shortcode in Australia is a joke - 8 digits is only 2 shorter than a mobile number so is hardly short! Indeed everything to do with mobile is still more expensive, slower and less polished than we are used to in Europe. I went to Vodafone when I got here and asked for a pay as you go sim card for my phone that had pay as you go data on it... I was met with blank stares - Telstra and Optus were both the same.

General Internet access is similarly expensive and slow compared to what we are used to in Europe. Given a relatively modern telecommunications infrastructure, why telcos are flogging the ADSL route instead of fibre / cable begs the question of why so many roads were dug up in the capital cities to facilitate this in the late 80s and early 90s.

What is also interesting is the lack of FOSS out here. Linux is relatively popular but no where like it is in Europe. Indeed corporate America has it's laser telescopic sight firmly trained on the Australian market and even getting Linux hosting is no where as simple as getting a site hosted on a windows server. Linux certification and knowledge is still seen as a specialist skill.

Overall I'm disappointed that Australia hasn't maintained it's lead in internet technologies. In part people like me are to blame for starting our careers here and then being drawn to the brighter lights of the UK and the US where visas are easily come by, pay levels are higher and the ability to work on cutting edge technologies are plentiful.

Perhaps we are on the verge of a change in Australia and I hope that some of the ground lost can be regained over the next five years.

The warm glow of site launch

2008-01-09T09:08:00.000+11:00

I've been in this game a long time but there is still nothing sweeter than launching a site after spending a months building it with your team and the client. As a TD, site launch brings a mix of emotion - fatigue from the lack of sleep for the 10 days prior to launch, relief that the site is launching on time and on budget and the client seems happy with it all and finally worry about whether the thing will work as expected, what will everyone else think about it and by god I hope the server doesn't fall over on Day 1 under load...

My grandfather was an engineer for Philips and he described to me the same feelings when they were launching a new product so I have a sense that irrespective of discipline, team based endeavours in engineering always foster the same heady mix of emotion fuelled by relief, adrenaline and fatigue.

Whilst I am an old hand at this within this industry these days, having been here since the dawning, it is great to watch members of the team for whom this is the first of many site launches in their career and their happiness that it is done and their complete pride in their work.

Having seen photos of workers completing railways and other major constructions in the 19th and early 20th century one can't help notice the parallels of young engineers completing a job regardless of whether they are working with steel, glass or lines of code.

Potent messages of impotent industries

2007-12-22T00:56:00.000+11:00

I should probably know better than to open my mouth but the obvious has to be stated on this one. For anyone that is netSavvy enough to know what BitTorrent is, the news that TorrentSpy has just lost its court case against the MPAA isn't exactly surprising.

Hearking back all the way to Naptser we seem to have an annual tag teaming of court cases brought about by the RIAA and the MPAA in order to bring these "nasty pirate companies" such as TorrentSpy to heel.

Sites documenting the ins and outs of the case are plentiful so I won't go into detail. (For more info see the BBC report as it's quite neutral)

After every one of these cases new technologies spring up to either to protect people's privacy better or make the technology better (Naptser giving way to Kazaa and others which gave way to the BitTorrent protocol).

The recording and movie industries are worried because they are no longer the gate keepers to content and can charge what they like for it. As such the "dirty pirates" must be prosecuted even if they are, as in TorrentSpy's case, nothing more than a pointer to where the content is being held.

The great amusement in this particular case is that the only reason the MPAA "won" in this instance is because of TorrentSpy's refusal to provide the tracker and user data because this was a breach of Dutch Data Protection laws. As such the MPAA won by default.

Had this truly been a court case, it would have come to light that TorrentSpy provide a framework for people to post tracker data about any files they have on their machines and indeed they don't have copies of any of the physical files. The MPAA probably would have still had them closed down but their legal case was always going to be shaky.

So TorrentSpy will be closed, they will be bankrupted but there will be a dozen smaller companies waiting in the wings to see if they can bleed the MPAA that little bit drier.

You see the big problem here is that the MPAA can't let up now. It doesn't have the mechanics in place to distribute online properly (unlike same music where iTunes and others provide the service) not least because of the antiquated territorial boundaries films get sold by.

As such we'll be seeing another legal case next year - maybe ISOHunt will be next - and another company collapsed but then dozens more set up for a brief stab at providing content to the people.

The quote from the MPAA spokesman is great:

"The court's decision... sends a potent message to future defendants that this egregious behaviour will not be tolerated by the judicial system," John Malcolm, the MPAA's executive vice president and director of worldwide anti-piracy operations, said in a statement.

"The sole purpose of TorrentSpy and sites like it is to facilitate and promote the unlawful dissemination of copyrighted content. TorrentSpy is a one-stop shop for copyright infringement."

What's most amusing is that according to many sources, music being downloaded from "official" sources is almost as much as that being downloaded illegally. Surprising how given the tools, a cessation of hostility towards the users and a price point that accurately reflects the product being sold and the consumer comes to the party once again.

The MPAA still has a lot to learn about the Internet - one wonders how much it will cost them in legal fees in the mean time.

My top 5 jQuery seasonal wishes

2007-12-21T02:41:00.000+11:00

I've waxed lyrical about jQuery before, I've been using it a lot to do worker code which I just can't be bothered to hand write any more. Not least because jQuery handles all the little browser inconsistencies for me so the code I actually call into a page is infinitely more maintainable, especially if someone follows behind who maybe isn't so up to speed with JavaScript as I am.

However, use a tool for long enough and closeness breeds contempt as they say. In this vein (and regular readers will know I don't do complimentary very often) and in the spirit of seasonal "Listing programmes" of every style, these would be the top 5 things I'd like to see incorporated into jQuery in the next year.

5. Documentation - Starting off slowly and easily I'd definitely like to see some better documentation. Ideally I'd like to say that new sublibraries aren't included until their documentation is properly up to scratch. Some areas are very well documented other areas are sketchy at best.

4. Wait(msecs, callback) - part of the effects sublibrary, we have all kinds of effects to enable objects to slide, fade and animate but we don't have a wait command. What I would give to have a command that you can just append to a sequence of animations and then wait for a period of time before calling another function or stepping to the next instruction.

As you can see from my jQuery Slideshow the common way to do this is to call animate() with the same instruction as your last step with a callback. It's not big or clever but it does the job.

3. fadeToggle(speed) - again part of the effects sublibrary; we have slideToggle which is a great bit of code, call it and the object either slides open or shut depending on it's state. It would be great to have the same thing with fade rather than writing detection code and then calling fadeIn or fadeOut.

2. State detection - Another worker function would be really useful here to actually determine the state of an object as to whether it is on or off in display terms. I am fully aware I can use document.getElementById(objname).style.display or equally $().css.display() however this will return "none" if it's off, but it could also return "block inline table table-cell list" etc depending on what it is.

Ideally I'd like $().displayState() and it would return "on or off" or indeed true or false as a boolean so it would make display code even easier logic wise.

And finally,

1. Cast to DOM object. One of the best things about jQuery is it's query language. Using elements from the CSS and Xpath specifications pulling objects out of the document is so much easier than using DOM traversal methods.

However sometimes the jQuery functions just aren't enough and we need to cast an object to real JavaScript to play with it - a simple method of doing this would mean the power of a great interrogation language along with the ability to cast to a real DOM object.

I fully expect someone to come kick me now telling me I can do some or all of these things and indeed the functions I'm asking for exist aleady however the documentation as mentioned in number 5 is lacking in some areas so it isn't obvious if it is doable.

Obviously this is a little toungue-in-cheek as if I was that worried about these issues I'd write the code myself and submit it to the team for inclusion in the next version. Indeed perhaps that could form the basis of one of my New Year's technology resolutions.

Happy Holidays all.

SMS Bamboozlement...

2007-12-19T21:33:00.000+11:00

I'm doing some work for a client at the moment who's industry is particularly technophobic. The absolute cutting edge is a bit of YouTube video thrown willy nilly into a page. I'd also point out that design is something that rarely makes an appearance in this particular industry.

So it was pretty refreshing when we went to them with a series of ideas from the more commercial sectors of New Media and one of the things they latched onto was SMS. Queue annoyance though when we had already got everything ready to go other than to push the big green "launch" button and another company got involved and started talking about location aware services and high end data capture etc.

At this point the client dissolved into a mess of indecision - "Why weren't we doing all of this?" was the question, to which the answer was "Because you don't need to - primarily because your text messaging service is built around raising revenue through donations!"

I've had this happen in the past, notably with SEO companies. I do pity the poor clients who get stuck in these situations where they've finally decided to push their technology base along but then get waylaid by all the glittery, flashing and hypnotic LEDs.

At the end of the day it is important to remember why you are doing something and not get sidetracked (and not get ripped off). Once a strong foundation of technology is laid there is always something new you can build - you don't have to have every shiny present under the tree to have a great christmas.

.NET / XSLT and how to import an external XML document

2007-12-11T04:45:00.000+11:00

I work with XML and XSLT every day of the week. Indeed working for a company called XML Infinity you can imagine how much we use it. I had one of those incredibly frustrating moments this afternoon that one typically when dealing with badly documented parts of .NET or XSLT.

The annoyance in question was to do with loading a document in to an XSL template on the fly. 99.9% of the time you don't bother with this as you have a master XML document which you transform according to the XSL template that is assigned to it. All your XML processing is usually done before you get to this point.

There is an xsl function though called document() which you can use to load in an external XML doc to the XSL template and then do work on it. I've used this before but the damn thing wouldn't work. Why not? Because our Transformation Engine wasn't using a loose enough resolver to be able to deal with externally referenced files... grrr. I know why MS did this because it's so the parsing engine doesn't go loading every document under the sun and potentially crashing.

That's great but they could have documented it a bit better.

The resolution by the way is to create an XmlUrlResolver, give it some credentials (in my case setting it to DefaultCredentials which allows you to access http::, file:: and https:: protocols) and then pass that into your Transform() method.

Job done.

Not quite.

Having finally been given access to an external XML document I then had to contend with XSL's arcane methods of dealing with XML fragments. Again documentation was the issue here.

Looking online there are some ridiculously complex ways of parsing an external document when by rights it should be as simple as just dropping the doc in a variable and then processing according to the variable. People were using recursive templates using xsl:copy and all kinds of things.

Turns out the way to do it is a little known second parameter.

If you do this:

<xsl:variable name="var1" select="document('http://example.com/file.xml')"/>

All you'll end up with is the text nodes. Not very useful.

If you do this, however (note the second parameter):

<xsl:variable name="var1" select="document('http://example.com/file.xml', /)"/>

You'll end up with a full fledged XML document complete with nodes and everything put into your $var1 variable and you can then use it to select data according to standard XPATH constructs.

If you don't want the whole document you can pass the second argument as an XPATH query and it will just return that nodeset - much easier to deal with.

In all the time I've been dealing with XML / XSL I didn't know about this and it was a great pain to figure out. Typically the only reason I was doing this was to mock something up for a client quickly and it then turned into a mammoth effort. Knowing now though will save time subsequently I guess...

PCI DSS will wreak havoc on SMEs

2007-12-01T00:15:00.000+11:00

One of my clients was asking me about PCI DSS certification today. Coincidentally I also received our letter about compulsory compliance to the PCI DSS standard.

Both of us are what are termed "Level 4 Merchants" - that is we process less than 20,000 card transactions through the company in a year. Arguably Level 4 Merchants will probably account for the largest number of business globally as they will incorporate pretty much every SME in PCI compliant countries that takes a card as a form of payment (according to Visa about 27 million businesses).

The standard itself is a worthy document - a dozen set in stone compliancy rules to which businesses have to adhere. Most of it is common sense like settin your password on your router to something non-default, make sure card details are encrypted if they are to be stored, that sort of thing. Most businesses in the SME world would, in fact, actually be compliant - mostly because they don't store data.

Here's the rub though. Barclaycard sent both my client and I a letter basically saying you have two options on compliance: First you do it yourself or otherwise you get someone to help you (and of course they recommend a company SecurityMetrics to help you do it all - at a discounted rate of course).

Obviously the first thing I did was go to the security metrics site and request a quote. As a Level 4 Merchant it will cost me merely $699 per year to be assessed quarterly. However they can tell me do do things to get me up to spec which is then going to cost me more again. At the end of it they give me a pass or fail certification and their audit is completely subjective.

After that I went and downloaded the whole specification and read it through twice. Every point I made a note against.

Typically, this isn't a document for the feint of heart. I'm lucky first in that I'm a techie and second that I did my formative programming years in a bank specialising in what was then the forerunner of InfoSec. There is not a single line of "plain english" in the whole thing.

A couple of non-techies I've shown it to got about a page in before giving up. Your average 1-5 employee company owner doesn't have a hope. Thus he'll end up paying $699 per year for what is essentially insurance.

Even amongst Level 1 Merchants, understanding and compliance are two different things as you can see on Evan Schuman's great article about recent stats to come out of the Level 1 camp.

Big companies have the resources to deal with this sort of stuff and they are also more likely to be saving data on customers so for them it is crucial. Whilst no less crucial for small businesses, the fact that a store owner who only takes card payments for people when they are physically in his shop will still have to go through this audit is patently ridiculous.

BarclayCard are indemnifying themselves by playing the FUD card with comments like:

To date these penalties have not been passed on to any Level 4 Merchants, but from 30th April 2008 your business will be liable for PCI DSS penalty charges and costs associated if you fail to comply or have a data compromise.

Penalty charges can be considerable (in excess of £100,000) so, to protect your business, it is vital that your prepare for PCI DSS compliance by 30th April 2008 and continue to maintain compliance in the future.

What the PCI DSS standard fails to deal with however is systematic failure of employee behaviour. It doesn't deal with issues such as people skimming cards if they are taken out of sight nor does it deal with employees writing details down on a piece of paper and passing them on when dealing with mail order, nor does it deal with phishing scams.

Indeed I had a card machine problem last week and the support officer at BarclayCard stated:

Just write the details down on a piece of paper and process them later

Hardly a piece of advice that should be followed to maintain security.

In the end businesses will have to make their own mind up about how to best deal with this new "virtual legislation" that is being thrust upon us. To me the whole thing reeks of the rise of the SEO industry piggybacking off Google's search technology.

In reality the biggest source of credit card fraud is that caused by skimming details through offline processes such as mail order (which I had done to me recently and my bank caught it on the other end within a day) or else identity theft whereby a new card is created in someone else's name.

None of the procedures outlined by the PCI DSS standard deal with these very real and growing issues - all they are doing are lining the pockets of consultant sharks that will feed on the SMEs who don't know any better and penalising the merchants for actually trying to conduct business.

Adding Cron Jobs to a QNAP server

2007-11-27T00:53:00.000+11:00

If you haven't come across them yet QNAP make these amazing little NAS boxes that are perfect for home or SME use. I've got mine running as a home server but might get one for the office as our old server is on it's last legs and a fully tricked out 1U dell server is a bit of overkill for a glorified file server.

The best thing about these devices though it that they run Linux OS utilising Debian Essential and as such they can be configured to do almost anything you want. Out of the box they already come with file serving, media serving, database and web servers.

One slight problem though is that the boot up process is not disimilar to that of a live CD. This is great in that it makes the system highly robust and it boots to a known state each time. The problem is that short of rewriting the firmware you can't introduce things into the boot process. What I don't want to do is have to re-run a load of scripts to configure the server how I want it after a power failure or forced reboot.

The boys over on the QNAP forums are really on the case and one of the chaps has created a nice little framework script which hooks into the boot process and allows the execution of a series of scripts. You can see his work here.

After installing this workaround you can add scripts to the scripts folder and take control of your server.

One of the things I wanted to do was add items to my cron list and this process is explained below.

1. SSH into your QNAP box

2. Install the custom scripts files at http://www.qnap.box.cx/ as per the directions there.

3. CD to your scripts directory in custom and make a file called joblist.txt in VI (Vi is the only editor you have on the QNAP drive).

# vi joblist.txt

When in vi make your list of cron jobs using the standard CRON syntax.

Mine was the following:

25 1 * * * /share/backup/script.sh

This will run a backup script I had written at 1:25am everyday. You can add as many or as few as you want. Save your document and exit from Vi.

4. Make your script that will fire on start up. I called mine cron_update.sh

# vi cron_update.sh

In there put the following code:

#!/bin/sh
# this script apprends a job list to the existing crontab
echo "Reconfigure CRON list:"
cronpath=/share/MD0_DATA/custom/scripts
#list the cron tab and put in a temp file
crontab -l > $cronpath/cron_jobs.txt
#append the items we want to the master cron jobs list
cat $cronpath/joblist.txt >> $cronpath/cron_jobs.txt
# replace the existing crontab with the new one
crontab $cronpath/cron_jobs.txt

Save and quite out of Vi.

You'll notice I've used a variable in here to specify where to find the files. This is because the autorunmaster script is a folder higher so we need to be explicit about where to find things.

5. Go back up a directory to your custom folder. In there edit your autorunmaster.sh file with vi.

# vi autorunmaster.sh

At the end of the file append:

/share/MD0_data/custom/scripts/cron_update.sh

Then save and close the file.

Now when you reboot you should have your newly added cron jobs appended to the crontab without removing all the old ones.

Why was data being passed on a disc and what was EDS advice?

2007-11-21T20:14:00.000+11:00

Readers in the UK will be aware of a Data Protection train crash that we have been watching unfold in front of us over the last few days. It turns out that 25 million records of a database managed by HMRC have been lost in the post because they were sent on a couple of disks using unrecorded mail.

There has been much speculation about which minister to blame and who in the cabinet (including the Prime Minister) should lose their job but one thing that is mostly missing is the notion of data security.

In the UK we have the Data Protection Act - policies enshrined in law to which I am constantly referring when talking to my clients. A typical day for me usually includes quoting something from the DPA at least once. Not least because a client wants to harvest user data and use it for something else that is outside the bounds of what is technically legal.

I've done a lot of work for government and I have to say in my experience they have terrible technical practices. Gone are the days of locked down machines with no floppy drives and only CD-Rs. In are mass market units from Dell with the latest in CD/DVD-RW (because they are cheap and mass produced) along with USB connectors that people can hot plug a pen drive into and download whatever they like. The current government has a woeful record on technology projects mostly because they don't understand it and they contract suppliers who talk a good presentation rather than deliver an effective solution.

According to the DPA

"Appropriate technical and organisational measures shall be taken against unauthorised or unlawful processing of personal data and against accidental loss or destruction of, or damage to, personal data."

This is why our PM said procedures weren't followed and he is bang on the money there. This relaxed attitude to data, particularly sensitive data, has been demonstrated in this debacle. If the data was going to be put on disc why wasn't it fully encrypted?

Indeed, why wasn't there a secure online facility for user data to be interrogated without recourse to physical copies to begin with?

In addition the data was supposed to have been "desensitised" before sending - a quaint term meaning removal of things like bank details, exact personal date and full address information. To do this EDS wanted to charge money for it. The department didn't want to pay so they took the lot.

EDS are complicit in this as much as the people from HMRC are. How hard is it to type into the database "Select name, age, postcode from person where...." instead of "Select * from person where..." Or else just remove the columns that were sensitive on output. It would have taken me a few minutes so it can't have taken an experienced EDS engineer that long.

EDS shouldn't have been charging for that sort of difference - but it sounds more complex so it was an opportunity to get some more cash in - probably.

Further EDS should have been saying "We advise you that the data you are requesting is excessive for the purposes of what you are going to use it for so we'll give you a more secure subset". That would have rammed home the implications of what the staff at HMRC were asking for.

In my history of working with government I have come across this sort of situation many times before. It is well known that government contractors over charge, shaking the fruit out of the infinitely laden money tree whenever they can. Our E-Minister is supposed to deal with this sort of thing but in practice he's a politician who knows as much about IT as my mum. The only way to resolve this problem is for wholesale changes to occur within government (locking down machines) and to make stiffer penalties the punishment for breaches of the DPA.

We now have a situation where 25 million adults in the UK are worried that their personal details are going to be used in some sort of mass identity fraud.

My view is pragmatic in that the CDs are propably laying in the corner of a sorting office at TNT somewhere - but they could well be in some gangster's tech lab being processed and that is the point of all this security.

Fuzzy logic could book more flights

2007-11-19T21:43:00.000+11:00

I've talked about fuzzy logic for use by the retail sector in the past and the project I'm involved in there is maturing nicely. This week I've really realised how as software engineers we need to grasp the nettle and move a lot of service based software toward fuzzy systems for usability reasons.

Nearly everyone these days has booked a flight online and when it came time to booking a holiday to Australia this winter, the first thing I did was fire up a browser and head to expedia and travelocity.

If I was planning to fly on specific dates I would be well catered for and I could get a list of prices and book a flight in a few easy steps.

I wasn't planning on flying on a specific date though. I work for myself so can take time off whenever I want in a general sense. Really what I wanted was the cheapest flight from London to Sydney in December.

After typing a few different dates in manually I did the sensible thing and called a human travel agent who was very helpful. Unfortunately, as helpful as she was, she only had access to the same systems I did so couldn't tell me the info I needed to know. Mentioning this to friends had the usual "you can't do that" response. Can't do it?! I'm the customer I can book when I want.

All airlines operate through the SABRE booking network which is basically a masive database of flights from point to point with availability and prices per leg on it. It sits on top of a nice mature API which makes it easy to program against and so that's where the developers leave it.

But as a customer this doesn't fulfill my requirements and this is where engineers need to spend more time thinking fuzzy.

In these days of multi-processor and multi-threaded OSes it is not that difficult to build offline agents that could go and find this information out for a customer and then email it back to them. Indeed I wouldn't mind registering to use this sort of service so now the company has my personal details and they can market to me.

The agent wouldn't even need to respond with all the availability. It could just give me the cheapest 10 or 20, all from a specific operator etc or those flights routing through Hong Kong as a stop over for example. It also doesn't need to be fast. A deprioritised thread could take a day to get this sort of information and if I'm being that vague then time is hardly an issue.

If someone reads this from the travel industry please ask your techies to build this feature. If you are a venture capitalist then give me a call and we can revolutionise the online travel sector!

The web has brought us an always on, on-demand, serviced-based method of interacting with our information but the casuality of this has been flexibility. The days of fuzzification are soon to be upon us and coupled with automated agents some amazing new systems will become available that will give us back our flexibility.

Why can't I have $100 laptop

2007-11-08T21:51:00.000+11:00

Don't you hate it when you can't get something you'd really like?

I've been following the OLPC project more or less since its inception. When I first heard about it I was mostly interested in how they were going to pull off building a laptop for only $100 per unit.

After realising they were going to do it I was interested in how useful the machine would actually be (it has no hard drive so it can't be that great right?).

After seeing it was running Linux and was designed to be wireless from the start, run on mains or able to wind it up to power the laptop and it was designed to be durable in harsh environments I was mostly interested in how I could lay my hands on one (or two even).

My disappointment was immense when the OLPC guys decided not to offer them for sale, and then when they u-turned and started the G1G1 initiative (Give One Get One) I had a momentary blip of joy until they said it would only be available in North America.

Why they've not rolled this out to Europe is beyond my comprehesion - I don't even care if I don't have a £ key - I can always map it to a key stroke anyway. And I'd even be happy to Give 2 Get 1 if shipping was the issue.

The other thing that amazes me is that given the connectivity of these laptops Western nations aren't falling over themselves to get them for schools - even if they had to pay a higher rate along the lines of the G1G1 programme it would still be cheaper than buying Dell machines into all the schools.

Bye bye OpenMoko

2007-11-05T21:26:00.000+11:00

Google announced today that they would be partnering up with a load of other companies including Samsung, Motorola and LG to produce a new phone "software stack". For those of us in teh technology game this basically means Google plans to release mobile phone operating system to rival that of Microsoft, Symbian and the various Linux flavours out there already.

What I find most annoying about this is that Google has for years now feasted upon the fruits of the Open Source Community, using many of their projects to enable additional features and indeed their core search facilities to work. While it may be argued that the Summer of Code gives back to that community, there is a sense that rather than sponsoring an existing project like openMoko (a Linux based, open source version of what Google has announced) they've decided to go out on their own and start from scratch.

Given Google's tremendous resources it won't be long before we see the platform hit the market.

Within the commercial market there is already Maemo (nokia's Internet Tablet platform which they actually open sourced) and QTopia, a commercial package available on the GreenPhone which is a development kit and is mostly open source too.

My guess as to why Google didn't run with any of these options is that there are already thriving communities surrounding them and trying to work with these existing communities makes it difficult for the Google techies to throw their weight around.

Hey ho. As a developer, mobile development is already a nightmare having to support various versions of Symbian, MS Windows Mobile, BlackBerry as well as smaller (but vocal) numbers os Maemo users we are now having to think about iPhone from Apple so adding "Google Phone OS" isn't that much more work.

For me, having had a mobile phone for the better part of 15 years and having had a data capable phone for nearly 10 years I've watches OSes come and go, killer apps be talked about every 6 months and watching the market mature the only two things ever to take off properly on a mobile was SMS and now e-mail.

I've got an E65 nokia and it is the best phone I've ever owned. Why? Because the web browser works seamlessly on standard web sites and the email is easy to use, even without a full keyboard. Oh, and it doesn't crash as do most of the rest.

Spending all this time and money in my opinion by Google is absolute folly, but then they have virtually limitless cash reserves and they have a staff of many thousands across the world that they have to retain doing something - they may as well be making a phone OS as anything else.

Who knows this might end speculation that we are about to have Google OS on our desktop next year as well.

CSS Structure - what a mess

2007-11-03T04:29:00.001+11:00

James posted a message on my blog some weeks ago and it's only now that a penny has dropped in my mind about what we need to deal with the issue of structure in CSS - the problem is we have none. As James points out you end up with a flat mess that with all the best will in the world definitions are hard to find.

I've ranted before about the annoyances of CSS - particularly to do with the lack of variables or constant definitions without recourse to server side scripting and about the nature of the W3C CSS working group not being well represented by techies - especially as CSS is nearly a language in its own right the same way Regular Expressions are.

As the web building fraternity finally weans itself off of dreamweaver and table based design and adopts a more semantic, HTML-lite way of building sites, the CSS files are getting bigger and bigger all the time.

At the moment, to get a degree of specificity one has to redeclare selectors:

div#header div.nav ul {}
div#header div#logo img {}

for example.

Many CSS zealots would say "But you can get rid of div#header altogether" and I can in this instance but what happens if my div#logo doesn't appear in div#header on a page and certainly it's not uncommon to have navigation in a header as well as a sidebar.

As can be seen, in order to get specificity we increase verbosity. Anyone that is fully converted to CSS design will tell you this, it's the casual "div stackers" who just declare a new class for every element in the document which ruins the HTML.

My solution then W3C if you're listening is this. Cascade in the style sheet, cascade in the CSS file.

In many programming languages there is a keyword to get you down to the level of an object that you are going to manipluate numerous properties of in one go for example "with" in VB. Thus I could say:

with myobject
.property1 = x;
.property2 = y;
endwith

The CSS equivalent would be:


div#header {
  div#logo img { css }
  div#nav {
    css
    ul { css }
  }
}

This gives you the specificity required, removing the redundancy and creates a cascade like structure to the document that would also make things much easier to debug what is going on.

Structural CSS along with variables would make a massive contribution to the developmental side of CSS as a language that could revolutionise the way we use CSS with the web.

FAH goes number 1 but we could do better

2007-11-02T22:44:00.000+11:00

Folding at home (FAH) has taken the Guiness World Record for being the most powerful distributed computing network with a top speed of over 1 petaflop - (a thousand trillion calculations per second).

This is a remarkable achievement and shows the immense power that can be brought to bear by spare computing power used in a distributed network. The key here though is massive parallelism which means the various nodes in the network (your PC or PS3) are all doing different jobs at the same time and are at various points through these jobs. This is what made FAH and the old title holder Seti at Home (a search for extraterrestrial life) so scaleable.

Individual computers on the network download work units from the central repository, process them individually and then resubmit them back to the central core for post processing.

This is in contrast to say the Earth Simulator of Japan, a massive supercomputer capable of running huge simulations with ridiculous numbers of variables and calculations very quickly but where everything is interdependent. Likewise the ultimate aim of the BLUE project from IBM and the US Department of Energy is to be able to simulate all the forces and atoms of a nuclear explosion to simulate what's happening to USA's aging atomic weapons stockpile as they are no longer allowed to perform live tests.

This doesn't take anything away from their achievement, however it does go to show just how much wasted processing capacity there is lying around on the network.

The FAH project ramped up from 250 Teraflops (trillions of instructions per second) to just over a petaflop by the introduction of 670,000 PS3 owners supplying their hardware, up from the 200,000 PC users who got it to 250 Teraflops. Given that there are over 6 million PS3s in the wild this represents about 10% of the total Ps3 userbase - a quick calculation indicates that PS3 owners alone, should they all connect up to the internet, could provide about 7.5 Petaflops of processing power... this is beore we take into account PCs, XBoxes and Nintendo Wiis.

What this illustrates to me is that many of these projects are limited by their publicity and how "glamourous" they are. Taking nothing away from the geekiness of searching for ET or the importance of seeing how protein folding will affect drug development in the future, a more elegent solution would be an open framework that users subscribe to which is then used by anyone who wants to create a distributed processing application.

For the end user it is seamless and the for the multitude of public projects requiring raw processing cycles it gives them to opportunity to get access to larger numbers than their marketing budget would otherwise provide for. Even private companies could pay to rent processing time thus investing funds back into the project for ongoing development or optimisation.

Let the new gaming witch hunt begin

2007-10-15T19:37:00.000+10:00

I'm in my thirties now and I've been playing computer games from the age of about four when my dad first brought home the venerable Vic 20 - partly because he was doing a computer science degree at University but mostly because he wanted to tinker.

Through my life I've borne witness to the rise of computer gaming as a media format to rival and now surpass film and I've seen countless witch hunts focus on computer games as being the root of many of society's teenage evils - everything from being the cause of the obesity epidemic to turning children into cold blooded murderers and violent criminals.

I'd like to think that I'm pretty normal - whilst I have my own individual quirks as everyone does, psychological assessments that I've taken for a couple of employers have branded me pretty average on the whole "serial killer" metric. And even though I'm now suffering from the onset of a bit of "middle-agd-spread", as a teenager and child I was pretty skinny.

In thirty years of gaming I'd say I'm "above average" in terms of the amount of time I spent gaming. I wouldn't have hit "compulsive" but as a kid I'd spend a good hour or two a day playing on the computer. Conversely though I'd spend an hour or two playing outside per day though the key factor was that I watched virtually no TV.

You see my parents had a rule in our house - TV or Computer but not both. When my mum thought we had been spending a bit too much time in front of either she'd pull the plug out of the wall and summarily kick us out the door with the instructions that "it was a nice day - go enjoy it" - this held true even if it was raining or the middle of winter!

So, in what seems like a biennial event another review of gaming has been started - the Byron Review this time is being headed up by the very smart Dr Tanya Byron - an expert in Child Behaviour (and TV personality to add some celebrity to the proceedings). Whilst the review is supposed to cover the full range of technology, Gaming and the Internet are always the first things to crop up as being responsible for the decline of morality amongst our youth.

What won't be taken into account properly though in my opinion is how the role of the parent has changed in relation to these technologies. My parents looking back on it were pretty good (though I know I didn't think it at the time) in policing our internet and gaming activities (our family had access to the internet through a BBS at my dad's Uni).

The modern parent has completely divested themselves of any responsibility for policing their childs' activities. This isn't just limited to gaming and the internet but is a wider social epidemic we are starting to see the symptoms of - everything from anti-social behaviour to academic performance.

I know of adults who have bought games for their children aged under 10 that are clearly marked as being 18 certified. All because of pester power and the guilt that they have over not seeing their child because they have to go off and work all day. What scares me is the "oh well" attitude of these parents - and the fact that because the console is in their kid's bedroom they don't see the actual content themselves. For me games were played in the living room in full view of the rest of the house.

In the face of this blatant irresponsibilty from parents, what can the games industry do? They've created a product they have submitted to the classification board, risking censorship and potentially loss through narrowing their market but then the parents ignore it and go buy the game for their child anyway.

After the fact, parents are the ones calling for tougher regulation and a realignment of the game makers moral compass when it comes to producing the content but it is their failure and own moral ambiguity that has caused the problem in the first place.

For all the public outcries about video game related violence and exposure to sexual content, there is deafening silence regarding the lack of parenting skills to avoid exactly this situation. My parents could do it as could those of my friends - how have we lost that skill in a single generation?

JQuery Slideshow

2007-10-14T17:26:00.000+10:00

It seems JQuery is definitely gaining some traction as a useful library - not least because of the development of the ThickBox Gallery library by Cody Lindley which is seeing huge amounts of use around the web at the moment as a means for displaying galleries for product or photos without being constrained by the page template you are building for and by maintaining the semantic integrity of the HTML you have put into the page. The last cool feature is that you don't have to use the dreaded pop up which brings into play the whole pop-up-blocker issues.

It seems redundant to talk about the ThickBox stuff other than to say it's a great bit of kit and well worth checking out if you need gallery display functionality, I've got my own little bit of JQuery code to document here.

This came about due to a client wanting a gallery then not wanting a gallery because they didn't want to maintain all the thumbnails etc and so it evolved into a "slideshow". They didn't want to use flash due to the cost, but they were already using JQuery for other parts of their site anyway. As such I decided to have a go with building a JQuery slideshow with the animation API.

For this example I'm assuming some degree of javascript familiarity so I can get to the guts of the code.

Obviously you'll need the JQuery library - I'm using the current 1.2.1 version that is compressed so it's a light download.

Next up we need a page with an image in it with an an id called "bigimage".

We also need some javascript to set up an array with the image names in it that we want to load so let's do that:

var imagearray = new array("image1.jpg", "image2.jpg", image3.jpg");We need to trap the moment the document becomes ready to work with so we set up the special document ready function:

$(document).ready(function(){
// now we get the image and attach an onload function to it.
$("#bigimage").css({opacity: 0});
var theimage = document.getElementById("bigimage");
addEvent(theimage, 'load', anim, false);
});

What this function does is set the opacity of the image to 0 (ie invisible) then we get a reference to it in standard javascript and finally attach an event to it which fires on the onLoad event for the image (more about this in a minute).

The addEvent function is given below and is a worker function to add an event handler for a particular object.

function addEvent(elm, evType, fn, useCapture) {
if (elm.addEventListener) {
elm.addEventListener(evType, fn, useCapture);
return true;
} else if (elm.attachEvent) {
var r = elm.attachEvent('on' + evType, fn);
return r;
} else {
elm['on' + evType] = fn;
}
}

Why do we want to add an event for the onLoad of the image? The answer to this lies in how we want to do the animation. Potentially we could have hundreds of images in an array. This slideshow fades an image in, displays it for several seconds, fades out, loads the next image and starts again.

By trapping the onLoad event of the image we can use this event to start the animation sequence which finished with an instruction to load the next image. Only once the image is fully loaded does the sequence begin again.

So our Document Ready method sets up the onLoad event handler, anim() which is listed below:

function anim() {
$("#theimage")
.animate({opacity: 1.0}, 1500)
.animate({opacity: 1.0}, 5000)
.animate({opacity: 0}, 1500, "linear", animNext);
}

This function is called every time a new image has finished loading, bringing the image from 0 opacity to 100% over a 1500 msec interval. Next it holds the opacity at 100% for 5 seconds and finally fades out over 1.5 seconds after which is calls the function animNext().

animNext is a function that deals with determining the next image in the sequence (in my case, wrapping back to the start if we get to the end) and then displaying it purely by changing bigimage's SRC property. This is pretty straightforward JavaScript so I'll leave it for the reader to do.

The key thing here is that by adding an event handler onto a low level object in the document along with a couple of animation commands a reasonable slideshow effect was created which works well for the users and was good for the client as it is maintainable and didn't cost a huge sum as it would have done in flash.

It's the ability of JQuery to expose enough variety of basic features to allow you to do this very quickly and easily. I have no doubt that after 10 years of writing javascript that I'd be able to do this all by hand. The questions are "Do I want to?" and "Is it good value for the client if I do?" - my answer to both of these is "not on your nelly".

JQuery saves the day?

2007-08-25T21:28:00.000+10:00

If you haven't come across it yet there is a javascript library called JQuery which is being developed as an open source project, designed to give us better control over our web pages and the things we can do with them.

Thankfully John Resig, Karl Sedburg and the others have steered slightly away from the profligacy of AJAX libraries doing the rounds at the moment and produced a library that actually deals with some of the problems you face as a web developer or a designer - namely things like clients saying "I'd really like the first paragraph after each header to be blue instead of black".

Now before I get shot down in a burst of "you can do that using classes in your p-tags" I'll say this - I don't want to, I shouldn't have to and it makes for ugly and unmaintainable code. Doing this just papers over the gaping holes left in CSS and makes your HTML even less semantic than it already is.

This is where jQuery comes in. The biggest area of development in this library has been in developing "content selectors" similar to the CSS selector specification. The brilliant thing about these selectors is that we don't have to wait until browsers with CSS 3 in them turn up before we can use them - thus saving us about 5-6 years of waiting time.

I'm a fan of Javascript in small doses - I'm not a fan of large scale AJAX where it is pointless to be loading information that you can get on a click anyway, 99% of my clients want to ensure accessibility and often Javascript breaks that. On a UI where responsiveness is key then AJAX is 100% appropriate but for the majority of sites it's a gimmick.

However in this context we have a javascript library that can add depth to the interface and add consistency and markers that could only be achieved by a lot of proprietary hacks. This benefits usability without sacrificing accessibility and portability. If JavaScript is switched off you lose nothing that wasn't there before anyway; if it is then you get a whole lot more texture to the site.

Watch this space as I think there will be a lot of development on this library over the next 12 months.

Is £180 good value for Wii Sports?

2007-07-03T18:07:00.000+10:00

I am definitely a Nintendo fan boy. I've had every Nintendo console released on the market plus so many Game & Watches it's not funny. Call me sentimental but Nintendo has been a part of and is one of the definers of my life.

So obviously when the Wii came out there was no question I was going to get one. The new controllers are awesome and just show you what a massive difference can be made in computer-human interaction by adding a couple of extra components and removing some wires.

But for some reason this time I didn't get one on launch or near to launch date. The reason? Because I didn't see any games I'd play. In the end it wasn't until June that I finally went and got one. The reason? My wife said she wanted to play Wii Sports! This obviously made it easier to add another console under the TV in the lounge room so in it went.

Now if you haven't played it, Wii Sports ships with the console so in that regard it is free. In it there are a series of very well built games ostensibly to demonstrate the different ways the controller can be used. Tennis, Golf, Baseball, Ten Pin and Boxing all make an appearance.

My other half and child love it. For my wife it is a change of all the things she hates about computer games (pushing buttons that bear no relation to the action on screen) and my child (who is nearly three) just likes swinging the controller and playing with mum and dad.

But something inside of me just isn't loving my Wii. I walk into GAME and check out all the releases and there's nothing I want badly enough to part with £40 for. I got Zelda more because I thought I should rather than because I thought it was amazing and it is a fantastic piece of software engineering but it has the feel of a MMORPG grind about it so whilst I've nearly finished it now it has left me tepid in a way that Ocarina of Time never did.

At present the Wii is the most sold "next gen" console and regular "sold out" signs at GAME down the road suggest they are still selling like hot cakes.

Back to my original question though which is is £180 good value for Wii Sports.

Personally I'm more likely to play a bit of Wii Tennis when I get home than watch TV. In real terms my wife and I have played a good dozen hours of Wii Sports each (mostly together) otherwise we'd have gone to the cinema or something which would be £15 a ticket for 2 hours. That alone is about £180 by itself and when you pass the controller over to a neice or nephew or child of a friend who hasn't got one you do get that warm techie buzz about someone "getting it" for the first time and I think that is probably worth £180 any day of the week.

DRMed for Life

2007-05-03T23:49:00.000+10:00

In the news recently has been the whole thing about not only the copy protection on HD-DVD and Blu-Ray disks being cracked but people posting digg links with decryption keys in them. I can understand Digg's position in removing said posts until the community kicked off and they then decided they'll go down with the ship if they got prosecuted. Hurrah for someone over there seeing sense.

One part of my brain always goes "Hooray for the hackers" whenever we hear stories about DRM being hacked in whatever guise it has been created. Another part of my brain, probably the more rational side I guess, does kick in afterwards and say that putting these things out in the wild will enable more software / media piracy and will incur costs for the companies that produce it which will make them either raise costs or step up counter-piracy methods. I never get to the "woe is me" stage like most media company execs do as they are truly multi-billion dollar organisations so it's hardly going to come out of the mail boy's pay cheque and they are unlikely to go bust.

What I do question properly though is the rationale that got us here in the first place. Since the 60s with tape-to-tape reels starting to replace vinyl records, music, film and software piracy has got bigger and bigger. What has happened though is nothing short of an arms race. Consistent through this entire arms race have been three key points:

1. That me, or anyone else, once they have bought a product has the right to play or use it for their own personal enjoyment whenever they see fit. This is the argument that most consumers will play - I might by a CD album but I want to play it on my MP3 player. I might buy a DVD but I want to play it on my Linux laptop as well as on my TV.

2. Companies that produce consumable media assume that anyone that wants to copy a product is inherently up to no good and they are now labelled as pirates and are probably taking the music / film / software and selling it in backstreet market stalls.

3. The profligacy of piracy is directly related to the first two points and how policed piracy is within the community.

One can directly see that paid for knock off copies of movies and music is completely against the law as you are selling someone's work and is tantamount to counterfeiting. However the framework for dealing with these people exists within the law and we are starting to see this go down.

The media companies will tell you that it's because of their anti-copy protection, however in reality it is because of better policing and it being viewed as being a black market operation and it having been historically a move away from "hard crimes" that has occured over the last 20 years.

This argument doesn't wash at all with consumers. Once I purchase a peice of media it is mine to use how I want on whatever device I want.

The barriers that are being put up by the media companies in their zero tolerance to consumers is assuring their position as the "big bad ogre" in all of this. Were they to engage with the consumers who are most likely to want to move content from one form to another they would probably be able to reach a solution.

Indeed were they to strip all DRM from their content altogether and then spend the money on producing better content or else supporting better policing they would probably turn a larger profit.

In the words of Nixon, "I am not a crook" - but I do want to watch Spiderman 3 when it comes out possibly on my TV from my XBOX, on my Linux Laptop, Windows Media centre and my PDA. At the moment I'll be lucky if one of those four work so I probably won't buy it at all.

super computer required to simulate half a mouse brain

2007-05-01T05:15:00.000+10:00

Scientists have published that the've used the IBM Blue Gene L supercomputer to simulate half of a typical mouse's brain. More accurately they've simulated about half the neurons and just over half the number of synaptic connections for 10 seconds - which because the simulation was running at about a tenth of normal speed showed about 1 second's worth of realtime information.

You can read the whole story here

Don't get me wrong, the guys at Navada Uni have my utmost respect. I studied a lot about cognition and neural networks when I was at Uni, in fact I specialised in it with a degree in Computer Science and Psychology so I know how hard this is to do.

What gets me going though is the reasons behind doing it. As can be seen here, the top Supercomputer in the world can be brought to its knees by modelling half a mouse brain for a very limited period of time. The reason for this is the sheer number of connections [synapses] that occur between neurons - a single neuron in a mouse can influence the behaviour of about 8,000 other neurons. It doesn't take long for the cascade to build up and your computations to start slowing down.

What I find most interesting is that Blue Gene is designed to simulate molecular interactions particularly associated with the degradation of US Nuclear Weapons but it grinds to a halt with half a mouse brain.

I must say though that when I was playing with this over 10 years ago we were talking about ant or fruit fly brains which are merely hundreds of neurons in size and our computers were falling over. Given that baseline, the achievement these guys have made is incredible, although using the most powerful computer on the planet just shows you how far we are from modelling a human brain.

Human brains typically have about a 100 billion neurons with many thousands of synapses. Rough estimates put the number of connections at about a quadrillion synapses which for those of you that like zeros looks like this: 1,000,000,000,000,000

Also to note was that when this is done typically one uses random assignation for where the synapses end up, it isn't a true model of how a brain works as there would be too much information to configure and would have to be done by hand. In these models the neurons are loaded into the system then randomly assigned a number of dendrites which randomly point to other neurons. You don't get real behaviour as in hearing and vision and the like but you do get a sense of how the flow of stimulus and response works.

My own conclusion from my studies and keeping abreast of the topic since leaving formal education behind is that small neural networks specialised to a particular task are more likely to have results than large scale applications like this. Even mother nature adopted this process as you can see in evolutionary history that old structures are built upon by new, more specialised ones - you only need to look at a reptilian brain and compare it with our own, particularly the basal ganglia cluster to see the similarities in structure and function. In pulling these structures together you can then start achieving something that is greater than the sum of its parts.

When CSS goes bad

2007-04-29T22:18:00.000+10:00

If you do a lot of CSS work you'll have seen particular bugs time after time and how to deal with them, however when things do go wrong [and they will trust me] finding bug related information can be a nightmare.

For instance, today I was doing some work on a site and it has had a bug for a few days now in IE. The typical peekaboo bug - if you haven't seen it, it is typically an IE6 thing whereby you rollover a link or element set to have a :hover state and then as well as the effect you want [bgcolor changing for example] something else happens too.

In my case I was rolling over a navigation item which then duly changed colour and then chopped off the page at the bottom of the screen. Even more interestingly it only did it when I had my secondary [nested] navigation up. If it was only the primary it didn't do it. More interesting again was that when I had the two navs up, mousing over the primary one caused the page to disappear and mousing over the secondary caused the page to reappear... argh.

Needless to say with a bug and change list spanning a couple sheets of A4 this was just left for a while.

A quick google for peekaboo bug didn't net much that was useful. Same with position is everything - and this is the problem - nomeclature of bugs and their effects can be so difficult to find in CSS as you have a mix of designers, researchers and techies all calling things different things. Typically my best bet has always been to try and describe the effect in as many different ways as possible on the basis of matching someone elses exact phrasing - as you can imagine this is like trying to hit a dart board on the moon from Earth with your eyes closed.

Perhaps someone will sit down and come up with an accurate way of dealing with web browser bugs that classifies them much like we have with CERT [Computer Emergency Response Team] who classify vulnerabilities in OSes and Software. This would make life a lot easier when a bug was found, especially when it is a variation on an existing one like the peekaboo one I had this morning.

In the end the document which helped me out was this one - http://www.satzansatz.de/cssd/onhavinglayout.html although it did take some trial and error to work out which element needed to have it applied. It works fine now though and my bug / change list is down under a page which is even better....

The things we take for granted

2007-04-24T23:48:00.000+10:00

I had one of those amazing moments as a techie last night where you get so gobsmacked by something that you think about it even a day or so later.

I was doing something on a mate's computer. Fair enough he isn't the most computer literate person in the world but he gets by - he's online and uses the web, email etc to get things done - he's even bought things online too. So he falls firmly in the "average user" category - certainly into the definition of the average use we use when we spec out systems and I'd say anyone that works at a software house or digital agency would look at Dean in the same way.

So what gobsmacked me?

The scroll wheel on his mouse... I was editing a large file for him and scrolling quickly up and down using either the scroll wheel to flick me around the document or doing that windows thing where you depress the button and flick the mouse and you start scanning the document up or down at various speeds.

Dean was sitting there amazed and just said "I had no idea you could do that". Initially I thought he was talking about the editing in the config file I was hacking but it turns out he didn't even realise there was a little wheel in there - as far as he was concerned it was a place to rest his finger and was part of the design of the mouse.

This got me thinking about all these things we as techies take for granted - not the big things like being able to get your email on your mobile phone or being able to mashup 10 different data sources to produce something new and novel or even build a website from scratch using nothign but a text editor.

I'm talking about things like people not knowing their mice can do different things, that your phone probably can browse the web as well as make calls. My wife still can't even write a text message. I come across people even now who don't have internet connections and plenty who are still on dialup.

As our industry grows, how does this moving "average user" impact on the people who are below average? Do we create a digital divide not necessarily based on the "haves and have nots" but on the basis of "cans and cannots"?

Seeing things like this has made me wonder again about people's perception and their interaction with their computing environment. I remember sitting in on a user testing session way back and seeing someone have one of those moments where they "got" the way the web worked - you could tell because she said "That's why that text has a line under it - I always wondered about that".

Anyone who's got a relli who is a timid computer user knows that they don't do things with it because they are afraid they'll break it. Trying to get them to do anything outside their comfort zone is really difficult.

With computers getting cheaper and cheaper and the next influx of users coming on board, we that build systems and interfaces must be sure to remember that the "average user" isn't getting smarter or more stupid - they will initially be less experienced and turning away these less experienced users may result in less traffic / sales whatever further down the line if someone else comes along that will hold their hand and guide them through the process.

In 10-15 years this won't be an issue - but it is now.

Why is CSS such a painful tool

2007-04-23T23:43:00.000+10:00

Looking at the title above you'd be expecting to see a rant covering the lines of CSS is rubbish, it doesn't work properly and why can't we go back to the days of nested tables and lots of little shim images.

I love CSS though, I love the fact that I don't need an editor to edit code any more [trying to do complex layouts in the past it was mandatory to use Dreamweaver to get any degree of speed], I adore that my code is more or less semantically correct and that it searches well. Being a techie and advising people on hosting I also love the fact that all that bandwidth isn't being wasted on buffer code in HTML to do trivial layouts that isn't then cached.

We all know CSS has it's quirks, particularly in IE but the one thing that gets on my goat more than anything else with CSS is that it is a layout language developed by a bunch of people that aren't programmers or designers. They are wannabe web typographists who happen to have the time to come up with and enhance a spec that the rest of us are then forced to use.

This gripe raises its head with me every once in a while but really badly today. This is because we're developing a site that is really thematic in terms of look and feel and the various sections really have their own colour coding. The overall behaviour is the same from one section to another in order to maintain consistency but things like backgrounds, titles and link colours shift to the new pallette.

The really painful thing with this is the inevitable rehashing of styles down the page - particularly if it is a really stylised design with a lot of lines in it etc.

You find yourself constantly typing the same border: 1px solid #123456 or color: #abcdef over and over again. Then when you are in a new section all these items have to be re-instantiated.

What I would give for some variables - say in php format so I can do this:

$red := #ff0000;
$blue := #0000ff;

$borderstyle := 1px solid $blue;
$linkstyle := $red;

Then in my CSS I could generate a selector and then apply the relevant style variable. I wouldn't even need any conditional logic to be happy just a quick and dirty token replacement function to make it work.

Admittedly I could set this up by just making a CSS file a PHP file instead and returning text/css instead of text/html etc but what happens if I'm using ASP or I have to do something that is just quick and dirty HTML for a couple of pages... or what about the performance hit if my site is serving up 1000 page views a minute [or more]. As a CSS advocate I tell people that it is faster to dish pages with CSS as it is cached, but then this method will kill most caching strategies.

We're now at the point where even if this was something that was deemed as being useful by the W3C powers that be we won't see it incorporated until at least the CSS 3.0 spec - which should be coming to a web browser near you some time about 2011/12 and probably won't have wide support until about 2015 - if NASA have their way they'll almost have landed a man on the moon again by then!

So please, W3C - we are in this mire because of a lack of consulation of developers who are implementing the standards you set out. Perhaps getting a development edition of a browser that incorporates these standards at an early stage so they can be played with would be a good direction of resources for the next phase of growth...

busy busy busy

2007-04-23T23:38:00.000+10:00

Damn time flies - and it's not always when you are having fun.

Too many projects going on over a variety of different activities. New clients coming on to our XML Infinity system as well as some natty IT projects for me to ply around with.

I'm going to try and be a bit more regular with my posting by going little and often.

So my Nokia N73 woes continue, even with some new firmware my phone is still crashtastic so I"m counting down the days until a new one arrives. On the plus side though my N700 Internet Tablet is being used for Sat Nav as I drive around the country and although you get the occasional crash when the mobile signal cuts out from my phone, it certainly gets you from A-B with no drama.

More coming on Fuzzy Logic, a chance meeting with someone who is into Social Networking in a BIG way and some other tech rants - this time on CSS.

Fuzzy's where it's at... or will be eventually

2007-03-06T03:07:00.000+11:00

I'm working on a project at the moment that took a remarkable turn recently. Most clients we work on are fairly staid in their use of technology - which suits our company as we are firm believers of the Keep It Simple Stupid methodology of programming.

I was in a meeting with a client who is a large retailer and we were talking about "filters" for being able to reduce sets of data returned from the database. Things like "style", "size", "price" etc - not dissimilar to Dabs or any one of a thousand other online retailers. Off the cuff I just said "wouldn't it be good to use fuzzy logic on the filters so instead of black and white result you get the shades of grey as well". To be honest I'm not even sure why I mentioned it...

Imagine my surprise when the client said "Show me"... Out came the pen and paper and 15 minutes later he was sold on the idea and I was left to code and example.

Fuzzy logic is a funny old beast - it is based around this notion that instead of black and white you deal in shades of grey - black and white are just extreme examples of the shades of gray. So black might equal 0 and white might be 1 but in between we can have 0.5 or even 0.3218956 if you so desire... everything belongs to every group at least in part - even if that part is tiny, or even 0.

I love fuzzy logic - I played with it a lot at Uni when I was studying Neural Networks - but it has never made it into mainstream web use - mostly because it is so difficult to implement with a database unless you do a lot of extra background work. Background work most clients won't pay for.

It does make a big difference though - take this as an example:

Say you have two products, Product A is £295 and Product B is £305.

Now suppose you have a filter, or a search query that says "Give me everything less than £300"

Obviously Product A gets returned but Product B wont if you are using discrete maths as it isn't 100% lower than £300.

In the fuzzy view of the world though we can say "Give me everything approximately less than £300". Now depending on your exact specification of what "approximately" means Product B may well be returned. Indeed most people prepared to spend £300 will probably spend £320 so we could say Product B has a 95% fit for this result. As we get closer to £320 the relevance gets less so it is less likely to be returned.

Ahh, I hear you say, I can do this by just pushing my filter up. Yes you can but then £321 is left out altogether again. Maybe £321 is not as relevant as £305 but it is more or less as relevant as £320 on this scale.

Fuzzy logic has made huge strides in engineering particularly with control systems for things like washing machines [if a load is heavy use more water, if light use less] and airconditioning units [if it is hot turn on harder than if I am more or less where I need to be] but it has never caught on big time on the web.

I think that now that fundamental systems are starting to get in place [e-commerce etc is no where near as difficult as it used to be] then we will start seeing clients and programmers starting to use their brains a bit more and looking at how we can deliver the best experience for our customers.

please nokia slap me again - no really

2007-02-16T00:56:00.000+11:00

I love Nokia phones. In fact you could probably say that I have had a love affair with Nokia devices for over the last 10 years ever since my first one.

Typically they sum up everything that is important to me about technology - that it must look great and must function well too. Nokia just always seem to push my buttons when it comes to putting down my cash and getting a new phone. Don't get me wrong, I've dabbled with the dark side of Ericsson and then Sony Ericsson, I'd love to love a Motorola - really I would, but there is just something about that Finnish company that every 12 months when it's time for an upgrade makes me want to give them another chance.

Over the last few years, incredibly though, their products have got worse...

My last three phones have been a 6600, an 7710 and now an N73.

Every one of these have had internet, bluetooth, big screens, cameras and all the usual gubbins.

Why then have they got progressively worse in terms of functions?

My 6600 had terrible battery and would often crash. My 7710 had great battery, would often crash and didn't support most web page content even though it was a wide screen touchscreen that was supposed to have a fully featured web browser.

My n73 is even worse, admittedly though the Carl Zeiss lense on the 3mp camera is a piece of art and takes crystal clear photos. The web browser is much improved thanks to the Mozilla engine and the 3G connection I have which renders web pages at high speed. The battery life is awesome as well. The only problem with this one is that I can't make or receive phone calls without it crashing - literally to a black screen of death - about 80% of the time.

Vodafone won't acknowledge a problem because Nokia won't so there's no sending it back - I pity everyone who has tried and had their phone returned a couple of weeks later saying "there's no problem". It isn't just me either check this out in google: http://www.google.co.uk/search?hl=en&q=problems+nokia+N73+crash&meta= and you'll se the extent of the problem.

So taking a punt before xmas I bought an n770 internet tablet. Again a fine and beautiful bit of kit, but again with it's fair share of bugs - not least a *very* buggy version of opera which Nokia has decided not to support any more, leaving it's internet tablet with a very bad internet web browser - ouch - 250 quid not well spent... well at least I can make a robot out of it.

So I'm looking around again and I know that just like a jilted lover who somehow thinks that "things will be different this time" I'll be back at Nokia's bosum probably with an N95 in my hand... mmmm just look at the screen and the slidey-outy-bit... 5mp camera, 3G with 11 GPRS slots for quick downloading... I really don't need to make a phone call.

Finally got around to setting up a blog

2007-01-31T05:02:00.000+11:00

So I've finally had time to set up a blog. Our company [ http://www.xmlinfinity.com/ ] has a no private publishing policy, which is fine as I can see why, but it does mean that when one needs to dump a whole load of info out of your brain there is no convenient way to do it.

Hence this blog...

To be honest, this is being set up more as a dumping ground for me to record information so I can find it again as a useful snippet rather than being entertaining. If you do find something on here that is useful then I'm glad.

So what is likely to be here: Most likely things to do with the technologies I am interested in, this is mostly XML and XSLT though will probably comprise other web dev things too - particularly if it is to do with mobile computing. Chances are there will be the occasional rant as well when things go wrong technology wise or I see ridiculous implementations of things.

Cheers