Over at search engine land Danny Sullivan is calling this the most significant changes to organic search. I’m still torn between this, the Vince update, social and real-time search being talked about and labbed.

This was something we all thought would happen once Google had enough data to be confident of providing a better user experience though personalisation most of the time, and look like that day is here.

There are plenty of arguments against personalisation by default – Danny points out the lack of disclosure to users. For me this highlights a much greater issue around the need for search engines to be more open about how they determine results. I’m not talking about giving away the crown jewels, but users need to know more about how information is being filtered for them.

Another area of worry surround the rich get richer scheme promoted by personalisation, with newer sites finding it even harder to break into results. I’m not sure this is any different to using links for rankings. Or in fact any other measure of popularity.

Google has stated that personalisation is not going to be so dramatic as to prevent variation in results. I’m curious as to how this may be solved once we move to greater personalisation beyond a few sites being effected for a few queries.

I think we need to look at all the other changes that have been taking place this year to put this into context. I suspect buzz data will provide the additional signals for freshness, and the social graph has the potential to make things even more personal.

Google’s recently made deals with Twitter and more recently MySpace, and while these may not be as big as Facebook, convergence of this data is inevitable.

I was a big fan of the Vince update, and I feel the same about personalisation. Many in the SEO business will worry about loss of traffic as only a fraction of people will see your results and you’ll find it even harder to get a site ranking with traditional techniques. But the traffic you get will be much more targeted.

Understanding your audience will be critical. Focus on the right user and all else will follow.

We’re just starting to see some of the biggest changes in search. Roll on 2010.

Porting the social functionality of FB provides a powerful connectivity layer which YouTube has so far lacked. The Fan base can be grown through the personal newsfeed, while privacy can be controlled and propagates through to YouTube or any other platform.

The thing I find really interesting is the idea of using your social connections and behaviour to validate who you really are.

Google recently introduced a feature which enables users who are logged into a Google account to leave comments. This looks very much like a move towards at least the possibility of social search. By analysing the search and browsing behaviour as well as your email/talk network, it shouldn’t be too difficult to identify a real user from a auto-generated fake account.

The SEO community have argued that social search (i.e. taking social signals to influence organic search results) is unlikely to happen until you can trust those signals.

I’m looking forward to seeing how Coke’s Green Eyed World benefits from the authenticity provided by FB connect.

If you want to understand a bit more about traditional login based identity compared to what it may look like in a network, I’d recommend reading Adriana’s post ‘Bringing identity home a VRM view.

The issue for Google has been the way it tried to identify those brands. Essentially the Google algorithm relied almost entirely on the published web to define relevance and credibility/reputation. This was easy to manipulate and doesn’t always represent relevance defined by the searcher. My favourite search query for ‘table’ is a good example of this misalignment.

With this new update still to roll out for Google UK, I’m seeing more brands ranking for terms they are not specifically optimised for.

Now for the pseudo science bit -

Searching for ‘flights’ British Airways are now in the top 10 from a previous position outside the top 30.

Searching on a pure anchor text search using ‘allinanchor:flights’ they are still outside the top 30. But on a synonymous search using ‘~flights’ they are in the top 10.

So BA has been identified as
The tilde (~) query returns results based on not just the pages matching the exact query words but those that are synonyms of the query words using something similar to Latent Semantic Analysis. I say something similar because we are seeing something more than just statistical analysis of on-page content. If this was the case, we’d be seeing BMW ranking for ‘car’ (we still may).

Looks to me like user data is being incorporated to identify those instances where the synonym matches searcher expectation.

Note ‘BA flights’ in the related queries section at the bottom of the result page for ‘flights’.

So we may be seeing brands that haven’t bothered to optimise for searches gaining a few positions for core terms. Though this is all a bit hit and miss and is no substitute for proper keyword analysis and content optimisation. At least not yet.

A more recent development has been query refinement based on your immediate search history. This has been rolled out on paid search where some of the ads reflect you previous query. A few months ago Google announced a similar approach to their organic search results, and we’re starting to see it over in the US. Note the ‘wording on the right’

I’ve yet to see this in the UK but based on the reports from my iCrossing colleagues in the US, this looks more like a rollout than a test. There’s no mechanism for turning it off either other than may be to turn off cookies (seems unlikely that Google is using IP to track users).

From a search marketing perspective, these developments pose a number of challenges not least to traditional metrics like search engine positions. There’s no point in looking at search engine positions when what you see could bear little correlations to what your customers see.

Understanding user behaviour for various task based search sessions will be fundamental to delivering the right traffic. Those sites that are narrow focussed will find it harder to appear in personalised or refined results. I.e. the travel site with just the search form a thin destination guide and a bunch of dodgy links is unlikely to create enough engagement to appear in personalisation.

The sites that are informative, covering the full spectrum of touch points will engage searches early in their session and is more likely to rank higher when they are ready to buy.

First thing to say about it, I hope the reason it keeps breaking down on me is due to the huge volumes of traffic interest it is already generating.

Next – I’m not a fan of the look and feel and general interface – It feels like a destination than a search tool, but that may just be me.

First thing I notice, is the claim of a 120+ billion pages index – wonder how many of them are spam
Next I’m reading the About Us page try to ignore the rock circle to the left and this line grabs me

‘Rather than rely on superficial popularity metrics, Cuil searches for and ranks pages based on their content and relevance.’

Ok, so Google beat Altavista by using popularity to judge credibility, but these guys have found away to judge the quality of content based on the content…interesting.

Then I read this line -

‘We believe that analyzing the Web rather than our users is a more useful approach, so we don’t collect data about you and your habits, lest we are tempted to peek.’

How could Google have gotten it so utterly wrong?

Time to search – first my favourite search, especially useful for the claim the web can define relevance better than users. Search for ‘table’ and sure enough, lots of tables from HTML, to table tennis and the periodic table. Must admit there’s more of choice of table than with Google, but like Google a distinct lack of plain old furniture. But in the Explore by category box, they do have what most people were looking for.

So categorisation of content is a big part of Cuil and I kind of like it.
Actually I lie – my first search was for me ‘Nilhan’ where I’m number one in Google and at least 6 of the top 10 refer to me. Cuil is not a fan ;) In the top 20 the closest I get is an SEOMoz profile – hmmm not using links for authority??

Next i’m thinking of stop words – Google likes to ignore stop words (it, the, from, to etc..) – Search for ‘flights from paris’ using Cuil, and you get pages containing the exact phrase and ‘from’ is included not ignored. Same search in Google, will return anything with ‘flights’ and ‘paris’ – ‘from’ is ignored or treated as a wild card. So again Cuil does well here

Next that other puzzle search engines have had to solve – when are two words a phrase and not just two words? So I search for ‘holidays to new york’ and Google brings back the usual suspects. Same search in Cuil and the first site talk about New Zealand, the next about apartments in Argelès-sur-Mer – Malaga, Disneyland Paris and self catering in France and there’s just one site referring to the big apple.

Then I search for ‘holidays in york’ and as expected, lots of listings about New York – Wrong!
Not having any geo-targeting capabilities makes it a real pain to use. Most of my results tended to be from the US (hmmm not using links at all?).

Must get back to some real work. First impressions, not a Google killer and not entirely sure what the fuss is. Some of the features like categorisation is cool, but doesn’t feel new. May be I’m really missing something here

You can vote up listings you like and leave comments, and vote up or down other peoples comments. All your edits can be tied to an account profile, which provides some level of protection against the Bot-people.

I think the biggest flaw in this type of system is the need for the user to stop what they are doing and then do something else. So most people will not be providing feedback. Google really need to collect that data as people are going about their business, and with personalisation that is happening.

Having said that, like Digg, there will be certain community experts who will take it upon themselves to police their areas of expertise, and you don’t need many of these to provide another valuable signal.

In the Financial services sector at least, I’m seeing Google placing different sites at position 4 for anything from a few hours to a day. These are usually sites that are normally outside the top ten, and in one case outside the top 30.

Position 4 is usually reserved for News results – almost like a place holder, and now it looks like Google may be using it to test user reaction for sites Google want to know more about.

It was inevitable that Google would increase reliance on user data – in areas like Financial services, especially car insurance, where it would be very difficult to differentiate based on the usual on-page and link factors – user data seem to be the only logical evolution.

I believe all this is not entirely new, the reason for the success of aggregators can be credited largely to how sticky they are. A Good aggregator can command a conversion rate of 30-50% which makes them very sticky, when you think how long it takes to fill out a insurance/mortgage quote form.

But position 4 testing is definitely something relatively new. The next few weeks should tell us how/if they use this test data.