Category Archives: search

Local is the new social – location data startups

    Start a conversation 
Estimated reading time 4–6 minutes

A few weeks ago I attended an event by Dreamstake featuring a collection of startup companies that are using open geographical data – such as the data released by Ordnance Survey. There was much championing of the possibilities of much money to be made by using data that organisations release for free. This seems obvious to me – someone else has paid to do all the preparatory work so others can cash in. No-one seems concerned about the ethics of this. If UK taxpayers have paid for the OS work to be done, should they not automatically be shareholders in any company that profits from the fruits of this investment?

The companies showcased all had new twists on using location data. What I found especially interesting was the emphasis on context. When selling services, place alone is not enough. Time is important and also the circumstances. So, a businesswoman on a work trip will want probably different products and services to when she is out with her family.

The speakers were
James Pursey of Sortedapp
Sadiq Qasim LoYakk
Craig Wareham of Viewranger
Tim Buick of Streetpin

Location-based marketing

James Pursey opened by giving a brief history of location-based marketing, pointing out that this was pioneered by the Yellow Pages (now His company attempts to match time, place, and location and makes the consumer the advertiser and the service provider the respondent. He explained this as a “reverse Ebay”. Instead of advertising your products and services, consumers post details of what they want, e.g. I need someone to clean my flat before my wife gets home (the data game still seems to be a man’s world!). The message is then pushed to local cleaners who have a window of time in which to respond. The app works on the location of your mobile phone, but you can alter that on a map so that you can be at home but arrange a service to be provided near your workplace, etc.

Chatting about a shared experience

Sadiq Qasim explained that LoYakk – local yakking – recognises that conversations are often focused around specific places and events. Social media links tend to be based on static lists of friends, with very little contextualisation. However, social relationships and conversations are often transient. You might want to chat to someone at a conference, but that doesn’t mean you want to become lifelong friends. By creating an app that mirrors the real world nature of such connections, people can drop in, chat to people in the vicinity and leave again. Events such as conferences, arts and sporting events, and holiday destinations are particularly well suited to this approach.

Mobile is local

Craig Wareham described Viewranger, which is an app for outdoorsy people. It combines guidebook information, a social community, a marketplace, based around location and has become popular with search and rescue teams.

Tim Buick of Streetpin emphasised that about half of searches on mobiles – perhaps unsurprisingly – are for something local. However, time is very relevant – he might be near a great pub that has a special offer on beer but he doesn’t want to be told about it at 8 in the morning when he has just dropped the kids off at nursery, but in the same location 12 hours later with his mates, the offer might be just what they want. The right information, to the right person, at the right place and at the right time is what matters.

The distinction between what is useful information and what is marketing becomes very blurred.

Place, space, maps

Thinking about this event along with the Shape of Knowledge event’s discussions of maps of cyberspace, and the Superhuman exhibition’s raising the question of the potential of transhumans to relate to space in a different way to current humans, made me wonder how location-based services will change in future. The technologically enhanced human will, presumably, need maps that make sense to computers as well as maps that make sense in real space and time. Navigation and location are most likely going to change beyond all recognition.

Building bridges: Linking diverse classification schemes as part of a technology change project

    Start a conversation 
< 1 minute

My paper about my work on the linking and migration of legacy classification schemes, taxonomies, and controlled vocabularies has been published in the Journal for Business Information Review.

Isn’t search the same as browse?

    Start a conversation 
Estimated reading time 4–6 minutes

I nearly wept when one of our young rising IT stars queried in a meeting why we had separated “search” and “browse” as headings for our discusssions on archive navigation functionality. So, to spare me further tears here are some distinctions and similarities. There won’t be anything new for information professionals, but I hope it will be useful if any of your colleagues in IT need a little help. I am sure this is far from comprehensive, so please leave additions and comments!

Differences between search and browse

Search is making a beeline to a known target, browse is wandering around and exploring.
Search is for when you know what you are looking for, browse is for when you don’t.
Search is for when you know what you are looking for exists, browse is for when you don’t.

Search expects you to look for something that is findable, browse shows you the sort of thing you can find.
Search is for when you already know what is available in a collection or repository, browse is how you find out what is there, especially if you are a newcomer.
Search is difficult when you don’t know the right words to use, browse offers suggestions.
Search is a quickfire answer, browse is educative.
Search is about one-off actions, browse is about establishing familiar pathways that can be followed again or varied with predictable results.

Search relies on the seeker to do all the thinking, browse offers suggestions.
Search is a tricky way of finding content on related topics, browse is an easy way of finding related content.
Search is difficult when you are trying to distinguish between almost identical content, browse can highlight subtle distinctions.
Search rarely offers completeness, browse often offers completeness.

Search is pretty much a “black box” to most people, so it is hard to tell how well it has worked, browse systems are visible so it is easy to judge them.
Search uses complex processing that most people don’t want to see, browse uses links and connections that most people like to see.
Search is based on calcuations and assumptions that are under the surface, browse systems offer frameworks that are more open.

Search works well on the web, because the web is so big no-one has had time to build an easy way to browse it, browse works well on smaller structured collections.
Search can run across vast collections, browse needs to be offered at human-readable scales.
Search does not usually give an indication of the size or scope of a collection, browse can be designed to indicate scale.

Similarities between search and browse

Search and browse are both ways of finding content.
Search and browse can both be configured in a huge variety of ways.
Search and browse both have many different mechanisms and implementations.
Search and browse should both be tailored to users’ needs.
Search and browse systems both require thought and editorial judgement in their creation so that they work effectively for any particular collection.
Search and browse systems can often both be created largely automatically.
Search and browse often both involve metadata.
Search and browse behaviours may be intertwined, with users switching from one to the other.
Search and browse may be used by the same users for different tasks at different times.
Search and browse both offer serendipity, although serendipitous opportunities are often hidden by interface design.

Should I offer my users search or browse?

Almost always, you should offer both. Unless you are very sure that your users will always be performing the same kind of task and have the same level of familiarity with your content. With small static collections of content, it may not matter too much, but for most content collections, users will probably want both, but which you make your main focus depends on the context and collection.

Shops might have lots of images and very little text, so a beautifully designed navigation system will help customers find – and buy – products they might not know about, while only a simple search system might be needed to cover searches for product names. A library will need to support lots of searches for titles and across catalogue text with a good search system, but will also need to help educate and inform users with a clear user-friendly browsable navigation system. A large incoherent collection of unstructured text with no particular purpose is likely to be difficult to navigate no matter what you design, so will need good search, but – apart from the web itself – such unbounded and unmanaged collections tend to be quite unusual.

Your organization is not the Internet

    1 comment 
Estimated reading time 7–11 minutes

Many people find it very difficult to understand why search within an organization can’t “just be like Google”. This is often because they haven’t thought about the differences between an organization and the Internet.

Your organization is smaller than the Internet

Search engines like Google work because they have access to big data. Google gets billions of searches to process, from billions of users. Even if your organization is a large one, it won’t have that many users either searching or contributing content, so it cannot number crunch on the same scale as Google. Your IT department is probably a lot smaller than Google’s and your enterprise search team’s daily budget is unlikely to cover more than the tiniest fraction of what Google spends. Last, but by no means least, your organization doesn’t have as much content as the Internet, so it probably needs to be far more careful about not losing any that is valuable.

Surfing the net is not many people’s job

There are important differences between how and why people search when they are at work and when they are not, and between how and why they search the Internet and their organization’s Intranet or archives. People rarely surf their organization’s Intranet for fun, to be entertained, or to while away the time. The differences in serious research behaviour and leisure searching are well documented, so I am going to write about another aspect of differences between the Internet and organizations that is often overlooked.

Putting stuff online is not the same as writing a business report

There are vast differences in the ways that people create and curate content on the Internet and within an organization. These differences have a significant effect on the way search functions. The key difference is in how much they link their content to that of others. Of course, there are people whose jobs are to create and curate online content – all the web editors, content strategists, copywriters, social media marketers, etc. – but they will be the first to explain that they have a very specialised set of skills focused on making their content searchable, commercial, or otherwise user friendly. They do a whole lot of things that most people as part of the day job neither know how nor have the time to do.

Links are a form of Knowledge Organization that Google gets for free

One of the key things that web professionals and unpaid web enthusiasts do with their content is to add and manage links. Links are what organize the web. Links are what group sites into clusters by content. Links are the web’s classification scheme. Clay Shirky back in 2005 said “there is no shelf” but it makes just as much sense to think of millions of shelves – infinite shelves going off in all directions, with new ones being created and old ones being discarded. The web is not linear – like a shelf – but it is not without structure. Google effectively picks one of the near infinity of shelves and offers it up as a linear list whenever you do a search. It chooses the shelf that seems to be the most popular, or that fits its commercial model. First on the shelf is often a paid-for advertisement or a Wikipedia entry, followed by other big well-established commercial sites. Out there on the Internet, people do an awful lot of shopping, and not much work, so that’s fine. (If they are doing more shopping than work when they are at work, your organization probably has bigger problems than search to deal with.).

For many other searches, especially more thematic research, people would be disappointed with the results, were it not for the magic of the way the web works – the links. As long as Google slings a site at you that has lots of links to other sites, it doesn’t have to take you straight to what you want, it lets you and the links do the rest of the work. Links gather together similar content, so they function like a classification scheme. The links associate content that is aimed at similar audiences, is on similar topics, is of a similar age. The links represent a huge amount of sorting, cataloguing, and classification work. Google did not have to pay for this work (genius business model). People do this work for Google for free. They do this work as part of creating and curating their content.

Many of Google’s volunteer librarians do this work for fun. They create fan sites, they write Wikipedia articles, they produce lists and generate indexes to their favourite content. They provide cataloguing descriptions and context. They do all this work partly because they enjoy it and partly because they hope to get “repaid” by their site becoming popular. They hope this will either lead to monetary reward (their band will get signed, they’ll get a better job, they’ll sell advertising) or social reward (they’ll make online “friends”, get positive feedback from comments, etc.).

From the commercial angle, people do this work because they expect to gain financial reward. They want to sell more products and make money. This is why there are howls of pain whenever Google tweaks its algorithms. Companies that balk at investing in internal search systems will spend fortunes chasing SEO.

Are your staff content curators?

If you want your organization’s search to be “just like Google” you need to think about how linked your content is. Do people who create content in your organization do so for the same reasons and with the same motivations as people create and link content on the web? It is very unlikely that you have lots of “fans” who will spend their free time creating lists of your companies’ best information resources, or collecting and rating and reviewing reports and documents. Most employees are too busy getting on with their day jobs to spend office hours pursuing their “fan” projects. Even if your staff have plenty of spare time, how many of them are big enough fans of some aspect of work to treat it like a hobby? If you want people to start looking out for similar documents on your Intranet and linking their own documents to them, you will probably have to find ways of motivating them to do this as a special initiative. It is not likely to come “for free”, like it does for the web search engines.

For some organizations, encouraging and incentivising “fan”-type behaviour may work. If the organization already has a strong collaborative culture, with people sharing ideas and using social media, it may be a small step to get them to think of their documents and presentations as blog posts. Including content creation and curation in people’s job roles and rewarding those who do well will foster a link-rich Intranet. By recognising and rewarding people who promote useful links and lists and get them to rank highly in your enterprise searches, you could bring an element of gamification to encourage this sort of behaviour. For other organisations, the culture may support this kind of web-style content creation, but people are generally too busy, have skill sets too far from what is required, or need training and encouragement. In such organizations it may make sense to have the equivalent of web editors, content strategists, user experience specialists, search engine optimizers, etc. working with the organization’s internal content to promote the most valuable resources. In other words, layer of “linkers” who work alongside the content originators.

For other organizations, where it would be inappropriate, too time consuming, or too far from established culture to encourage web-like information behaviour, enterprise search will never work “just like Google”. More formal and standardized metadata management processes are likely to be needed. Organizations that generate a lot of very specific content that is unlikely to be useful in broader contexts, confidential content, or large volumes of very similar structured content are likely to find it hard to move away from directed and standardised searching.

Many organizations will have a “mixed economy” with different types of content and different departments operating with different styles (e.g. what works in a marketing department is unlikely to work in the same way in a finance department).

Without links, search is a lot of dead ends

Without links, each search result is isolated. This stops the searcher in their tracks and means they cannot surf in the way they do on the Internet. They will have to check search results one after another in a linear fashion. If your search engine is not getting the most relevant results to the top of that list, your staff will be spending a huge amount of time working their way through that list. They cannot plump for one likely looking result then follow the trail of links, as they do on the web. The links as a form of classification do not exist, so you need another mechanism (taxonomy, ontology, index, directory) to help people find groups of related content and browse through from one document to another.

So, even though you may have the technology and the budget to match Google’s, unless your content creators are linking freely, you will never completely succeed in turning your Intranet into a mini-Internet.

Data Ghosts in the Facebook Machine by Fantasticlife

    1 comment 
< 1 minute

Understanding how data mining works is going to become increasingly important. There is a huge gap in popular and even professional knowledge about what organisations can now do “under the surface” with our data. For a very clear and straightforward explanation of how social graphs work and why we should be paying attention read Data Ghosts in the Facebook Machine.

Classification meets the Web – UDCC Seminar 2011

    Start a conversation 
Estimated reading time 2–2 minutes

This post is 4th in a series about the UDC consortium international seminar in The Hague, 19-20 September, 2011.

Interoperability of knowledge organization systems with and through ontologies

Daniel Kless from the University of Melbourne pointed out that problems with ontologies arise when combining them, as errors in combination can have disastrous effects on subsequent reasoning. A well-defined modelling method is needed to minimise this. Standards such as OWL and RDF do not address the problems of methodology or terminology control.

Towards the integration of knowledge organization systems with the linked data cloud

Vincenzo Maltese of the University of Trento, Italy, explained how it is vital to make clear the semantics and purpose of any ontology when attempting to share Linked Data. Ontologies may differ in their scope, purpose, structure, terminology, language, coverage, formality, and conceptualization. He drew a distinction between descriptive ontologies and classification ontologies. It is very easy to convert a descriptive ontology to a classification ontology and the process can be automated, but extremely difficult to convert a classification ontology to a descriptive one and the process requires human intellectual and editorial effort.

Classification and reference vocabulary in linked environment data

Joachim Fock of the Federal Environment Agency (Germany) talked about how they transformed their keyword thesaurus to a Linked Data format.