Four #real-time search options leave Google behind

While there is no sign of panic yet, there is certainly evidence that established web giants such as Google and Facebook are frantically playing catch-up as the clamour for real-time search grows. Not only are there a range of tools to monitor Twitter in real-time there are browser add-ons that enable you to add Twitter search results to your Google results page. Why is Google being so slow to grasp the demand for real-time search?
The bottom line for journalists is that Google no longer has the best answer to the simplest question: ‘What are people saying about [my query] right now?’ For those of us steeped in Google search experience – it’s a scarey thought. Google is, apparently, working on a real-time offering but, in the meantime, are there competitors to the real-time results available from Twitter and the various Twitter monitoring tools available?
Here’s a heads up on four tools that don’t rely exclusively on Twitter and offer real-time monitoring for your search queries.

1 Collecta, just a few hours old, has the cleanest and most intuitive search page. Your search term is tracked and results listed in a central column. Clicking on those results gives you fuller content and a link to the source. The left hand column lists your recent queries and gives you the chance to include blogs, blog comments, images and updates (tweets) in your results. Collecta also tracks the time since your query. I’ve had impressive results so far.

2 Scoopler, has another crystal clear search page with the main section split between ‘real-time’ search results and ‘popular’ results that include content from news sites and videos. The ‘real-time’ results are drawn from Twitter, Digg, Delicious and other networking sites. This middle column gives you the options of previewing the posts or you can click through directly to Twitter profiles.

3 OneRiot, claims to do the same as Scoopler and Collecta but when I tried its ‘realtime’ search for results containing ‘tehran’ and ‘iran’ there weren’t a convincing number and they failed to appear with anything like the frequency I would have predicted. On the plus side, you can use its ‘pulse’ search option which uses an algorithm that looks at dozens of factors to give “weight” to certain results. OneRiot has used various factors to influence the weighting including: freshness; source credibility; and, ‘acceleration’ whereby posts that are gaining momentum (links) on the web are ranked as more important.

4 I looked at Icerocket in January. Not only has it set its blog search as its default option, it has also added a twitter search and a real-time ‘Big Buzz’ alternative. Big Buzz pulls in very recent results from blogs, Twitter, Video, News and Images and gives you an ‘auto refresh’ option to update those results every minute or so as you are working on a story.

I am @colinmeek on Twitter

Tags: blog search, collecta, crystal clear search page, Facebook, google, icerocket, intuitive search page, oneriot, real-time, real-time monitoring, real-time results, real-time search, scoopler, search experience, search option, search queries, search results, search term, Twitter, web giants

Google’s advanced operators for journalists

Master commands for precision surfing. Presentation.

Confusion is rife about how and when you can use Google’s advanced operators. Used effectively they can transform your research by helping you get better results faster. Here’s my recently updated presentation on advanced operators with some context and example results.

Tags: advanced, google, journalism, operators, search, techniques

Twitter monitoring tools

During a couple of recent courses delegates have been keen to know more about real-time twitter monitoring tools. So here are a few options:

Tweetdeck is a free download that lets you monitor posts in real-time as well as send tweets directly. Perhaps its best feature is the ability to create columns so that you can track different terms using your own dashboard.

Twitscoop is a very simple tool that allows you to monitor specific trends and buzz around a topic. You can monitor the use of specific terms over the last few hours or days. It doesn’t let you compare keywords however.

Twhirl is another desktop application that you can download for free and then use it to monitor Twitter for specific keywords. You can also integrate this with your Twitter and other social networking accounts.

Tweetscan is a Twitter search tool that automatically refreshes the search every few seconds. This also lets you limit the search to specific users.

Twazzup is similar to Twitscoop but it also highlights related hashtagged terms, the most popular links, featured tweets, and singles out specific Twitter members as ‘trendmakers.’

Monitter also allows you to monitor specific terms as they appear in tweets but this tool allows you to monitor three in their own dedicated columns. Monitter also, very slickly, lets you monitor keywords within a specific distance of a specified location.

Update: See my posts on Tweetgrid and Tweetbeep for more on this.

Tags: search tool, social networking accounts, twazzup, tweetdeck, tweetscan, tweetscoop, twhirl, Twitter

Turbo-charge your Firefox browsing

As most people know, if you’re not taking advantage of the many Firefox add-ons and plug-ins then you’re not making the most of this browser. Even so, where do you start? I’ve read several blogs recently listing the ‘best 20′ Firefox add-ons with others running the list to 50. But if you genuinely want to take your research to the next level you need a few hand-picked additions that will help you do more in less time. If that sounds too good to be true – here are a few ideas.

1: Scrapbook: This add-on is an incredibly powerful research tool that enables you to save web pages, page snippets and whole sites. You can organise your saves just like bookmarks (by dragging and dropping in trees) but, crucially, scrapbook saves the page (or pages) not just the link. If you need reliable access to sources, this is the add-on for you.

• save pages using a drop-down menu or by dragging the page favicon into the Scrapbook Firefox sidebar.
• drag and drop page snippets and save linked pages just by dragging the links to the sidebar.
• highlight sections in saved pages.
• annotate pages.
• use ‘in-depth’ capture to save whole sites and create site maps (see below).

Scrapbook is the answer if you need access to a range of pages and sites offline and to ‘capture’ a whole site and its links to external sites. Scrapbook even comes with a filter tool that means you can capture only the pages belonging to a target site while ignoring external links.

2: Picnik: Not necessarily a research tool, but beautifully simple and useful. Picnik is a quick way to do what you want with pictures – online, in your browser. You can create files of pictures, pull them from you own accounts on sites such as Flickr and your own hard drive. But, from a research and publishing perspective, you can download images from sites, give them a quick edit, change their format ready for use within seconds.
The Firefox add-on makes life even simpler. Right click on an image (or ‘ctrl’ click for Macs) and you can ‘edit image in picnik’. The image then automatically loads to your library in Picnik. No need for the laborious task of saving images to a photo editing application then exporting locally before you can upload online.

3. The Evernote Webclipper:

This add-on creates a handy button on your Firefox browser that you can use to quickly save a selection of a web page or an entire page to your Evernote account. If you need some background on why Evernote can transform your online life then check my recent post on this app.

4. Juice: This add-on is one of a new wave of intelligent search tools that let you access linked content without you having to navigate away from the pages you are viewing. By highlighting and dragging a selection, Juice searchers for reference material, movies, news and pictures and presents the content clearly in a separate Firefox column. You can switch Juice on or off easily by using a simple button on your browser bar.

Juice’s rocking webcast from Linkool Labs on Vimeo.

5: Semantic Radar: For those of you interested in the development of the Semantic Web then Semantic Radar is another tool that gives us a glimpse of what semantic tools are bringing to the web. Semantic Radar recognizes all RDF content and displays custom icons in Firefox to indicate presence of the data in languages such as SIOC and FOAF. This screengrab shows how Semantic Radar has detected RDF content on a Livejournal page. Click on those icons and you can access the RDF content directly. For more on the Semantic Web see my interview with John Breslin.

Tags: EverNote, filter tool, intelligent search tools, John Breslin, juice, Linkool Labs, picnik, RDF, research tool, scrapbook, semantic radaer, semantic tools

Furl users transfer to diigo

The much loved research tool Furl is being absorbed by Diigo – a social bookmarking tool for serious research.
Like many, over the years I’ve found Furl crucial for research when I’ve needed access to saved versions of pages – not just bookmarks. Unlike delicious, Furl let me save whole pages to its servers rather than just the link. It also came with a heap of other tools that let you network with other users. Out of the blue, however, this week Furl’s one million users were told that diigo has acquired Furl after Furl’s owners – LookSmart – changed focus.
‘We worked hard to find Furl a home where loyal users like you could continue to benefit from best-of-breed social bookmarking and annotation tools,’ Furl said. ‘Hands down, Diigo was the winner due to its innovative approach to online research tools and knowledge sharing.’
Again, Diigo is probably a more reliable and flexible research tool than delicious. You can:

• highlight parts of web pages and archive those section;
• attach ‘sticky’ notes to pages;
• save pages and your notes to Diigo’s servers;
• share saved files to with a project team;
• network with other Diigo users by contacting them directly or watching what they save; and,
• explore by tag.

As I’ve stressed before, if you’re involved in serious research one of the biggest problems with delicious is the fact that web links can become inactive very quickly. If you need reliable access to your source material quickly – you need another solution. I’ve no experience with Diigo so I’ll monitor its service over the next few weeks.

In the meantime, if you’re a Furl user, Diigo has set up an easy way to transfer your archive. More on Diigo soon.

Tags: annotation tools, diigo, Furl, Furl.net, online research tools, research tool, social bookmarking tool

Oh My God – another alternative search engine

Given the plethora of new search engines that emerge I’ve tried to cover only those that offer genuinely new tools or points of emphasis that are seriously useful. For those reasons I’m impressed with Omgili – a search engine with a serious difference in that it compiles its results from millions of online discussions worldwide using around 100,000 boards, forums and other discussion based resources. Use it when you’re looking for opinions about something – not facts.
The search tool comes with a variety of options including ‘buzz’ which gives an overview of the most popular recent discussions. ‘Health’ lets you find answers discussions and communities on health topics; ‘Reviews’ is focused on consumer reviews; and, ‘Graphs’ lets you examine how often a range of terms crop up in results over a period of time. In the graph below I compared the terms ‘fred goodwin’ ‘RBS’ and ‘HBOS’.

I’ve used Omgili for a few days and found it draws results from a reassuringly wide range of sources, furthermore, included in those are results from worthwhile discussions that you might otherwise visit individually; discussions from MoneySavingExpert.com, Channel4 and No2ID for example.

Tags: omgili, online discussions worldwide, Search Engine, search engines, search tools

Notebook options: Yahoo Search Pad; Snapbits; and, notebook G

I’ve covered notebooks such as Evernote in some depth but there are a range of other tools available that help you organise clips, bookmarks, notes, documents and screenshots in ‘notebooks’.

Here are three others:

notebook G
“notebook G is an online personal information organizer you can use to keep track of the various notes for home, school and work. Use notebook G as a replacement for a paper-based spiral notebook, or as a supplement to paper-based organizers.
Since notebook G is web-based, your data follows you wherever you go. And your data is protected from viruses and spyware, as well as backed up to a world-class data protection firm.”

SnapBits
“SnapBits is a free service that may be used to securely store, search, sort and easily retrieve “Bits” of information. These “Bits” may be anything you wish to remember or keep a note of. You may store as many as you like and unlike conventional paper notes, you will never lose or fail to find your information.”

Yahoo’s Search Pad
Yahoo’s notebook application is due to be released soon. Here’s a preview:

Tags: notebook applications, notebook G, notebooks, snapbits, Yahoo Search Pad, Yahoo!, Yahoo! Inc.

People behind sites – an insite guide

Looking for strategies to find out about people behind sites? Dozens of ‘Whois’ lookup sites exist with many offering similar tools. Instead of simply listing the choices, this post looks at some of the best tools for the job, how they differ and how to use them to find nuggets of information about a target site and its owner.

Many of these tactics can save you time by alerting you to linked content and commentary.
If you are using Whois lookups to trace a domain or site owner then every clue may be vital.

To illustrate these tactics I’ll look at how these tools can be used to get background on the fake site: www.genochoice.com or Dwayne Medical Center which claims to have pioneered ways to ensure a child’s genetic health.

1: Quarkbase. One of the best all-purpose sites for summary and detailed information about a domain is Quarkbase which compiles site profiles covering:

• snapshots and summaries;
• summary contact information including site owners;
• related sites;
• popularity gauged by references on social bookmarking sites, Twitter and inbound links from wikipedia;
• popular pages;
• top feeds;
• traffic analysis (including by country, subdomain and pageviews);
• associated people; and,
• inbound links from blogs and news sources.

Quarkbase displays a large snapshot of the Genochoice homepage with summary information that reveals the site contains ficticious information. The site’s traffic is too low to register but it does reveal inbound links from delicious and Twitter. Quarkbase also carries links to several blog posts about the parent site – RYTHospital.com – providing clues about the site’s history and development. Interestingly, some of these posts don’t appear to question the RYTHospital site content – underlining the need for a healthy reliability radar.

2: Domain Tools. ‘Whois’ lookup sites offer more focused information specifically on the domain. Use these to identify site or domain owners.

For example:
www.whois.sc – obtains the following information:
• traffic analysis by country;
• IP address;
• IP location;
• Whois record including registrant and registrant type, address.

Domain Tools tells me the site’s IP number and that Virgil Wong is the domain name registrant. His address is given as a PO box somewhere in Herndon, Virginia.

Whois lookups obiously don’t always identify the domain owners you are trying to trace. Domains can be registered through third parties (such as site designers) and domain owners can choose to remain anonymous. When this happens you need to deploy other tactics. For example:

• Reverse IP lookup – allows you to see what other sites are hosted on the same server. By doing this you can sometimes make connections between sites and between owners.
• Registrant search – allows you to find the other domains owned by the registrant; another great way to make connections between sites.

Domain Tools will tell you if that information can be obtained, but it will come at a price. There are, however, a few other places you can go to find this additional information:

3: Webhosting.info – offers a free reverse IP lookup. Find out what other sites are located on the same server. By carrying out a lookup using the IP address for Genochoice we find that the site is located on a server hosting several thousand other sites – indicating that it is a commercial hosting service. Sometimes, however, you can make interesting connections when only a small number of sites are hosted on dedicated servers. See Webhosting.info

4: Dialog (http://openaccess.dialog.com/ip/forms/PTCDomainNames.html) Allows you to carry out a registrant search but it comes with a health warning. The tool does enable you to track down all domains owned by specific people and companies but the database this tool uses was abandoned in 2004. The site, therefore, becomes less and less valuable as time passes. With ‘Virgil Wong’ we strike lucky. He has – in the past – registered a few dozen domains including malepregnancy.com, genochoice.com, virgilwong.com and paperveins.org.

If Dialog can’t help with a registrant search, but Domain Tools has told you that the registrant owns other domains, then you may have to resort to buying that information from Domain Tools. The more domains someone owns, the more expensive the information will be.

By following up the leads obtained from these tools and linked content we find that Genochoice is the creation of multimedia installation artist Virgil Wong who created RYTHospital several years ago in the form of an exhibition and website. His work is described on paperviens.org. It is possible to do all of the above without visiting the target site.

For basic and fairly complex lookup tasks, those four tools offer a rich profile and dozens of potential leads to follow-up. Insite will cover other domain lookup tools soon.

UPDATE: See my new post on WhoIsHostingThis for a tool that can more accurately pinpoint a site’s real host.

Tags: domain lookup tools, lookup, registrant search, registrant search tool, target site, whois

Oops. Google flags all sites as harmful.

Earlier today Google responded to every search with the message ‘this site may harm your computer’. Google has apologised – with the official line that the mistake was caused by human error. The company said the glitch lasted for 40 minutes – which, in my view, is quite a long time for the world’s biggest and most influential search engine to remain effectively useless. The company has apologised to the owners of sites that were incorrectly labeled as ‘harmful’. It will be interesting to monitor how the company handles the inevitable criticism.

Thanks to Henk Van Ess who alerted me to this problem through his comments on my post on Icerocket.

Tags: google, Google Inc., Henk Van Ess, Search Engine

Icerocket skates in where Google fears to tread

If, like me, you’re wondering why you can’t do a ‘Twitter Search’ from the big two search engines then John Battelle has a few theories. To you and I it might make perfect sense for Yahoo! and Google to include a Twitter search but, it seems, they are both willing to sacrifice functionality because they are reluctant to endorse a potential competitor. ‘Fascinating’ if you’re an industry observer; rubbish if you want a search engine to do the obvious.
That’s where Icerocket steps in. As Phil Bradley noted this week, the very nice Icerocket search engine has recently added a Twitter search option. But perhaps even more interestingly it has added a ‘Big Buzz’ option that pulls in very recent results from blogs, Twitter, Video, News and Images. It even gives you an ‘auto refresh’ option to update those results every minute or so as you are working on a story.
As Phil Bradley notes: ‘the whole area of news and social media is one that is seemingly passing Google straight on by.’ More on Icerocket soon.

Tags: favourite search engine, google, Google Inc., John Battelle, Phil Bradley, Search Engine, search option, social media