Making the most of RSS: readers reviewed

By Colin Meek (@colinmeek) and Judith Townend (@JTownend)

Reports of the “death of RSS” are not only greatly exaggerated – they are simply wrong. Not only is RSS part of the fabric of the internet, tapping that resource through RSS Readers is still an important and valuable component of a researcher’s toolset.

As a quick re-cap, RSS (Really Simple Syndication) readers allow you to track the content feeds from different websites. In other words, you can monitor a number of sites without visiting those sites individually. Use an RSS reader to work more efficiently – that’s the theory.

The practice is often disappointing however. People often find RSS readers (and web-based readers in particular) frustrating and difficult to configure. If you subscribe to a range of general feeds (such as BBC news feeds) and content-specific feeds you can feel overwhelmed by the tsunami of posts that appear. Subscribe to several dozen feeds and you quickly need some way to filter and focus the content further.

Of course, Twitter and email alert services can be used in conjunction with RSS, but think of Twitter as the live broadcast, and RSS as the TV catch-up and what’s on guide. No matter how long you’ve been offline, it’s really easy to see what new content has been published on your chosen sites.

If you do need to monitor many active feeds then you need to choose a reader that can filter out the noise so you can monitor posts for specific keywords, categories or tags. If you’re determined to see everything written about a subject, follow the results of a specific search term by RSS – it can be easier to track than Google alerts to your email inbox, for example.

In this post, we review a number of different RSS readers that will help you stay on top of a range of feeds from a number of sources.

Web-based: Google Reader

Google Reader is a fantastic basic reader to collect feeds, but can be infuriating to use as your main tool. It can slightly lag and is a little cumbersome for organising your feeds into different folders.

It has its advantages however: it’s easy to add feeds to Google Reader from your browser and you can sync it with another desktop or mobile reader. It’s easy to publicly share stories with your followers, and send stories to other social networks, such as Twitter, StumbleUpon and Tumblr, for example.

Another nice feature is that Google recommends feeds to you. The recommendations are automatically generated and take into account your existing feed subscriptions.

Items can be tagged and ordered fairly easily. Google has a “magic” sort, which is its attempt to put your feeds in the “most relevant and interesting order”. You can view feeds by ‘list’ or ‘expanded’ view.

Here’s how Google Reader looks in your browser:

The other reviewed readers can be synced with your base Google Reader. The advantage of this is that you have a web-based back up of your feeds and it’s easy to switch news aggregator when something better comes along. But the big plus for doing this is that you can view your feeds from wherever you are – on phone, laptop, work and home computers – and they’ll all be synced.

There are a number of Google Reader bookmarklets to pop in your browser bar for quick read, subscription and share options.

Additionally, there are a number of  shortcuts you can use while in Google Reader. See below:

Finally, for a bit of fun, have a play with Google’s ‘Reader Play’. Perhaps not the most efficient way of scanning stories, but it could be a pleasant way of flicking through your reader while you’re eating your breakfast:

Google Reader does not have more sophisticated filter options yet, but its forum suggests using an external service like FeedRinse for creating filters within your different feed folders. Feedrinse is an effective way to filter your feeds but if you have to monitor a range of feeds for various terms it is definitely not the most elegant solution.

So, which readers do offer an easy filter feature that are easy to configure?…

Desktop: NetNewsWire

For Mac, we recommend NetNewsWire. One big benefit is that it’s fast loading and easy to scroll through.

You can bookmark pages on the Delicious service, or by flagging or adding them to a clippings folder.

If you wish, it can order feeds by attention and unread count, with the aim of bringing the most important items to the top.

It has a “tabbed browser feature” which means you can click on an article to view it and then click back to continue scrolling through the next stories, without losing your place.

You can create “special subscriptions” which let you search tags on Flickr, or Delicious, or search terms on Twitter and Yahoo – giving you an integrated way to monitor social media and  your RSS feeds from one interface.

Smart folders are also handy: they organise items from your feeds according to pre-determined rules. For example, you can specify a specific keyword that must appear in the title and NetNewsWire will show items that fit this rule in the relevant Smart List. But there are other criteria you can specify for items as well: by author, link or its ‘read status’, for example.

Here’s a bit more detail. First, select the ‘New Smart List’ option under the ‘File’ tab.

Next, specify which key words you would like to filter. You can create more filter boxes by using the ‘+’ option on the right hand side. In the example below we’ve added four extra filter boxes:

There are several options for filtering as you can see below. It’s useful to specify the group name of a set of feeds (eg. ‘photography’) and within that, stories that match a specific search term (eg. title contains ‘police’).

The end result? A feed with all the stories matching those conditions.

Another option for Macs is NewsFire which has an uncluttered and intuitive interface but less functionality.

As a Windows equivalent, FeedDemon works very well. It has some really nifty features, which make it especially good for research purposes.

You can create Search Feeds that tell you when a keyword appears in any feed; whether it’s one you subscribe to or not. You can manually tag items by keyword and then view by tag.

It also has filtering options, called ‘Content Filters’ but these are only available in the paid for Pro version. However, in the basic free version you can set up Watch alerts for when certain keywords appear in any feeds you’re subscribed too.

Resulting in a feed like this:

Mobile: Reeder

We tried out a few readers for iPhone, but found Reeder to perform really well. It syncs efficiently with Google Reader. The bottom navigation allows you to jump between ‘starred’, ‘unread’ and ‘all items’.

It’s a very simple interface, but for reading RSS by your phone that’s all you really need. You can easily scroll up and down stories to read and there are a range of sharing options via social media and also quick functions for copying and emailing links, saving to ‘pinboard’ or making a note.

Of course, they are plenty of alternative RSS readers and aggregators to try out too. You can find some more suggestions on this list provided by BBC News. Download software and find even more via CNet at this link. Wikipedia has a very comprehensive list here. We’ll be reviewing more in the near future.

Tags: feeddemon, filter tool, filter tools, google reader, net news wire, reeder, rss readers

Top five advanced research tools for Twitter

By Colin Meek (@colinmeek) and Judith Townend (@JTownend)

In our recent three part series we tested the best tools for researching Twitter. If you are engaged in serious research on topical issues you can’t ignore Twitter. And, if you can’t ignore Twitter, then you need to grasp the best ways to tap the resource. Of all the techniques we tried out, these are the ones we rated as our top five:

1. It’s an obvious one, but TweetDeck really is indispensable for us. Not only does it help manage conversations, the powerful Twitter monitoring desktop client allows you to monitor multiple topics, keywords, lists, people or hashtags in columns.

2. Twiangulate provides a very intuitive way of analysing networks and connections.

3. It has its flaws, but Twitter Search ranks pretty highly for us – especially once you’ve got your head around the various advanced operators.

4. Listorious, with a database of more than 2 million Twitter users allows you to search for  Twitter users by topic, keyword or Twitter lists.

5. Tweetscan is a simple, but sensible way to back up your tweets for offline use. It doesn’t catch them all – your last 1,000 messages – but it then combines them with results from its index, going back to 2007.

• In part one of our Advanced Twitter research series we looked at Twitter search: read at this link.
• In part two we examined the best ways of finding people to follow: read at this link.
• In part two we  showed you the best ways to track hashtags and archive your tweets: read at this link.

** Learn more about sophisticated search techniques on our one-day Advanced Internet Research course, Tuesday 5 July in London.**

Tags: advanced twitter research, Online social networking, search techniques, Technology/Internet, tweetdeck, Twitter, Web 2.0

Twitter advanced research techniques 3: tracking #hashtags and archiving tweets

By Colin Meek (@colinmeek) and Judith Townend (@JTownend)

Our third post in the Insite series on Twitter research looks at how to follow Twitter trends marked with #hashtags. If you’re reading this you have probably already clocked how useful #hashtags are, allowing you to isolate specific topics on Twitter.

This is how Twitter defines them:

The # symbol, called a hashtag, is used to mark keywords or topics in a Tweet. It was created organically by Twitter users as a way to categorise messages.

Sometimes they’re using #inajokingmanner to no real purpose, but most of the time they allow you to follow a category of Tweets more easily – about a news story, or an event, for example. The most popular #hashtagged topics will appear in your Twitter sidebar as trending topics (advertisers can pay for their ‘promoted’ topic to trend at the top).

Hashtags are effective advertising and marketing tools, as well as useful aids to follow conversation, live events and breaking news.

• You can follow a specific keyword, using Twitter search. For example, if you’re interested in what people are saying about the television programme Newsnight: http://search.twitter.com/search?q=%23newsnight

• If you click on a #hashtag in any tweet on your Twitter homepage, you will see all the tweets on that topic. For example: http://twitter.com/#!/search?q=%23libelreform

This article on Twitter media examines what makes a hashtag popular. Maybe no surprises to learn that in the main it’s repetition, repetition, repetition. “You need to see a hashtag four or five times before it really clicks,” says the report but not too much: there’s a “sweet spot” after just a few.

For a bit of Twitter history, have a look at the first ever hashtag on Twitter. The ReadWriteWeb reported that according to serial social technology innovator Chris Messina it was #barcamp. It’s logical that a tweet about a technology meetup would be the first topic on Twitter, before it got taken over by the journalists, the spammers and the celebrities.

Tools for tracking hashtags

1. TweetDeck: is a simple, intuitive and immensely powerful Twitter monitoring desktop client that you can download for free. Many, many people admit to ‘not getting’ Twitter until they use TweetDeck. The beauty with TweetDeck is that you can monitor various topics, keywords, lists, people or hashtags in columns. New posts slot in at the top of the columns to create a cascade of posts on your topic. Here TweetDeck is monitoring the Scottish election hashtag #SP11, posts on Syria using #syria and the journalism.co.uk user news feed:

2. Hashtags.org: This site allows a simple tag search, showing you trends and recent tweets on topics. Or go directly to trendistic.com, which supplies the graphs.

3. Twitseek shows you the trending topics of the day, split into different sectors: celebrities, sport, news, stories, people (although topics could fall into more than one category on that list) and includes the URL of the tweets.

Finding old trends & archiving tweets

One problem with Twitter search is that tweets drop out in time – as you’ll see if you go to look for one of your past tweets.

Tweetscan claims to query both Twitter and its own index going back to December 2007 to provide, what it says (it would!) “the most complete and convenient backup available”.

It also allows you to backup your Twitter account if you give it access to your account, a very handy tool, if you have comments and @replies you wish to use in the future, for research or commercial purposes.

“Your last 1000 messages are downloaded from Twitter and combined with results from our historical database. Tweet Scan can retain your tweets to improve later backups.”

Tweetscan also allows offline search and storage.

Alternatives to Twitter Search

Dave Larson has an excellent list of Twitter search engines here, which includes: Searchtastic;SnapBird; TwimeMachine (browse up to 3200 of a user’s old tweets); Topsy; The Archivist (Windows only or browser version); TweetBoard (private alpha); BackTweets (subscription required for full access); FriendFeed; and http://research.ly/ (subscription required, from $9-$99).

Topsy is particularly handy because it notifies bloggers when someone has tweeted their story, via a pingback. But it’s Searchtastic and SnapBird that Larson considers the best for your “Twitter arsenal”.  Searchtastic has one particularly useful feature: you can back up tweets and export to a spreadsheet.

Read Larson’s post in full for a more detailed explanation of each engine and see his chart comparing their functionality here. And if you’re still thirsty for more, Larson has listed 30 more Twitter search options at this link. Enjoy!

• In part one of our Advanced Twitter research series we looked at Twitter search: read at this link.
• In part two we examined the best ways of finding people to follow: read at this link.

** Learn more about sophisticated search techniques on our one-day Advanced Internet Research course, Tuesday 5 July in London.**

Tags: advanced twitter research, hashtags, Online social networking, Real-time web, Technology/Internet, Twitter

Twitter advanced research techniques 2: finding the right people

By Colin Meek (@colinmeek) and Judith Townend (@JTownend)

This second post in our series on Twitter research tackles tools and tips for finding the right people and networks. We also look at some of the most high profile Twitter services that offer search services and other functionality. But, first a word of warning. New Twitter tools are popping up all the time. This a fast changing area and new services can be abandoned by developers. Don’t rely on one tool and keep an eye out for alternatives as they come along.  Additionally, don’t forget Twitter’s own tools, which are frequently updated.

Simple Twitter User Finder

For simple topic searches Listorious is a good start. It has a database of more than 2 million Twitter users and you can search for  Twitter users by topic keyword or Twitter lists. Here we used the search phrase ‘oil spill’.

Network research and analysis

To tap the real potential of Twitter, however, you need to make connections between people. This is important for many reasons: to profile people accurately; to discover new contacts; to make unexpected connections between individuals and groups; and, to identify influential people within networks.

One of the best and most intuitive tools to help you with this Twiangulate and here’s how you can use it to analyse networks and connections.

Firstly, you can map connections to create a useful visual network of Twitter users. You can either identify a specific Twitter list or you can add as many Twitter users as you like.

Twiangulate then automatically creates a ‘map’ of the network and how they connect to each other. In this example we ‘mapped’ a list of Twitter users related to the Gulf oil spill.

Each user in the network is identified on the map and the value allocated to each member indicates the percentage of other members they are linked to. When you hover your cursor over individuals in the map you pull up profile information for that member including recent tweets and bio information. For a quick and visual way to get to grips with a specific network this tool is excellent.

Twiangulate also gives you the power to look at a Twitter user’s ‘inner circle’. There is nothing interesting about whether someone follows Stephen Fry. The Twitter contacts that are far more interesting are the users who have a more modest following. These people are more likely to have firmer connections to the user you want to profile. Twiangulate calls this their ‘under-the-radar’ service because it serves up the 100 friends with the fewest connections followed by specific users. Those Twitter contacts are more likely to be close colleagues, friends or business associates.

In this example we have looked at two high-profile BBC journalists. In addition to this visual, Twiangulate lists their 100 friends with the fewest contacts. It also highlights those common to both.

Another way to find connections is to trace mutual friends – the people who two or three Twitter users follow in common. Use this to get to the  heart of particular networks and identify influential people in groups. In this example (below) we looked for people followed in common by two BBC journalists and the science writer Ben Goldacre. We found that one user is followed by all three while at least two of the users we included in our analysis share hundreds of friends. The more the friend lists overlap – the closer you are to the heart of a network.

Another way to analyse networks is to find mutual followers – who follow the same two, three or four people. These people have made common ‘follow decisions’ which suggests they have a well-defined focus of interest. In this example we selected three journalists and commentators on video gaming and games development.

We found that 86 Twitter users follow all three of these people. Obviously, if you identify three important resources on Twitter then people who follow all three may also be important targets for your research. Hence: ‘twiangulate’.

Tools to help you find tweeps

LocaFollow: A fairly high-profile service promising to help you find local Twitter users in small towns, using Google search. The immediate thing to be aware of with this tool, as with many others, is that you need to give it access to your Twitter account for full functionality. Users are increasingly becoming more aware of security issues associated with third party access, so think carefully about which services you use.

So, what does LocaFollow do? The service aims to help you search for Twitter handles with certain characteristics, eg. location, keyword in bio and then follow them from that page.

However, the site seems to have reached a hiatus. The last blog update was in July 2010.

The problem here is that you can do your own advanced keyword searching to identify the right people without LocaFollow. If you are familiar with Google’s advanced operators you can search for keywords in Twitter bios and specific locations. For example this search: site:twitter.com “bio * doctor“ (the operators are in red) typed directly into Google’s search field returns hundreds of thousands of Twitter users with the word ‘doctor’ in their bios.

This similar search: site:twitter.com “location * edinburgh” finds people who specify their location as Edinburgh. For serious research, other tools do a better job than LocaFollow.

Tweet Adder is another high profile Twitter tool which is advertised at the bottom of the LocaFollow page and you have to pay for everything other than the free demo.

And, again, for functionality you’ll have to give it access to your Twitter account.

We tried the free demo. It’s pretty nice. It allows you to search Twitter users by various factors: eg. topic/location and brings up a list which you can check and create lists from. You can then ‘follow’ by bulk. There’s also an option to search profile data and the followers of users and lists.

The ‘tweet generator’ tool, however, probably isn’t going to pass muster for a journalist. There’s some things that should be done by a human. Plus, we couldn’t actually get it to work in the demo. The feature for tweeting from an RSS feed is likely to be more appropriate for journalists and researchers.

Overall, Tweet Adder is much more intuitive than LocaFollow.  But its primary role is not research. Instead, it is designed to help you get followers. Whether it’s worth the money to a researcher or journalist is the key question, especially when there are so many excellent free tools available.

Other tools

Twitscoop offers a range of services such as tag and trend monitoring in real-time. Like Tweet Adder, it also offers tools to monitor your follower activity, once it has access to your account. Here we have logged in with an account:

It shows trending topics, as well stats for certain tweets. In the example above, it showed us that @bengoldacre’s link had received 872 clicks. It uses Zobmark to archive and organise your Twitter links – this is a service that collects your links when you tweet #zob with your message.

However, when we tried out Zob, our tweets didn’t show up and it doesn’t look like it has a very big user base. Plus, Zobmark’s own bookmarks haven’t been updated since 27 January.

With bookmarking tools like Delicious and Diigo for storing tweets and services like FriendFeed that capture your tweets, the benefits of this service aren’t really obvious – plus you have to stick an odd looking hashtag on all your content.

Moving onto the next, Twitstat. To have any functionality at all with Twitstat you need to be logged in. Once in, it’s pretty ugly.

Looks aside, it lets you search a Twitter user, establish your relationship and view lists and see your retweets (by you, and your tweets retweeted) from your home page. Not much you couldn’t do on the Twitter page.  Plus, Twitter is prettier.

Tweet Adder, LocaTweet, Twitscoop and Twitstat are better as marketing rather than tools for serious or effective research. For the latter, we need to look elsewhere.

In our next post in this series we’ll examine #Hashtags and Trend Monitoring.

** Learn more about sophisticated search techniques on our one-day Advanced Internet Research course, Wednesday 16 March in London. Book soon though as only two places available at the time of writing. **

 

Tags: advanced online research, Advanced Techniques, advanced twitter research, Listorious, real-time search, search techniques, Twiangulate, Twitter, twitter searching

Twitter advanced research techniques 1: searching twitter

By Colin Meek (@colinmeek) and Judith Townend (@JTownend)

If you are engaged in serious research on topical issues you can’t ignore Twitter. And, if you can’t ignore Twitter, then you need to grasp the best ways to tap the resource. Understanding how Twitter is used is key to getting what you want without ending up knee deep in posts about coffee breaks and late trains.

Pivotal to success is understanding that the information carried within Twitter is often not the content with value. The real value for the serious researcher are the networks of people you can access, the connections that exist between them and the links people post. You can also locate those posts in time and place.

Like ordinary search engine research – you need to be precise and have a clear idea what content you are after and where it is likely to be found. For example, the screen grab below shows results from a search for posts originating in Cairo before 27 February.

Tactics like this are easy to master. In this insite series we’ll group tactics and tools into a range of research categories including: Finding the right People; #Hashtags and Trends; and Searching the Twitter Archive. This first post explores Advanced Twitter Searching.

Searching Twitter

A lot of confusion exists about advanced searching in search engines and services such as Twitter. Most, including Twitter, offer an ‘Advanced Search‘ form. This gives you clearly useful tools to help you interrogate the Twittersphere. But more useful, flexible and powerful are the ’advanced operators‘ that are typed directly into Twitter’s search box and give you the power to construct much more precise and powerful search strings. This guide explains how to get to grips with these tools. See the insite guide to Google’s advanced operators.

The Twitter guide to each advanced operator is set out below with links to the corresponding search results. The ‘operators’ are highlighted in red.

Operator	Finds tweets…
twitter search	containing both “twitter” and “search”. This is the default operator.
“happy hour“	containing the exact phrase “happy hour”.
love OR hate	containing either “love” or “hate” (or both).
beer -root	containing “beer” but not “root”.
#haiku	containing the hashtag “haiku”.
from:alexiskold	sent from person “alexiskold”.
to:techcrunch	sent to person “techcrunch”.
@mashable	referencing person “mashable”.
“happy hour” near:“san francisco”	containing the exact phrase “happy hour” and sent near “san francisco”.
near:NYC within:15mi	sent within 15 miles of “NYC”.
superhero since:2011-02-26	containing “superhero” and sent since date “2011-02-26″ (year-month-day).
ftw until:2011-02-26	containing “ftw” and sent up to date “2011-02-26″.
movie -scary :)	containing “movie”, but not “scary”, and with a positive attitude.
flight :(	containing “flight” and with a negative attitude.
traffic ?	containing “traffic” and asking a question.
hilarious filter:links	containing “hilarious” and linking to URLs.
news source:twitterfeed	containing “news” and entered via TwitterFee

[Source: Twitter.com]

Operators such as the ‘OR‘ command or double quote marks to designate phrase searching are common to many search engines and can be extremely useful when treated carefully. Turning to the Twitter-specific operators – some are more important for serious research than others. Here are real examples from the best of the bunch.

The ‘people’ operators let you trace posts ‘from‘ and ‘to‘ specific people. Here we used the search term: from:blibrahim to obtain all the recent posts from that user in Cairo.

You can use the ‘to‘ operator to trace all posts sent directly to that user and the @ operator to obtain posts that reference a specific user.

The ‘location’ operators help you identify Twitter users geographically. At the height of the uprising in Libya we used this search to find Twitter users in Tripoli: near:Tripoli

You can use the ‘within‘ operator to search for posts ‘within’ a certain distance of a specific location. For example, your search string can be: near:tripoli wthin:25mi to search within 25 miles of that city. The Twitter search page also gives you drop-down options for searching ‘within’ specific distances.

You can search for answers by searching for questions using the ? operator. Use this operator to find the questions other Twitter users are asking about a specific topic. For example, this query: Ajdabiya ? was used to find posts about that town on the day it was bombed by pro-Gaddafi forces.

Finding links. Discovering other resources and breaking news about a search term is one of the main reasons researchers and journalists turn to Twitter and the best way to do that is to use the filter:links operator. This looks for posts that contain your search term and internet links. For example, here we used this search string: Ajdabiya filter:links.

Searching within specific time periods. To search for post up to a specific date use the “until” operator like this: until:2010-02-28. Equally, you can search for posts after a specific date like this:since:2010-02-28. This is particularly effective when you want to examine the reaction to a specific event before and after it has occurred. Searching by timeframe is not always reliable however. If you use a common search term then Twitter may tell you your specified date is too old.

Combining operators. As with Google, the most powerful way to use advanced operators is to combine them in innovative ways to get to the material you want with the minimum of fuss. In this example, we searched for posts that contain the term “tahrir square” and originate in Cairo. The results contain leads that are exactly on topic.

The next post in this series will look at ways you can identify the right people who are using Twitter (tweeps)  and how you can identify networks and connections between people.

** Learn more about sophisticated search techniques on our one-day Advanced Internet Research course, Wednesday 16 March in London. Book soon though as only two places available at the time of writing. **

Tags: Advanced Techniques, advanced twitter research, Real-time web, Technology/Internet, Twitter, twitter searching

Spotting manipulated photographs

Guest post by Judith Townend

The fascinating field of digital forensics couldn’t be more relevant for a journalist, researcher or editor. Specialised analysts can test the authenticity of a photograph: how many times has it been saved? Have additions been made to the original, and if so, in which order? Are parts of the image generated by a computer?

Professor Hany Farid, based at the Department of Computer Science at Dartmouth College in the US, explains that photo forensics refers to mathematical and computational techniques “that seek to determine if a photo has been altered from its time of recording, and how/where the photo was altered.

“This field of study is highly technical, and while some simple analysis can be done without much technical training, most of the forensic techniques require a highly skilled practitioner.”

What sort of things should we be looking for? “Lighting and shadows can be a powerful clue to reveal tampering,” says Professor Farid. “However, great care needs to be taken, as the visual system can often be quite unreliable in judging lighting and shadows in an image.” (See, for example, his co-authored paper on ‘Image Forensic Analyses that Elude the Human Visual System’, Farid and Bravo 2010).

“JPEG compression artifacts are often confounded with tampering, so it is important to understand what these compression artifacts look like, so as not to confuse them with tampering artifacts.”

Professor Anthony TS Ho, from the Department of Computing at the University of Surrey, agrees that this makes analysis difficult. His research on image authentication examines watermarking and how it can survive manipulations such as JPEG compression.

Information, including copyright, source and time stamp, can be embedded in digital material. Different parts of the image can be watermarked in order to detect changes, such as altered digits on car number plates.

(Image: The Plate Market on Flickr)

Researchers in Professor Ho’s multimedia security and forensics group are also experimenting with applying mathematical algorithms based on natural laws, such as Zipf’s Law and Benford’s Law, in image forensics.

Professor Ho warns that some tools may only detect changes in a high-level quality image – those saved around Quality Factor (QF) 90 to 95. Images saved in Photoshop will be at QF=75 by default. Ongoing research will try and find ways to increase accuracy when analysing lower quality images.

So what tools can be used? Standard EXIF readers can be useful for examining an image’s metadata, says Professor Farid. “This can be useful in determining if a digital image was modified from its recording since most photo-editing software alter the metadata.”

But, it is also the case, he adds, “that the metadata can be re-edited to conceal these changes, so care should be taken in making very strong conclusions based on an image’s metadata.”

Internet resources

Online tools and non-academic information are scarce. One site, ‘Error Level Analysis’, claims that it can help determine whether a photograph has been manipulated from the original. Its premise is simple: paste the URL for a image file in the box and ELA will indicate how the photograph has been treated.

The ELA site’s creator explains on the site:

“It works by resaving an image at a known quality, and comparing that to the original image. As a jpeg image is resaved over and over again, its image quality decreases. When we resave an image and compare it to the original, we can guess just how many times the image has been resaved. If an image has not been manipulated, all parts of the image should have been saved an equal amount of times. If parts of the image are from different source files, they may have been saved a number of different times, and thus they will stand out as a different colour in the ELA test.”

The key is to look for different levels of brightness in the photo. We did a quick test for ourselves with a simple photograph. This is how the original looked when put through ELA. The top picture is the original, the bottom shows the image under ELA.

The we drew an arrow and circle on the image. See how it stands out brightly when put through ELA.

The tool is based on Dr Neal Krawetz’s image forensic work. Dr Krawetz is quoted on the site:

“Error level analysis (ELA) works by intentionally resaving the image at a known error rate, such as 95%, and then computing the difference between the images. If there is virtually no change, then the cell has reached its local minima for error at that quality level. However, if there is a large amount of change, then the pixels are not at their local minima and are effectively original.”

Dr Krawetz’s methods have received some coverage in the tech press, but not without some debate: see the comments and updates on this Wired post, and also the addition to this Cnet story.

As image analysis tools are developed, it would be sensible for editors and reporters, as well as researchers, to keep track of them. It might help avoid disastrous mistakes, such as the time Reuters ran a doctored image of an Israeli air strike in Lebanon. But they should take care when using a site like ELA. Firstly, there is limited peer review available. Secondly, the final image on ELA comes with a fairly important disclaimer (our emphasis):

“It is worth noting that edges and areas red in colour are often depicted as brighter in the ELA tests. This due to the way the photos are saved by various programs. It is not proof that image was manipulated. If you are unsure how to interpret the results, please do not claim the results of this tool as proof of anything.“

Even when the image shows different levels of brightness, the final results are tricky to analyse. As users in this Flickr forum discussed, a photograph saved to Flickr may show different results from the original file, even if the picture itself had not been tampered with. ‘Muzzlehatch’ speculates this is because of the Flickr compression process.

Compare and contrast this photo we tested. This shows the same unaltered image we used earlier, but this time as a jpg file uploaded to Flickr.

ELA may not be a solution for detecting manipulated photographs but it provides an important reminder about the need for a little detective work before accepting an image is what it appears to be.

While we still need experts for thorough analysis, don’t forget that basic observation can be used to detect suspicious things too.

1. Look out for duplications

In this famous example of a digitally altered scene of a British soldier and Iraqi civilians, featured on a Los Angeles Times front page, a duplication was spotted in the final image, which turned out to be composed from two separate images.

2. Look at the background

The new Photoshop Content-Aware Fill tool could lead to some very clever alterations. This video demonstrates the tool in action.

3. Ask basic questions about the circumstances of the photograph

Use your common sense to detect fishy details in photographs. Doctored photos were around a long time before Photoshop: what other features might have been engineered for the image?

4. Remember that some alterations might be considered acceptable in certain circumstances

See the Reuters Handbook of Journalism for the Dos and Don’ts of photo editing for journalistic purposes. Different organisations might draw lines in different places, too. In this blog post, James Estrin, of the New York Times Lens blog, describes his orthodox view of Photoshop use for journalistic purposes. “Less is more,” he says.

Extra resources:

• Reuters’ Handbook of Journalism, Photoshop guidelines
• How Little Green Footballs discovered doctored photographs from Beirut
• eHow.com: How to Detect Faked Photos
• Photopreneur: World’s Most Infamous Staged Photos
• Photopreneur: World’s Most Famous Photoshop Fakes
• PC Pro: Spotting the Photoshop Fakes
• Washington Post ‘Photoshop Gone Wrong’ gallery
• PSdisasters.com
• Digital Forensics: 5 Ways to Spot a Fake Photo
• Digital Forensics: Altered Lance Armstrong Photo Explained

Judith Townend is a freelance journalist/researcher and a PhD research student at City University London’s new centre for law, justice and journalism.  She was formerly a reporter for Journalism.co.uk where she wrote about digital tools and techniques for journalists. She is interested in seeking out new online research methods for both journalism and academia and is @jtownend on Twitter.

Tags: Adobe Photoshop, error level analysis, image analysis, image forensics, Photo manipulation

Advanced Internet Research – next scheduled one-day course

Colin’s next one-day intensive course in Advanced Internet Research will be in Central London on March 16th.

Tags: Internet Research

Four #real-time search options leave Google behind

While there is no sign of panic yet, there is certainly evidence that established web giants such as Google and Facebook are frantically playing catch-up as the clamour for real-time search grows. Not only are there a range of tools to monitor Twitter in real-time there are browser add-ons that enable you to add Twitter search results to your Google results page. Why is Google being so slow to grasp the demand for real-time search?
The bottom line for journalists is that Google no longer has the best answer to the simplest question: ‘What are people saying about [my query] right now?’ For those of us steeped in Google search experience – it’s a scarey thought. Google is, apparently, working on a real-time offering but, in the meantime, are there competitors to the real-time results available from Twitter and the various Twitter monitoring tools available?
Here’s a heads up on four tools that don’t rely exclusively on Twitter and offer real-time monitoring for your search queries.

1 Collecta, just a few hours old, has the cleanest and most intuitive search page. Your search term is tracked and results listed in a central column. Clicking on those results gives you fuller content and a link to the source. The left hand column lists your recent queries and gives you the chance to include blogs, blog comments, images and updates (tweets) in your results. Collecta also tracks the time since your query. I’ve had impressive results so far.

2 Scoopler, has another crystal clear search page with the main section split between ‘real-time’ search results and ‘popular’ results that include content from news sites and videos. The ‘real-time’ results are drawn from Twitter, Digg, Delicious and other networking sites. This middle column gives you the options of previewing the posts or you can click through directly to Twitter profiles.

3 OneRiot, claims to do the same as Scoopler and Collecta but when I tried its ‘realtime’ search for results containing ‘tehran’ and ‘iran’ there weren’t a convincing number and they failed to appear with anything like the frequency I would have predicted. On the plus side, you can use its ‘pulse’ search option which uses an algorithm that looks at dozens of factors to give “weight” to certain results. OneRiot has used various factors to influence the weighting including: freshness; source credibility; and, ‘acceleration’ whereby posts that are gaining momentum (links) on the web are ranked as more important.

4 I looked at Icerocket in January. Not only has it set its blog search as its default option, it has also added a twitter search and a real-time ‘Big Buzz’ alternative. Big Buzz pulls in very recent results from blogs, Twitter, Video, News and Images and gives you an ‘auto refresh’ option to update those results every minute or so as you are working on a story.

I am @colinmeek on Twitter

Tags: blog search, collecta, crystal clear search page, Facebook, google, icerocket, intuitive search page, oneriot, real-time, real-time monitoring, real-time results, real-time search, scoopler, search experience, search option, search queries, search results, search term, Twitter, web giants

Google’s advanced operators for journalists

Master commands for precision surfing. Presentation.

Confusion is rife about how and when you can use Google’s advanced operators. Used effectively they can transform your research by helping you get better results faster. Here’s my recently updated presentation on advanced operators with some context and example results.

Tags: advanced, google, journalism, operators, search, techniques

Twitter monitoring tools

During a couple of recent courses delegates have been keen to know more about real-time twitter monitoring tools. So here are a few options:

Tweetdeck is a free download that lets you monitor posts in real-time as well as send tweets directly. Perhaps its best feature is the ability to create columns so that you can track different terms using your own dashboard.

Twitscoop is a very simple tool that allows you to monitor specific trends and buzz around a topic. You can monitor the use of specific terms over the last few hours or days. It doesn’t let you compare keywords however.

Twhirl is another desktop application that you can download for free and then use it to monitor Twitter for specific keywords. You can also integrate this with your Twitter and other social networking accounts.

Tweetscan is a Twitter search tool that automatically refreshes the search every few seconds. This also lets you limit the search to specific users.

Twazzup is similar to Twitscoop but it also highlights related hashtagged terms, the most popular links, featured tweets, and singles out specific Twitter members as ‘trendmakers.’

Monitter also allows you to monitor specific terms as they appear in tweets but this tool allows you to monitor three in their own dedicated columns. Monitter also, very slickly, lets you monitor keywords within a specific distance of a specified location.

Update: See my posts on Tweetgrid and Tweetbeep for more on this.

Tags: search tool, social networking accounts, twazzup, tweetdeck, tweetscan, tweetscoop, twhirl, Twitter