Speaking at the International Journalism Festival in Perugia, these questions were posed by Pier Luca Santoro, a social media editor at La Stampa and founder of Data Hub, to a panel of experts on the issue of metrics.
"It's gone to this extreme now where people follow the pageview to the exclusion of everything else," said Gregary Galant, co-founder and chief executive of Saw Horse media.
Pageviews are often taken to advertisers as proof of a campaign's performance but pageviews offer no insight into the quality of content, or the impact it may have had on a reader, said Tony Haile, chief executive of analytics firm Chartbeat.If we're optimising for the link then no wonder we get link baitTony Haile, Chartbeat
"If we're optimising for the link then no wonder we get link bait," he said, "and people who create quality content get frustrated."
Provocative headlines and images encourage people to click but it does not mean they enjoy the content, said Haile, and the spreading of articles over multiple pages has also allowed many sites to boost their metrics, for example.
As a result, publishers are "left with a world of infinite inventory, when the marginal cost of additional page views is effectively zero" he said, and "in that world the value of advertising space will always decline."
So what alternatives are there?
Santoro suggested that, with the advent of social media, the amount of times a piece of content has been shared online could be both a quantitative and qualitative measurement of a piece of content's success.
"What makes that metric really interesting is that it's standardised," said Galant, "it's the same for any publication and it's public."
Pageviews are recorded internally and verified by independent organisations, while social shares are recorded by third parties and open for all to see.
"For the first time ever in the history of journalism we have a public, standardised metric," he said, but Haile questioned whether that truly solved the problem of measuring quality of content over a more quantitative value.
"The number of shares and the amount of attention do not sync up," he said, citing a recent Chartbeat study into audience habits on social media, that reported many people will share a story without reading it.
Andrea Ianuzzi, executive editor of the national contents division at Gruppo L'Espresso, said that it should not be a question of sharing or traffic however.
"Social media are not there to be distribution platforms," he said, "and if we keep thinking of them as such we will always be failing in some way."
A more accurate metric in showing how engaged readers are in articles or other types of content may be time, suggested Haile, as it gauges the quality of content and therefore the success of a news organisation.
"The fact is advertisers have been buying on time for far longer than they have on anything else," he said, referring to television or radio advertising, and whether it's for two hours once a day or 20 seconds five times a day, time shows value because "you're competing against the entire sum of human endeavour" in gaining someone's attention.Advertisers have been buying on time for far longer than they have on anything elseTony Haile, Chartbeat
"We've now established a common methodology," he said, "which is when you're actively attentive you're looking at the screen and doing something – scrolling, moving mouse or tapping," and Chartbeat have begun to measure the time readers spend on content as engagement.
This goes beyond being a monetisable metric for advertisers, he said, but means journalists get feedback into where the audience may be losing attention in a piece and therefore what elements work and which do not, an idea that can also apply to design.
"If what we care about is holding your attention as long as possible then any bad design decision has to go," said Haile, "because we want to make this a beautiful experience for you."
The use of time as a metric is not without its flaws though, said Galant.You're not just looking to generate entertainment but looking to illuminate issues and readers and give people information they can do something withGregory Galant, Saw Horse Media
"I spent as much time reading the first Greenwald article [on NSA surveillance] as many of the other, more trivial articles I read that year," he said.
One of the central ideas for news journalism is in providing a public service to the reader, by informing and educating them.
"You're not just looking to generate entertainment but looking to illuminate issues and readers and give people information they can do something with," Galant said.
"If we manage to give the right information in the shortest time, the user would have stayed a short time but received some important infomation," said Ianuzzi.
He admitted that impact is hard to quantify, but pointed to a number of examples from his organisation in which the number of social interactions and traffic had surprised the newsroom and changed how they placed the story on social media or the website.
"But what kind of standardised metric could we have for that?" asked Galant. "And how could we measure that using social media?"
Steffen Konrath, chief executive of Liquid Newsroom which specialises in 'real time web news', said the key metric for news organisations should be content's relevance to the reader, as it adds value to a relationship and "helps establish me as a trusted source" he said.
"If you've conducted primary research focus groups, if you know what people are interested in then you know what to focus on," said Konrath, a policy that would result in more traffic and better engagement with an organisation as the audience values the quality of its output.Quality is what makes people stick to a siteSteffen Konrath, Liquid Newsroom
"Quality is what makes people stick to a site," he said, stressing the relationship readers have with outlets as not only an important gauge of the editorial content but as something that could be monetised.
"You can monitor feedback through social media and understand people's interests or preferences," he said, information that could then be used to ensure advertisers are aiming their campaigns at people who are genuinely interested in what they have to offer.
The key to delivering quality content to an audience is in understanding who wants to read it and why they want to read it, he said,
While praising the concept of content relevancy as a "fantastic idea", Haile was concerned that, like the concept of impact, it was not a "scaleable metric".
"The important thing is that it is important to use the right metrics in the right context," said Haile.
Ianuzzi said there are three types of people interested in metrics – the editor or manager, the journalist and the audience – each with different motivations.
While a manager needs to sell advertising, editors or journalists are more interested in the quality of the content they are producing and the effect it may be having. For the audience, however, it is a question of trust and loyalty.
"Monitoring how users are using our website is very useful but doesn't mean we should become slaves to one metric," Ianuzzi said, and Haile suggested the possibility of "combinatorial metrics".
"They rely on two different attributes together where you have to win on both to do well," he said. "You have to be able to drive a lot of people to the page and then keep them there, it's not just having someone click and then have nothing to back it up."
"We should be striving for impact and relevance," he said, "But if we have to patch what we've got – with what we have today – then attention matters right now."If we focus on just one metric we will almost always be wrongAndrea Ianuzzi, Gruppo L'Espresso
While impact and relevance are the most important points in terms of the relationship between journalists and their readers, Haile again stressed that time may be more useful on an industry level for comparing organisations and approaching advertisers.
"Advertisers can see the number of impressions but we can put forward the quality of those impressions," he said, allowing outlets to differentiate on "standard" and "quality" impressions around different types of content on different parts of the site.
"A metric is nothing without context," he said, and different metrics should be applicable to different situations.
"If we focus on just one metric we will almost always be wrong," agreed Iannuzzi.
Free daily newsletter
- Tip: How to make metrics more effective at work
- Tip: Bookmark this advice for measuring the success of feature stories
- Inside the Financial Times' special projects strategy, six months on
- Readers perceive question-based headlines more negatively, study shows
- The Center for Investigative Reporting open-sources its Impact Tracker platform