How to gauge importance/gravitas?

[padolsey]

So, with Air, the public platform, there is a concern of not being able to properly highlight serious events because they're being drowned out by the more frivolous occurrences.

E.g.

Glastonbury 2013

(this would be a case of a major event having many sub-events, see the evolving Glastonbury 2013 Case Study).

There's a mob of around 170,000 people at Glastonbury 2013. Imagine a bulk of them posting photos, videos, quirky reactions to lineups etc.

There needs to be a way for the official stuff and the "important" notices/sub-events to float to the top. A typical example:

One of the event organisers posts a weather warning or a lineup cancellation.

The problem is, Yip, as it is, doesn't have the concept of "organisers". It only has the concept of users with varying levels of credibility. So, I can imagine a problem where the important items are not being discovered because they're being posted by people with lacking 'cred' … people that are, in the real world, very credible sources for info, but on the Air, they're not considered credible...

[barbaracassani]

Importance vs Gravitas ->

• important (I guess) it's what most people perceive as interesting / relevant / worth mentioning
• gravitas: relates to "usability" and "usefulness", in part (i.e. "avoid the Northern line as it's delayed" vs "hey, there was a guy earlier on the Northern line that was carrying a bunch of incredibly cute puppies, here's the picture"). But not completely so: some news are serious but not usable. Also, the "sentiment" of an event is only a partial match -- it's tempting to equate gravitas with negativeness. Also, I am concerned that we are trying to grasp a concept which might be subjective to some extent. I cannot think of an example that applies to the case study you mention, but what about the Royal Baby vs the laws proposal that were occurring at same time? We IT peeps would place more importance on the latter without a single doubt, but what about anyone else? And what about emotional impact vs practical impact?

It might be useful to ask reporters to rate the importance of the reported events.
Importance can possibly be gauged by speed of diffusion from / into the epicentre in the first moments of life of the news. The topographical width of the first moments of spread can give an idea of how global the event is. A speedy surge indicates, obviously, importance (but not gravitas). If the reporter can rate, maybe we should give a different weight to the perceived importance depending how:

• how geographically far they are from the epicentre
• how far in time they are from the first reports

I think it's crucial to explore what you call "gravitas" but we should keep in mind that, unless we stick with "usefulness" or we find some accurate or arbitrary-but-constant gauge of the impact of an event, the risk to (even if only unwillingly) manipulate the visibility of news is one to avoid (or we would end up doing exactly what traditional media are doing, only less intentionally)..

[padolsey]

Thanks for your input, Barbara!

I think it would help to be explicit about those differences between gravitas and importance. I didn't really intend on dichotomising importance and gravitas, because to me they mean the same thing.

If we use the term "usefulness" instead of "gravitas", that'd be better IMHO. What we seem to mean by 'usefulness' is: An effect on the individual, or some level of actionability, or: 'The event warrants a change to the person's plans/future/activities'.

Take a presidential election in the USA. It's a massive event and would be deemed important, but probably it's not really going to directly affect people in the UK, for example (ignoring temporarily geopolitical statuses). It is not a very useful event to know about in comparison to say, your commute train-line closing. But, as you mentioned, this is potentially subjective.

---

So, I am not sure if there is a way to algorithmically determine usefulness without the users telling us what they think is useful... And that would just open us up to a lot of ambiguity. I don't think people will easily grasp the difference between importance/significance and usefulness/actionability. I think, maybe just measuring importance/significance, especially with the epicentre mechanism (which I think is awesome!), is enough -- we can divine usefulness from that metric IMHO.

Anyway, back to discovering 'Importance': In addition to an epicentre mechanism, I propose:

• Establishing the importance of an event from its location -- and how many people it is likely to effect. If we have an idea 1) of the population of the area and 2) any other events occurring in the area, then I think it'd be possible to gauge the likely significance of the event.
• Establishing the importance of an event by comparing it to topically similar events that happened in the past. If the similar historical events were deemed important then we can say it's more likely for this current event to be deemed important too.

At this point, it seems like the calculation for importance could be something like:

SizeOfDiffusion *
 SpeedOfDiffusion *
  TopicalImportanceFromHistory *
   LikelihoodOfImportanceGivenLocality

[barbaracassani]

I like your algorithm. Let's reflect on it.

SizeOfDiffusion: (widespread vs local) && (absolute size vs relative size) [where relative size means how it compares to other current events]. I remember I was at a music festival, coincidentally, when poor Amy Winehouse was found dead. The Twitter hashtag for the festival was drowned of news about her, virtually obscuring anything about the festival itself. The emotional impact seems to override easily any other factor in determining what goes viral.

SpeedOfDiffusion: I think it's crucial, and tricky at the same time. It's very much related to emotional impact, which is something that must be weighted carefully (this requiring a precise ontological stance on the whole endeavour). But it's a no brainer that a story that spreads like fire and whose importance is rated high is, well, important.

TopicalImportanceFromHistory: some clarification on that is needed. Do you mean analysing the language and extrapolate the topic to compare it to the perceived importance of similar topics in the past?

LikelihoodOfImportanceGivenLocality: hmm - big city vs small city?

I am reflecting on the fact that, the more I think about it, the more journalism seems to me to prioritize news according to arbitrary perception of emotional impact (when it's not, or not only, to further a political agenda). Breaking free from this convention is as thrilling as dangerous (and likely not to be done in a radical way. Crowdsourcing the arbitrary perception of news-worthiness seems to me revolutionary enough).

[padolsey]

TopicalImportanceFromHistory: some clarification on that is needed. Do you mean analysing the language and extrapolate the topic to compare it to the perceived importance of similar topics in the past?

Yes, the most simplest would be a similarity rating on the event's tags (each event would have tags like 'music', 'festival', etc.). I imagine this particular metric would need to be implemented down the line though ... At the moment I'd like to focus on the Diffusion metrics (speed and size) because those are metrics that we can implement without historical data.

LikelihoodOfImportanceGivenLocality: hmm - big city vs small city?

Yes. I imagine it'd be tricky to get enough data to make good decisions with though.

I am reflecting on the fact that, the more I think about it, the more journalism seems to me to prioritize news according to arbitrary perception of emotional impact

And hopefully the diffusion metrics will help us divine this emotional impact. :)

Re: relative vs. absolute. Everything will end up being relative, since all events must be displayed alongside other events in any given search.

So, initially, it looks like we'll only be gauging importance via diffusion. I guess this is sufficient. Would it be worth considering a user-controlled importance-rating -- or would that just end up being abused? The idea of doing it all automagically (without a user's direct input) is very nice indeed....

[padolsey]

Earlier I said:

Would it be worth considering a user-controlled importance-rating -- or would that just end up being abused? The idea of doing it all automagically (without a user's direct input) is very nice indeed....

This makes no sense. Sorry about that! Obviously we need to know when an event has been shared or noticed. Otherwise we have no way to measure diffusion. So I guess the question becomes: How do we establish that an event is being 'spread' by any given user?

• The user views the event
• The user somehow upvotes the event (we would have a voting feature?)
• The user somehow shares the event (We would have a sharing feature?)
• The user edits or contributes media to the event

It might be a case of weighting each of these actions accordingly and letting all of them affect the diffusion metric.

Starting to get a tad complex... But that's the nature of humans.

I imagine much of this will just have to be field-tested.

Ideally, IMHO, if should come up with ways to measure importance/utility without the user having to click buttons like:

• "I found this important!"
• "I found this useful!"

These are open to abuse and ambiguity.

[barbaracassani]

Re: upvote / downvote - I'd love to figure out something that it's completely visual, a bit like the mechanism that Google+ uses to add users to circles. I am not only talking about the UI but also as a general concept, something that relates more to the natural paradigm of appreciating the value of something and reacting to it, more than pressing a button and watching a number increase does. I agree with what you've written completely. At the same time I think there's a need for the user to be able to state how they feel about the importance of an event they report - more on that later.