'Big data’ is a phrase that’s getting a lot of airplay at the moment. The basic premise is that nowadays we have the computing power at our fingertips to be able to crunch massive quantities of disparate information and use it to unearth previously unappreciated things - about people.

The US election came down ultimately to just few counties in a few swing states according to various media reports. And that’s because both parties had huge databases of information that they had gathered about voters that allowed them to predict very accurately how states and even counties would vote long before polling day. This allowed them to micromanage the campaign. Spending their resources only on the people that ‘mattered’.

On the one hand this seems really smart. It’s a bit of a holy grail for marketers this kind of stuff. To roll out the oft quoted phrase attributed to John Wanamaker. "Half the money I spend on advertising is wasted; the trouble is I don't know which half.” Increasingly with the web and big data to help them, marketers DO know which half is wasted. (Or they think they do.)

Take this thinking to its conclusion and you can see that in politics this approach could be massively undemocratic. People who live in states which are bound to vote a particular way regardless (according to the data wonks) aren’t worth talking to at all. Leave them alone – don’t even bother to tell them anything much at all. Conversely, imagine if you just happen to live in one of the key swing state counties that the data wonks have worked out really matter – your vote is suddenly worth exponentially more than the votes of millions of people elsewhere. It would be worth moving to one of these counties just to have that kind of influence.

I’ve recently read The Filter Bubble: What The Internet Is Hiding From You by Eli Pariser and it’s just brilliant. Its basic premise is quite similar. As algorithms get better and better at knowing what you want (or what they think you want), they’ll just keep dishing that up and you’ll never see anything else. So, click on the same friend a few more times on Facebook and their updates get pushed up in your newsfeed – do that enough and it’s possible that you’ll increasingly see only the updates of same few friends that Facebook thinks ‘really’ matter to you at the expense of all the others. Likewise for Google. Keep searching for say information that suggests you have a bias towards voting Republican (stuff that’s pro gun ownership maybe?) and slowly the search engine will start serving up more of the same and you’ll see less and less stuff that’s more Democrat leaning (stuff that’s pro gay marriage for example). You’ll begin to be locked inside a bubble of stuff that is highly 'relevant' to you to the extent that it will shut out all conflicting points of view.

Both of these concepts are fundamentally about the ability of technology to know us better than we know ourselves or to be better at deducing subtle connections than we can ever hope to be. And maybe (maybe) it is. But is that a good thing?

You know all that stuff you stick on your Facebook page? It’s not just Facebook using it. It gets sold on to huge third party companies that use it to model and predict behaviour. They are getting better and better at it. Soon they will know that because you like Homeland you are more likely to buy one brand of soft drink over another or one car over another. Did you know that when you read a book on Kindle, Amazon is watching what you read? So you spent the afternoon reading a book of fiction which has a chapter in it featuring a car chase where the hero drives a BMW? It won’t be long before the ad you get served up next time you hit Amazon will be for… a BMW.

So… you might have guessed that I’m not a huge fan of big data. I believe the Net should be about helping us make more interesting and unexpected connections – about serendipity and about humanity ahead of technology and profit.

What has all this got to do with content then? This week a company called Percolate raised 9 million USD in series A funding. Percolate has developed a platform to help brands use social media more effectively. It uses an algorithm to suggest content ideas for social media community managers (ie the people that run a brand’s Facebook page or blog etc). This means they can spend their time producing more content that’s ‘appropriate’ more quickly.

To quote some of the stuff on their website: Percolate’s goal is to make content creation easy by prompting community managers with ideas and inspiration…  Percolate is constantly scanning for interesting areas for the brand to explore for stock content, whether that’s a long-form blog post, an infographic or a video.

Who needs research and consideration to write content? Just get an algorithm to come up with the ideas for you. But how would you feel if you were reading stuff created off the back of prompts from a piece of technology rather than by a real person?

9 thoughts on “Are algorithms better than editors?

  1. Provocative piece, Jeremy. I wouldn't worry, though. Someone will invent an algorithm that lets individuals automatically respond to content online without having to actually read it, as a way to maintain Klout scores in absentia.

  2. Good as per Jeremy. Just coz you can't see the man, doesn't mean he isn't after you :)

    Three supps...

    1. When the internet came out in the late 90s mys sister had it about 3 years before anyone else (worked at a Uni in the States & in London). She gave me one piece of advise. Treat every email as if it were a postcard ie anyone could read it. Feel the same way now about twitter now. Good advise then I sometimes fail to follow.

    But we've had "targeted" ads on gmail for a few years now and it doesn't bother me. I take malicious pleasure on clicking on my my competitors ads *snigger - that's just cost them a quid, my work here is done*

    "Free" as a swap for "personal data" seems a fair transaction, no? Christ the available data on a person is astonishing. Ever done a credit reference check?

    2. Remarketing ads are pish, although am amazed at how many people rate them. I had a 30 min argument with Google adwords bloke about them last week. He'd supped the Dublin Kool Aid on them. I just find them creepy...and as a part-time marketeer ineffectual.
    Just an opinion.

    3. Did you watch the live coverage from PhocusWright last night on Tnooz? Steve Kaufer of Trip Advisor was being "chatted to" about the early days. At the start they weren't into UGC, they had editors to scour the web looking for interesting stuff they could nick from papers and blogs and whack next to hotel details and planning tools. Only when they studied the analytics and log files deeper did they realise UGC was working better and dropped the ed side (or push the UGC stuff more). Interesting that they were doing that 10 years ago. Pre-dates a lot of the movement in Big Data maybe? We'll see...

    Sorry for the ramble
    *shuts gate so lambs don't escape*

  3. Hi Stu
    I don't think the free stuff in exchange for personal data thing is 'fair' at all. The user has no idea exactly what data is being collected or more importantly how it will be used. I've deleted a load of stuff off my Facebook profile, but I'm sure it's still out there sat on a Facebook server. My experiences messing about with Klout showed me this happens in practice. I deleted my profile and set a new one up using my name again and it knew immediatelyt who I was and what my Facebook and twitter profiles were. Just because you delete a profile with a service doesn't mean it's deleted. It's just not live anymore. The genie is kind of out of the bottle to be honest. My current thinking is to do stuff that's totally contrary to what I'd normally do... just to put them off my trail. ;-)
    Also - the nerds 'think' their alogrithms and data manipulation tell them everything about me. But I'd like to think I am sufficiently contrarian and random that actually they will never be 100% right. But who knows. Maybe they will be.

  4. Yeah I think I'm 130 on my FB profile, I live in Mongolia and my DOB is 1st January. I might "like" Glee too just to completely throw them orf.

    Interesting that it only seems to be middle-aged men who care about this stuff. The yoots don't really seem to give a hoot... :)

  5. Cheers Jeremy, good stuff. Some tenuously connected observations:

    - Don't most of us put ourselves in our own filter bubble anyway? My regular/daily news sources are the Guardian and RSS subscriptions to Tnooz, this site, David Whitley's site and a few SEO/marketing blogs & sites. And I bet that's relatively diverse compared to most. Perhaps Google and Facebook are just doing what we already do to ourselves?

    - Marketers have been collecting "big data" on us for yonks. Even offline. What about the Tesco club card, the Nectar card, et al. Do you see anything fundamentally different between that and Amazon or Gmail showing you a targeted ad?

    Cheers, Matt

    1. Hi Matthew
      Thanks for commenting.
      - If we choose to put ourselves in a filter bubble, I think that's quite different from being put in one *without even knowing* by a website or search engine. Yep, that's the important bit we aren't made aware it's happening. For google you can kind of see this by trying the same search query logged in and not logged in. It's quite interesting to see.
      - The basic principle is the same yes, it's just the breadth of data the likes of Amazon and Facebook acquire that is even more amazing. People share the most imtimate of details without a second thought. My example about knowing that you will probably buy a particular brand of soft drink because you like a particular TV programme is a good example of how things are moving. I personally don't like the idea of some machine somewhere knowing stuff about me I don't even know myself. Not sure there's much I can do about it, mind.

  6. Ya know what - if you are a person of limited interests then maybe, just maybe you won't even care when some algorithm serves the same old crap up to you. It might even be comforting. Fact is, from what I have experienced of this stuff, people are simply too diverse to be pigeon-holed in this way and sets-of-rules are too unsubtle to keep up. In any event the web is generally not a push medium; sure there will be some limited shaping of certain results, such as all those adverts I never see because I use Adblock Plus.

    I'll believe in what Percolate is selling when they show (n) examples on their website and I remember seeing one of those examples being recommended to me by a friend. That is a point not to be underestimated - every time someone ever says "and this means (x)" on the web they should have a TON of evidence. Collecting data is cheap as chips - so if it works - prove it.

    People love stuff that makes them look clever even when they don't quite understand it - hence 'a' for algorithm getting thrown around a lot!

    1. Hi Mark
      Thanks for commenting. I agree. You might not care, but the point for me is the fact that you don't know it's happening. People still believe that if something is no.1 in a google search it's 'the best'. And they need to understand that that's not necessarily the case. Google works so 'well' (ie it's fast) that people just trust it. Which is scary when people who work in SEO know full well that it's very gameable. Still, despite all the updates etc.

Comments are closed.