The Paywall Dilemma

Sharing links to articles is immensely popular on social media. The percentage of Twitter, LinkedIn, and Facebook posts that share a link is very high. In fact, clicking on links in social media posts is how many people consume their news and learn about what interests them. 

Yet, there is a storm brewing in the social media world. No, I’m not talking about any politics. It’s about the tension between the social media companies and the publishers of the articles to the links that we post. 

The media companies have been under a great deal of pressure to transition their online revenue away from advertising to the more predictable subscription revenue. The reason for that has been that Google, Facebook, LinkedIn, and Twitter have taken over a big chunk of advertising budgets. Yes, those are the same companies that enable us to share the links to articles in the publications they are putting out of business. 

The drive to switch from ad-supported free readers into subscribers forces the online publications to set up paywalls that stop us from reading their content and entice (or even force) us to subscribe. Publications such as The New York Times, The Washington Post, Forbes, or even TechCrunch are becoming more and more determined to make us pay for their content. And I don’t blame them, good content can’t be free. 

But herein lies the problem. The social media companies that live of social interactions - that to a large extent consists of sharing links to articles - are also the ones forcing the media companies to raise the paywalls that prevent the sharing. 

As a result, our experience is becoming more and more frustrating. Every time I click on a link that someone shared, I get asked to subscribe. It’s getting to a point where I don’t want to click on any links anymore because I know that I won’t be able to read the article anyway. That reduces my engagement which in turn will hurt the social media companies – the very companies that have been siphoning advertising dollars away from the media publications! 

As a user, I don’t mind subscribing to one or two daily newspapers and half a dozen magazines, but I can’t subscribe to all of them. Syndication makes the problem even worst, as I can read a particular article as a subscriber in the San Jose Mercury News, and yet I can’t read that same article on The Seattle Times website! 

It’s a real dilemma. We want to share and consume articles via social media. That’s a big part of the social media appeal. At the same time, the publishers deserve to get paid for their articles. Yet, we can’t possibly subscribe to all the publications and we share links to articles from the publications we subscribe but our friends/followers don’t.  

So, what should give? Should we ban sharing links on Twitter, Facebook, and LinkedIn altogether? That’s not likely, even though that kind of social media experience would address many of the disinformation problems we are facing online today.  

Should the social media companies start paying kickbacks to publishers for every time someone clicks on their article? That sounds intriguing, but Facebook, Google, Twitter, and Microsoft have been enjoying a really profitable business model and they will not easily start sharing those profits with the publishers. To a large extent, the publishers also benefit from the traffic social media sends their way. Actually, maybe the publishers should pay the social media companies a kickback! 

Should perhaps technology solve the problem? Maybe the social media companies could implement an AI-based abstract service that would give us a 2-3 sentence summary of each article without actually taking us to it? Yeah, that sounds like something Google would do (in fact, I think they tried) but the publishers and readers would hardly accept that kind of solution. 

Or, what if there could be a paid meta-subscription service that enables me to subscribe to every article on a particular subject, no matter where it is published? Let’s say, I am interested in astronomy and could subscribe to all astronomy articles, no matter which magazine they appear in. The subscription service would handle the complex commercial relationships with all the publishers. As a result, I could read any article on astronomy via a link shared on social media, without having to be a subscriber to all of those astronomy magazines. 

I am not sure if any of those solutions will work. But I predict that something will be changing in the next couple of years as more and more articles are published behind paywalls while social media drives more and more traffic to those articles. The readers are stuck in the middle and that’s never a steady state. 

 

Managing Paper in the Enterprise

Today, we observe the World Paper-Free Day to remind ourselves that we all are on a mission to get rid of the paper waste in the enterprise. I am a strong proponent of reducing the use of paper, even if I sometimes struggle. I read most of my books and magazines on my iPad, I use tools such as Evernote to take notes, and I have pretty much never any cash on me. If you look around my office, you’d probably score me as an 8 out of 10 on being paper-free except for my bookcase full of books. I really like books...

Many of our customers, however, struggle going paper-free. Indeed, flipping the switch from one day to another might be a little daunting. In reality, this transition has to be made easy to be realistic - this is more often a paper evolution than a revolution. That reminds me of some of the scenarios where our customers manage paper in the enterprise today:

1. Inbound
This is the most obvious situation where our customers deal with paper. This is the front-line in the war on paper. Many of our customers still receive paper-based information via snail mail and fax. They use our capture software to scan the paper documents right in the mailroom and to automatically extract as much information out of the scans as possible via optical character recognition (OCR) and data extraction which recognizes important data in the document (i.e. address, date, PO number etc.) to extract the metadata. Same thing happens with faxes that are captured using our fax software where the same OCR technique can be applied.

2. Outbound
At the tail end of many business processes is a new piece of content that has been produced to communicate to the stakeholders. This communication comes in two forms:

- Publishing - which is a form of communication using the same content asset(s) for a given target audience (more than one person). Publishing can occur online, on a portal, via mobile devices, email, etc. but it can also happen using paper - for example as a book or a marketing brochure.

- Customer Communication Management (also known as Output Management) which is a communication that has been personalized for a single individual. An example of such communication can be a utility bill which contains data about your monthly charges but it can also include useful, personalized tips on how to lower your next bill. This type of communication can again occur via a multi-channel delivery, one of which is often paper.

3. Physical Records
Managing records involves often the capability to manage physical records as well. The physical records - usually pieces of paper, but sometimes also objects such as police evidence - need to be kept the same way as electronic records, except that they don’t fit into a digital content repository. With physical records, the cost of storage is a major issue and records disposition usually means freeing up physical space on a shelf in a warehouse where those boxes of physical records are stored. The physical warehouse space is a major cost factor and many customers are approaching us today with projects to convert existing physical records stores into electronic records en masse.

4. Paper Processes
Yes, I know that the main idea of business process management (BPM) is to route information quickly from step to step and task to task - which is ideally done in an electronic form. But a few of our customers have to live with the paper-based process for now and yet they find inefficiencies in using BPM to track the status of each process instance. The workers complete their tasks on paper and then they “check off” the task in the BPM system to alert the next person that a task is coming. I know, I know...this is not the kind of BPM I usually recommend to our customers but I’ve seen it happen. Actually, this approach still delivers many of the BPM benefits. The manager can monitor the status of all the workers and processes, the processes can be optimized, the bottlenecks can be identified and the work teams can be re-aligned as needed. Those are some really cool benefits of BPM. Still, the plan is usually to add the capture software to get rid of paper altogether!

These are some of the use cases where our customers deal with paper - often as an intermediate step on the way to a paper-free enterprise. The paper-free vision is a great one but we will be probably dealing with paper for a long time. Any step that moves us in the right direction deserves a credit.

Here is to a Paper-Free World! 

Semantic Search at the Globe and Mail

Recently, I had the opportunity to meet with Kevin Schlueter, enterprise architect at the Globe and Mail. The Globe and Mail is one of Canada’s largest newspapers and they run one of the largest Web sites in Canada. Kevin told me an interesting story that I thought was worth sharing.

The Globe and Mail runs one of Canada's largest news sites

Newspapers live from advertising and so they are keenly interested in attracting the largest possible audience and keeping the readers on their site as long as possible. People usually come to a newspaper Web site to find some specific information. This becomes particularly relevant in times of significant events of interest - such as the recent federal elections in Canada. While the home page usually provides up-to-date information about the main election race, most users are also interested in their particular candidates and so they search for them.

Search optimization became very critical for the Globe and Mail. Since a newspaper is in the information selling business, the goal was stated as “show me what I want to know even if I don’t ask for it”. And this is the tricky part - exposing the readers to relevant articles that the user will likely be interested in. And that’s why the Globe and Mail employed semantic technology.

Semantic search is the next level of searching. The basic search is looking for the most statistically prominent keywords that are contained in the text body. It can find out about who, what, when, where, and perhaps even why. Full text search is often augmented by a metadata search which can reveal information such as the author, section, page, or publication data. But a semantic search can leverage automatically generated semantic metadata which is information about topics, people, places, products and concepts.

With semantic metadata, a reader can search for an article about a particular topic - say the Canada’s Governor General David Johnston. Unlike Wayne Gretzky, David Johnston is a fairly common name and a conventional search would find a whole bunch of them. Just try to google that name.  This is where a semantic search helps by identifying correctly all relevant concepts - such as the David Johnston who is Canada’s GG, the one who’s a Harvard professor, or the one who’s a known author and journalist. These concepts can be either presented to the reader as options or they can be used to deliver the content relevant in a given context.

You may think that this is what the online retailers have been doing for years - recommending similar products based on your current selection. But there is a big difference here. The retailers work with product catalogs which contain very structured data. When you are looking at a pair of shoes, the retailer can automatically recommend another five pairs in your size that are similar but perhaps a little more stylish (aka expensive). All of that is based on defined database fields. The semantic search can make such associations based on information contained in unstructured text in an article or a group of articles which is much more difficult.

Papers are changing

And that’s exactly what the Globe and Mail is doing - using the semantic search technology to generate semantic metadata that improves search results, increases search engine optimization (SEO), and makes the site more “sticky”. And stickiness means more advertising revenue which is what the paper lives from.

According to Kevin, it works really well as he could see during the recent election when the traffic on the site peaked to over 12,000 hits per second after the first results were published. Kevin plans on additional uses for the semantic search technology such as faceted navigation, similarity, or automatically generated topic pages. All that to keep the Globe and Mail site competitive in the Canadian news business.

The Future of Book Publishing

Two pieces caught my eye recently. First, I read in an article by Julie Bosman in New York Times about the fact that the latest Ken Follett book is more expensive as an e-book than as a hard-cover. Then I saw a blog post by Ron Miller that independently comments on the fact that the pricing of e-books is too high and not in line with customer expectations. And since I have been thinking a lot about the price of content recently, I had to comment.


First, I agree that the current pricing of e-books is wrong. They are simply too expensive. E-books are significantly cheaper to “manufacture” than paperbacks and thus, their price shouldn’t be the same as hard-covers or even higher. If it is cheaper to wait for the paperback than to download an e-book, people will wait for the paperback like they always did in the past.

Compared to paper books, e-books don’t have any cost of material, cost of manufacturing, and presumably a much lower cost of distribution. Could we do without the publishers altogether? Can the authors publish directly through the new breed of distributors with their reader gadgets such as Amazon, Barnes&Noble, and Apple? After all, that’s what’s being done with mobile applications.

Probably not. There are at least two other important functions that the publishers fulfill: quality assurance and promotion. The quality assurance is done through editors who today work for the publishers and their job is it to assure the quality of authors’ work. The authors could have their own editors but those could too easily develop into ghostwriters and so having editors independent from the authors is probably a good idea. That said, the editors could just as well work for the distributors to keep the authors honest.

The second, far more important, job of a publisher is promotion. Starting with talent discovery all the way to blockbuster book tours, the publisher makes sure the right books get noticed. There are about 200,000 books published each year in the US alone and to get noticed is critical. This function is even more important in the world of e-books where some of the traditional means of differentiation such as the book size or binding are not available. We can see the problem with the lack of coordinated promotion in the world of mobile applications – it is extremely difficult for applications to get noticed among the 250,000 apps available on AppStore. But even this role of publishers could be taken over by either the distributors or the authors themselves using PR agencies and other services.

While they still play an important function, the publishers need to understand that they compete with other forms of entertainment and they need to evolve their business models. In the world of digital content, they cannot hold on to the same pricing model as in the paper world. It happened to music and films and it will happen to books too. The publishers have a role to play as promoters but they need to adjust their pricing accordingly. If they don’t, they will become obsolete, just like software distributors and resellers have become for mobile applications. Or maybe, Apple will start selling e-books at $0.99 per chapter…