by Coleen Singer at Sssh.com.
For those of you patient enough to have followed my blog postings here on EroticScribes, you’ve noticed that almost every month or so, I write up the latest disaster for the adult industry when it comes to corporate censorship of adult entertainment on the internet. These have included various Google antics (Google+ bans, Google Adwords ban, Google Blogger.com ban, and the constant other players like Amazon, Yahoo, and mainstream blog/ad networks that have all now thrown another log on the fire for getting all this filthy porn burned off of the internet (or at least their squeaky clean brand-sensitive platforms).
But, the recent news that the heartbeat of well-curated quality porn on the internet, Tumblr, has now joined the fray, was not a surprise (Tumblr is owned by Yahoo), but another disappointment for those of us that work hard to provide good adult entertainment in an ethical way (i.e. not throwing hardcore porn on the web for 14 year olds to look at). Over the years, Tumblr has been a great place to promote our studios and brands by, rather than blasting out hardcore videos for free, posting nice adults-only photography that also link to our blogs and websites.
In the case of Tumblr going down the censorship route, they did it a little differently in a way that is possibly even more cowardly than Google and company simply throwing everyone under the bus in one fell swoop. Rather than “ban” Tumblr blogs, they simply made them “unfindable” in their search function, as well as put in place a simple mechanism to prevent search engines from spidering tumblr for listing adult blogs and content.
Double Whammy! And in many ways, makes Tumblr “Teflon” as they are not “censoring” anything, per se, but simply keeping it off of search. Is that censorship? “Looks like a duck, quacks like a duck….” comes to mind.
How does all of this work to keep the search engines from spidering blogs and posts in Tumblr? It’s actually pretty simple old web technology – a little file that sits on the server and tells search engines what to look at and what not to look at called “robots.txt”. When Google or any other search engine that “plays by the rules” sends its robots out to spider the web, the first thing they look for is the robots.txt file. The vast majority of websites and pages “allow all” to come on in and index the site for inclusion in their search results. Some sites under development or that have duplicate text content from their other, say, paysite tours, use this to prevent spidering and penalization. But, it’s very easy just with a few lines of code, to tell the search engines NOT to look further or list the pages. Bacchus at ErosBlog.com posted this image for an example of a “banned” Tumblr blog:
Bacchus explains: “In robot, that means, roughly “All robots: stay out!” No search spiders allowed. No Internet Archive crawler. The Wicked Knickers tumblr is there, but you have to know about it, or you have to be linked to it. You won’t find it in Google, you won’t find it in any other search engine that honors robots.txt, and when Tumblr decides to stop hosting it, you won’t find the pages in the Wayback Machine — it will be gone for good, lost to humanity unless somebody with the technical chops and outlaw sensibilities of Archive Team finds a way to archive it anyway, robots.txt be damned.”
In a recent article by Violet Blue on ZDNet.com, she hits the nail on the head, saying:
The removal from internal search cuts off the adult blog user from the rest of the Tumblr community; since Tumblr has enabled robots.txt and inserted the rule (Disallow: /) that tells search engines not to search pages on all Tumblr adult blogs, the adult blog user is now cut off from Google search as well.
In fact, that rule prohibits all standards-compliant search engines from indexing the blogs, a list that includes Google, Ask, AOL, Baidu, Bing and Yahoo.
Longtime sex blogger “Bacchus” of Eros Blog – who is not a Tumblr user – discovered two months ago that Tumblr had quietly made it so all blogs marked as “adult” are now omitted from Google search, cut off from the rest of the Internet.
Tumblr users were not made aware of the fact that adult blogs are not indexed by Google no matter what preference the user has expressed.
Give it a try, gentle reader!
1. Go to Tumblr.com
2. Search for the word “erotica”
3. Result? No listings for erotica, with the message “That’s about it for erotica. Try another search?” (there are two helpful links at the top of the page, however, letting you know “related search” terms include “Madonna” and “Romance”). hmmmm…..
Same results for “porn”, “porn for women”, “Fetish”, “BDSM” and just about any other term a surfer might look for to find “the good stuff” on Tumblr. One notable exception is whoever is running blogs that combine the word “pornography” with the word “romantic” are managing to get a few search listings as of this writing. Ironically, the two blogs listed that are “Romantic Pornography” are not really romantic at all and some of the most NSFW hardcore stuff one can imagine! Nature abhors a vacuum it would appear and at least a few porners have managed to game the Tumblr system for at least the time being.
Perhaps the bitterest pill to swallow for all of us that have been running adult blogs on Tumblr and elsewhere is that we inadvertently brought this upon ourselves by being honest and ethical in how we labeled our blogs. When you first start up a new blog on Tumblr, there is a nice little area in the control panel that looks like this:
Here’s the paradox: If you flip the “adult-oriented” switch to “yes” in order to be responsible and keep the kiddies out of it and avoid embarrassment to some guy surfing the web during lunch break in his State Farm Insurance cubicle that avoids NSFW sites during working hours, now Tumblr has a very simple entry in their database to do with you as they see fit. Mind you, the adult industry has been “self-labeling” as “Adults Only”, “NSFW” and other terms for 15 years with great success with such filtering organizations as “RTA” and other adult warning tags in the web page coding to let parental filters know that the site has boobies and sex. But, the times they are a-changing…….
In this new day and age of corporate censorship by search engines, blog networks, and whatever crazy stuff the U.K. government is up to this week, it makes for an easy-peasy way for anyone that wants to exclude or even punish your site to do so very easily, either throwing you under the bus, or marching you into the gas chamber, by simply flipping the switch based on how you self-categorized your content.
The small consolation prize that Tumblr gave to the adult industry is, to date, they have not actually taken down all of the “porn and erotica blogs”. Why?
Theory: An estimated 10% of all Tumblr blogs and posts are “adult” (read: porn). They really don’t want to kick that number of people off the platform.
Do the math:
Tumblr’s front page informs us that it has 96.7 million blogs with 83.1 billion posts. That would translate into 9.6 million adult blogs and over 8 billion “porny” posts that all bring in some residual traffic from tweets and folks’ self-hosted blogs, etc. Even if not listed in Tumblr’s, or the Google/Bing/Yahoo/Ask/AOL Borg search results, that would be a hell of a hit on their incoming traffic and however it is they make money from all of that traffic. Thus, removal from search in every way possible is the closest thing Tumblr could do to deleting the blogs altogether, without actually removing 10% of its user base.
One result of Tumblr doing this is it removes yet one more responsible and ethical place where people can go look at naughty pictures with an option to set up “SFW” filters on the family home computer. I am taking a wild guess here that the porn tubes are pretty delighted with this Tumblr news as, indeed, “nature abhors a vacuum” and all those surfers that want a little erotic thrill, now are pushed a little further into relying on unrestricted access tubes and fileshare networks that are happy to provide lots and lots of free porn, at the expense of the performers, studios, distributors, intellectual property, copyright, etc in the legitimate adult industry.
The other equally bad result is that this simply encourages the more aggressive porn and adult producers not to self-label as such to avoid damage, and find ways to “game the system”, allowing kids and that poor guy at State Farm Insurance to stumble into things better not seen by them at that moment in time in their lives.
Come on, Yahoo CEO Marissa Mayer. Don’t be part of the problem. Be part of the solution for creating safe and ethical places within the internet for adult content to serve the prurient interests of what built the internet to begin with. Porn isn’t the enemy. Google is (but you knew that already.)
Erotic Cinema For Discerning Adults
Anonymous Adult Search