Amazon, CloudMQ Create "Social Data Cloud"

I was shocked to hear a Social Data Cloud (SDC) is in full operation, created by Freedom OSS using Amazon Web Services (AWS) technology. CloudMQ is making social data available “in the cloud“, and all the major social networks are participating in the exchange.

Social Data Cloud (SDC)

Facebook, MySpace and LinkedIn all publish data about changes occurring on their social graphs via CloudMQ. FriendFeed and other aggregators then subscribe to this data. Social networks can even subscribe to each others’ data. CloudMQ is the first Social Data Cloud (SDC), and is the beginning of a trend in which all social graph information will be made available real time, on a granular level “in the cloud”.

The emergence of this Social Data Cloud (SDC) raises important questions about user privacy, social network collaboration and future industry business models.

CloudMQ and the Social Web Ecosystem

Perhaps to avoid the bandwidth costs resulting from web page scraping by aggregators (FriendFeed , Twine and Mixx) and spider traffic from search engines (Ixquick, Google), data is now being automatically broadcast by the leading social networks to select partners. Apparently, all major social networks participate in this CloudMQ exchange, accessing and providing massive amounts of real time information about the social graph. The actual data made available to CloudMQ differs by social network, as each have different rules for what profile changes, etc. should be released into the exchange.

Future Impact of the Social Data Cloud

The emergence of a Social Data Cloud will impact both users and social business models.

Users’ privacy expectations are likely out of sync with a massive, AWS powered distribution of their every action into a Social Data Cloud. Users may not expect their social activities to be broadcast this way, but it is certainly how the industry is evolving. Social web business models are moving toward an open model where they feed a Social Data Cloud, and various partners access and syndicate that data.

Facebook Example: Users Resist Openess

Facebook users seem to cry foul every time there is a change that helps Facebook to communicate their information to other Facebook users and/or business partners. Facebook had protests when launching mini-feeds, ran afoul of user expectations with Beacon and got stopped from making a Terms of Service change that gave Facebook the right to exploit users’ content across all platforms, in perpetuity.

Facebook Example: Opening the Business Model

Facebook left other social networks in the dust primarily by opening its user data to third parties, namely Facebook application developers. Users didn’t complain about that move toward openness. Instead, Facebook rose to become the #1 social network globally.

Facebook did not open its business model enough, however. Twitter emerged into prominence with microblogging activity that is completely open, part of a “public timeline”. Facebook is responding in a number of ways, including the “everyone button” Nick reported yesterday.

Nick’s right, the “everyone button” primarily reflects some bizarre Twitter envy on the part of Facebook. But the “everyone button” also shows how immense the competitive pressure is to open up social platforms for more public consumption.

In other words, Twitter’s success is pressuring Facebook to release more information into the Social Data Cloud.

Silver Lining “in the Cloud”

What’s the silver lining? When Facebook steamrolled users’ expectations and chased Twitter’s open business model, it paved the way for new players rise to prominence in the social web landscape. These players are CloudMQ/ FOSS and Amazon Web Services – new entrants who are making the Social Data Cloud a reality.

AWS and CloudMQ take Messaging to the Next Level

I did a double take when I heard there is a messaging system operating on the scale of real time social graph data. Freedom OSS credits Amazon Web Services’s infrastructure on demand with making the CloudMQ service possible. CloudMQ’s Pub Sub model, when used for social data, occurs on an “Internet scale” that can dwarf the number of transactions in similar financial pub/sub exchanges.

Just imagine the amount of social data generated by status updates, user actions and profile changes that occur on social networks. As the social web grows, the amount of information that will be available in the future is mindboggling. Apparently, the cloud infrastructure of AWS is up to the challenge.

FOSS powers CloudMQ with AWS

Amazon Web Services (AWS) infrastructure underlies FOSS’ CloudMQ innovation. AWS provides infrastructure on demand (access to as many servers as you need). AWS is a recent service from Internet retailer that makes Amazon datacenters available to software developers as a platform, and on a pay as you go basis. Amazon’s massively scalable processing power is enabling new business models like CloudMQ to emerge in the social web.

Many services you use everyday are actually powered by CloudMQ and AWS. FriendFeed now uses CloudMQ messaging services to drive real time aggregation of social graph data, a quantum leap beyond the 1X a day batch processing that preceded CloudMQ .

CloudMQ Social Web Data Exchange

CloudMQ is creating a large scale exchange for real time data under a “pub sub” model. Social networks are able to publish and subscribe (pub/sub) to data about changes to their respective social graphs. CloudMQ created a system for social networks similar to those in the finance industry, where players like stock exchanges make financial transaction data available to one another as fellow members of an ecosystem. Interested parties can then follow exchange data via a stock trade “ticker”.