Cloud Interoperability – Five Challenge Problems

March 30, 2009

I attended the OMG Workshop on Cloud Interoperability in Arlington, Virgina that took place on March 23, 2009.

There were presentations about cloud interoperability from the Cloud Computing Interoperability Forum (CCIF), the Open Cloud Consortium (OCC), the Open Grid Forum (OGF), the Distributed Management Task Force (DMTF), as well from several other companies and organizations.

(Full disclosures: I gave the OCC presentation and have some opinions of my own about cloud interoperability.)

To say the least, it was clear from the meeting that there was no consensus yet about how to begin to draft standards for clouds and for their interoperability. This was not surprising. But what was surprising, at least to me, was that there was not even consensus about what were the real problems and challenges.

After the meeting, I decided that it might be helpful to create a list containing some challenge problems that, hopefully, with the right standards, would be much easier to solve. Here is a very rough first attempt at creating such a list.

Problem 1: Assume I have a web application that uses three communicating virtual machines in Cloud 1. Move the application transparently to Cloud 2.

Problem 2: Assume I have a web application that uses three communicating virtual machines in Cloud 1. Increase the capacity of the application by add three more virtual machine instances from Cloud 2.

Problem 3. Assume I have a cloud application in Cloud 1 that relies upon a cloud storage service, such as a storage service provided by the Hadoop Distributed File System (HDFS). Assume that Cloud 2 has cloud storage services, but that they are not implemented using Hadoop. Move the cloud application transparently to Cloud 2.

Problem 4. Assume I have a cloud application in Cloud 1 that relies upon a cloud storage service as well as a cloud compute service that is based upon MapReduce or one of its variants or generalizations. Assume that Cloud 2 has cloud storage and compute services, but that they are not implemented using Hadoop. Move the cloud application
transparently to Cloud 2.

Problem 5. Assume that there is a HIPAA compliant application in Cloud 1 (managed by Provider P) and another in Cloud 2 (managed by Provider Q). Assume that a patient has given permission to Cloud Provider P to copy any of his information (say, it is a file) that is in Cloud 2 and move it to Cloud 1 in order create a consolidated record in Cloud 1.
Create the consolidated record (say, the concatenation of the two files) automatically in Cloud 1.

The first two problem concern clouds that provide on demand computing instances, such as provided by Amazon’s EC2 and S3 services. The next two problems concern clouds designed for data intensive cloud applications, such as applications using Hadoop services. The fifth problem concerns sharing information between two clouds.

As always with any new standards, the goal is to rely on existing standards as much as possible and only to introduce something new when absolutely required. Therefore the first challenge is to identify the gaps in current standards for these five problems.

None of the problems are precise enough yet to determine whether new standards are required, but perhaps there are suggestive of problems that can be made precise enough for this purpose.