VOPipe: an Interface for Asynchronous Data Messaging Using VOSpace
Bernhard Bauer (TUM)
Tamas Budavari and Alexander S. Szalay (JHU)
We present a new interface for scalable async data messaging built on top of the VOSpace infrastructure. VOPipe makes it easy to create and run a distributed application, like SkyQuery, over the Virtual Observatory framework. In VOPipe, VOSpace containers store the data blocks, which can be quite large, and a message queue is used to control the data flow between the nodes. Once a VOPipe is set up, applications can place consecutive data blocks into the pipe and read from it with a very simple interface. The architecture is similar to how Amazon's Simple Queue Service (SQS) interface is used in conjunction with its S3 storage service. A simple VOPipe service has been implemented on top of the JHU implementation of VOSpace. The talk will discuss both design/implementation considerations and various application scenarios, in particular a scalable SkyQuery service.
Mode of presentation: poster