Software-Defined Storage
BlueMind now helps you manage large email volumes by enabling Software-Defined Storage (SDS).
SDS is a marketing term used for policy-based data storage software, regardless of the underlying hardware (See Wikipedia).
While one email message alone doesn’t — or shouldn’t — take up much more than a few kilobytes on a hard drive, increasing and widespread usage of email by a large number of users requires massive and ever-expanding storage spaces. All the messages stored on a BlueMind platform can require several dozen terabytes of space. Managing it is a major challenge in terms of IT infrastructure, in addition to maintenance, back-up and data management issues.
What are the benefits of SDS?
Without SDS, as the volume of data increases, you have to make adjustments to your storage system – additional hard drives, archiving some files on network drives etc. — which is cumbersome and entails service stoppages.
SDS can help remedy this as it is:
- scalable: broadly speaking, SDS space can be viewed as illimited. This is because the SDS abstraction layer makes making changes to the equipment easy. In addition, some commercial offerings can provide supervised and pooled storage which can be adjusted down to the last byte and therefore enables precise on-demand billing.
- resilient: the data is duplicated and distributed within clusters.
- integrated: software abstraction means that any feature possibly imaginable can be added – from access control lists management to version management to APIs.
Implementing SDS within BlueMind
Technical considerations
There’s a variety of SDS solutions, most of which implement the same APIs — S3 and Swift.
Ours was developed based on OpenIO’s open source S3-compatible implementation, provided via a Docker container — which we are also testing with Ceph.
Ceph is an open-source distributed storage reference platform.
OpenIO combines efficiency, hyper-scalability and consistent high performance with the benefits usually found on other platforms.
We’ve implemented the Rest S3 API in Java to facilitate maintenance and minimise dependencies – and related issues – with the Cyrus IMAP code. Most object storage implementations offer a Rest S3 API. Two low-dependency C libraries have been built into Cyrus IMAPD for Java to register the sds-proxy component.
How it works
When Cyrus receives or creates a new email, it creates a temporary file on the local file system, then sends this file to S3 and finally deletes it. When Cyrus needs to read the file’s information, it downloads the file from the Object Store, then writes it to a temporary file, reads it and finally deletes it.
The BlueMind Core also implements an access to the SDS solution so that it can retrieve, when it needs to, the email from the Object Store.
In practice
Storage gain does have its caveats: performance – which will depend on the SDS’s performance – and increased network workload. This is why SDS is only recommended for large data volumes.
Don’t hesitate to contact us for further information.
How can you enjoy it?
This feature – relevant for large data volumes only – will be available in BlueMind 4.1 and will have to be enabled in the admin console.
It also requires an SDS environment, e.g.:
- OpenIO: either by your own means, either by getting a subscription from a supplier.
- Ceph
- others!
About OpenIO:
OpenIO is an SDS technology developed by the eponymous French start-up company. The core of its solution is open source and dates back to the 2000s. Major IAPs have had to contend with the exponential growth of stored email volumes as users have moved away from locally storing emails to storing them on remote servers which enables them to access their messages from different devices and while on the move.
OpenIO’s high-performance object-based SDS solution provides massive scalability and is able to exploit standard servers, having hosted up to 45 million mailboxes — and their related storage spaces – and combining data protection, service quality and cost control. And, to round things off, OpenIO helps preserve data sovereignty as clients can build their own OpenIO cluster on premises or host it on any third-party cloud server they choose.
The synergies between BlueMind and OpenIO are obvious. The two solutions complement each other in the pursuit of a European, independent, collaborative mail solution and offer an alternative to today’s US domination.
The BlueMind <> OpenIO S3 API-based connector facilitates integration.
With OpenIO, storage is scalable to infinity and you get responsive standard, standard24/7, or premium support depending on your needs.
