This post is part of the blog series related to Storage Field Day 11. Find out the entire SFD11 content, presentations, articles, presenting companies and delegates here.
M&E – a specific use case?
Last week I was in the Silicon Valley to attend Storage Field Day 11 as a delegate. One thing that struck me as well as other delegates (and also industry friends) was the fact that nearly every company presenting was showcasing Object Storage products in the context of the very specific needs of the Media & Entertainment market. In fact I was jokingly saying that this event should have renamed Media Field Day 1. First of all, if you want to learn more about Object Storage, you should absolutely read what Enrico Signoretti and Chris M Evans (two authorities on Object Storage IMHO) are regularly writing. I’m not an expert by any means and just relating my Storage Field Day experience and thoughts.
I can’t remember who said that (either a fellow delegate or one of the presenters) but I vividly recall someone saying that it is easy to work with the M&E industry because they have specific needs, they are willing to talk to everyone and they are also at the forefront of innovation and new technology adoption.
The Media & Entertainment industry has enormous needs for storage capacity. Traditionally, M&E has been an avid consumer of scale-out NAS solutions such as EMC Isilon. Scale-out NAS systems present the customer with one or more continuous namespaces while allowing a customer to scale capacity by adding nodes transparently. But many other applications generate large amounts of data. Think of scientific research, medical research, hospitals…. and the list is quite long, these also produce huge datasets. So the problem is not only with the M&E industry. But since all our presenters decided to stick with M&E, what is compelling this specific industry to adopt object storage? Can’t they just scale-out their scale-out storage, allowing them to scale while they scale?
A blend of cost & performance
I may be repeating the obvious: the large amounts of data produced by the M&E industry are unstructured (compare this with structured data, such as database files for example). Unstructured data has multiple contexts and formats based on the industry, in the case of M&E this data will be mainly large video or audio files, chunks of processed data to be stored and reassembled later on, unedited raw cuts, etc. Imagine the amount of data needed to make a single animation movie – it’s far beyond the 2 minute Blue Angels fly-by sequence that you recorded on your iPhone, and also the data is either uncompressed or uses different codecs, is produced at higher resolutions etc. Since massive amounts of data need to be stored but are unlikely to be used frequently, it makes sense to store this unstructured data into a separate, cost-effective tier and honestly I really like what Enrico thinks about this.
The cost effectiveness, high-capacity and ease of use offered by object storage based systems, seconded by the omnipresent interoperability with Amazon S3 APIs offers a credible alternative to primary storage scale-out NAS systems. Object Storage systems can now exist in data centers as a warm storage tier below the primary storage hot tier, while allowing at the same time the possibility to replicate or move unstructured data to colder tiers the cloud, including storage on Glacier for long term retention use cases.
Max’s Opinion
M&E is a niche market, yet the large amounts of data they are generating plus their stance of being on the lookout for innovative storage solutions make them a perfect use case for object storage vendors – and probably makes M&E a golden goose for quite a few of those vendors. Also, the large amounts of data that needs to be stored makes up for impressive numbers. We should however take the vendor « massage » with a grain of salt. Some M&E companies may just be using open-source object storage on top of commodity hardware, after all.
And it’s much more than M&E. Think of satellite high-resolution pictures. Hospital X-Ray scans. Research data. There’s a lot more to object storage than M&E. I liked most of the presentations we’ve had on object storage, I just wish there would be more focus on how object storage can benefit the good old boring enterprise world.
Further reading
Disclosure
SFD11 Disclosure: this post is a part of my SFD11 post series. I am invited to the Storage Field Day 11 event by Gestalt IT. Gestalt IT will cover travel, accommodation and food during the SFD10 event duration. I will not receive any compensation for participation in this event, and I am also not obliged to blog or produce any kind of content. Any tweets, blog articles or any other form of content I may produce are the exclusive product of my interest in technology and my will to share information with my peers. I will commit to share only my own point of view and analysis of the products and technologies I will be seeing/listening about during this event.