对象存储论文

2136 查看

Facebook

Haystack 是Facebook设计来存储其小文件(图片)的系统,系统设计思想为小文件合并为一个大文件来存储,减少文件个数以及访问文件时的io次数。

Beaver, Doug, et al. "Finding a Needle in Haystack: Facebook's Photo
Storage." OSDI. Vol. 10. 2010.

http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf

F4 是 Haystack 的替代者。

Muralidhar, Subramanian, et al. "f4: Facebook's warm BLOB storage
system." 11th USENIX Symposium on Operating Systems Design and
Implementation (OSDI 14). 2014.

https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf

Windows Azure Storage

Windows Azure Storage 使用一套架构的基础上,同时提供了 queue, blob, table 服务,架构精巧,值的学习。WAS 也发表了一些相关论文,介绍了整个架构。另外,WAS还单独发表了一篇论文介绍其对 Erasure Coding 的实践。

Calder, Brad, et al. "Windows Azure Storage: a highly available cloud
storage service with strong consistency." Proceedings of the
Twenty-Third ACM Symposium on Operating Systems Principles. ACM, 2011.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.231.6486&rep=rep1&type=pdf#page=153

Huang, Cheng, et al. "Erasure coding in windows azure storage."
Presented as part of the 2012 USENIX Annual Technical Conference
(USENIX ATC 12). 2012.

https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf

LinkedIn Ambry

LinkedIn 开源了它的对象存储系统 ambry,并且在 SIGMOD2016 上发表了论文,详细介绍了整个系统的架构实现。

Noghabi, Shadi A., et al. "Ambry: LinkedIn’s Scalable Geo-Distributed
Object Store."

http://dprg.cs.uiuc.edu/docs/SIGMOD2016-a/ambry.pdf

Ozone: The object store on HDFS

Ozone 的 jira issue,可以看到开发进度与文档

https://issues.apache.org/jira/browse/HDFS-7240

hortonworks 对其的介绍

http://zh.hortonworks.com/blog/ozone-object-store-hdfs/

Ceph

Weil, Sage A., et al. "Ceph: A scalable, high-performance distributed
file system." Proceedings of the 7th symposium on Operating systems
design and implementation. USENIX Association, 2006.

http://www3.nd.edu/~dthain/courses/cse40771/spring2007/psnowber-ceph.pdf