Haystack 是Facebook设计来存储其小文件(图片)的系统,系统设计思想为小文件合并为一个大文件来存储,减少文件个数以及访问文件时的io次数。
Beaver, Doug, et al. "Finding a Needle in Haystack: Facebook's Photo
Storage." OSDI. Vol. 10. 2010.
http://static.usenix.org/legacy/events/osdi10/tech/full_papers/Beaver.pdf
F4 是 Haystack 的替代者。
Muralidhar, Subramanian, et al. "f4: Facebook's warm BLOB storage
system." 11th USENIX Symposium on Operating Systems Design and
Implementation (OSDI 14). 2014.
https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-muralidhar.pdf
Windows Azure Storage
Windows Azure Storage 使用一套架构的基础上,同时提供了 queue, blob, table 服务,架构精巧,值的学习。WAS 也发表了一些相关论文,介绍了整个架构。另外,WAS还单独发表了一篇论文介绍其对 Erasure Coding 的实践。
Calder, Brad, et al. "Windows Azure Storage: a highly available cloud
storage service with strong consistency." Proceedings of the
Twenty-Third ACM Symposium on Operating Systems Principles. ACM, 2011.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.231.6486&rep=rep1&type=pdf#page=153
Huang, Cheng, et al. "Erasure coding in windows azure storage."
Presented as part of the 2012 USENIX Annual Technical Conference
(USENIX ATC 12). 2012.
https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf
LinkedIn Ambry
LinkedIn 开源了它的对象存储系统 ambry,并且在 SIGMOD2016 上发表了论文,详细介绍了整个系统的架构实现。
Noghabi, Shadi A., et al. "Ambry: LinkedIn’s Scalable Geo-Distributed
Object Store."
http://dprg.cs.uiuc.edu/docs/SIGMOD2016-a/ambry.pdf
Ozone: The object store on HDFS
Ozone 的 jira issue,可以看到开发进度与文档
https://issues.apache.org/jira/browse/HDFS-7240
hortonworks 对其的介绍
http://zh.hortonworks.com/blog/ozone-object-store-hdfs/
Ceph
Weil, Sage A., et al. "Ceph: A scalable, high-performance distributed
file system." Proceedings of the 7th symposium on Operating systems
design and implementation. USENIX Association, 2006.
http://www3.nd.edu/~dthain/courses/cse40771/spring2007/psnowber-ceph.pdf