5 years ago, I’ve led the design of a local filesystem tuned for large objects. Today the sketch of it was published under the name IMULA.
The design came to life when I worked for 9LivesData, a company developing HydraSTOR, a crazy-efficient distributed storage for backups. Back there we needed a filesystem for keeping data locally, so that we could build a distributed filesystem on top of it. Initially, ext3 was used, but its performance was suboptimal to us. ext3 fsync()s were too costly, it was getting fragmented over time and controlling it was a nightmare, so we thought “how hard can it be?” and decided to build our own. It turned out to have not been that hard and worked really nicely!
I’m happy that it is now partially public and I totally think the article is worth a read.
Marek Dopiera 