Consolidating audio files logic
In this paper we analyze the design and implementation of the most widely known user-space file system framework—FUSE—and characterize its performance for a wide range of workloads.
We instrumented FUSE to extract useful statistics and traces, which helped us analyze its performance bottlenecks and present our analysis results.
Most storage systems that write in a log-structured manner need a mechanism for garbage collection (GC), reclaiming and consolidating space by identifying unused areas on disk.
In a deduplicating storage system, GC is complicated by the possibility of numerous references to the same underlying data.
Memory-driven computing (MDC) brings together byte-addressable persistent memory, a fast memory fabric, task-specific processing, and a new software stack to address these data growth and analysis challenges. Her recent research is in the areas of NVM-aware data stores and data analytics frameworks.
At Hewlett Packard Labs, we are exploring MDC hardware and software design through The Machine. She has also worked in the areas of storage and information management, No SQL databases, storage dependability, intelligent storage, and workload characterization.
We describe two variants of garbage collection in a commercial deduplicating storage system, a logical GC that operates on the files containing deduplicated data and a physical GC that performs sequential I/O on the underlying data.
However, copy-on-write increases the demand on the file system to find free blocks quickly; failure to do so may impede allocations for incoming writes.We further present microbenchmarks demonstrating that common placement strategies are extremely sensitive to file-creation order; varying the creation order of a few thousand small files in a real-world directory structure can slow down reads by 15–175x, depending on the file system.We argue that these slowdowns are caused by poor layout.This talk will review the trends that motivate MDC, illustrate how MDC benefits applications, provide highlights from our Machine-related work in data management and programming models, and outline challenges that MDC presents for the FAST community. Kimberly Keeton is a Distinguished Technologist at Hewlett Packard Labs. She was a co-architect of the Express Query database, which provides metadata services for HPE's Store All archiving solution.She is an ACM Distinguished Scientist and a Senior Member of the IEEE, and has served as Technical Program Committee Chair for multiple USENIX, ACM, IEEE and IFIP sponsored conferences.