Systems @Scale 2019 - Accordion Better Memory Organization for LSM Key Value Stores
Eshcar Hillel, Senior Research Scientist, Verizon Media
https://code.fb.com/core-data/systems-scale/
Log-structured merge (LSM) stores have emerged as the technology of choice for building scalable write-intensive key-value storage systems. Though inherent to the LSM design, frequent compactions are a major pain point because they slow down data store operations, primarily writes, and increase disk wear. Another performance bottleneck in today’s state-of-the-art LSM stores, in particular ones that use managed languages like Java, is the fragmented memory layout of their dynamic memory store. In this talk, Eshcar shows that these pain points may be mitigated via better organization of the memory store. She also presents Accordion — an algorithm that addresses these problems by reapplying the LSM design principles to memory management. Accordion is implemented in the production code of Apache HBase, where it was extensively evaluated. Eshcar demonstrates Accordion’s double-digit performance gains versus the baseline HBase implementation and discuss some unexpected lessons learned in the process.
Видео Systems @Scale 2019 - Accordion Better Memory Organization for LSM Key Value Stores канала Justin Miller
https://code.fb.com/core-data/systems-scale/
Log-structured merge (LSM) stores have emerged as the technology of choice for building scalable write-intensive key-value storage systems. Though inherent to the LSM design, frequent compactions are a major pain point because they slow down data store operations, primarily writes, and increase disk wear. Another performance bottleneck in today’s state-of-the-art LSM stores, in particular ones that use managed languages like Java, is the fragmented memory layout of their dynamic memory store. In this talk, Eshcar shows that these pain points may be mitigated via better organization of the memory store. She also presents Accordion — an algorithm that addresses these problems by reapplying the LSM design principles to memory management. Accordion is implemented in the production code of Apache HBase, where it was extensively evaluated. Eshcar demonstrates Accordion’s double-digit performance gains versus the baseline HBase implementation and discuss some unexpected lessons learned in the process.
Видео Systems @Scale 2019 - Accordion Better Memory Organization for LSM Key Value Stores канала Justin Miller
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Delivering Terabits of Content External Considerations - Netflix - Network@Scale2017Moving Fast with Facebook's New Long-Haul Network - Facebook - Network@Scale2017Systems @Scale 2019 - Welcome KeynoteTracking Service Infrastructure at Scale - Shopify - SRECon2017Reducing MTTR and False Escalations: Event Correlation at LinkedIn - SRECon2017Systems @Scale 2019 - Observability Infra Uber and FacebookScaling Android Application Testing at Spotify - Spotify - DevTools@Scale2017How to use Impala's query plan and profile to fix Performance - Part 2Don’t Repeat Yourself Introducing Exactly Once Semantics in Apache Kafka - Kafka Summit 2018Scaling Up Job Scheduling With a Match making Service - Facebook - DevTools@Scale2017Systems @Scale 2019 - Apache HiveZero Trust Networks: Building Trusted Systems in Untrusted Networks - PagerDuty - SRECon2017Networking @Scale 2018 - Katran Layer 4 Load Balancing at Facebook EngineeringSpotify's Love-Hate Relationship with DNS - Spotify - SRECon2017A Close Look at Alibaba's High Performance Packet Processing Platform - Alibaba - Network@Scale2017The Service Score Card — Gamifying Operational Excellence - Uber - SRECon2017I'm Putting Sloths on the Map - Indeedcom - SRECon2017Panel: Training New SREs - SRECon2017How to use Impala's query plan and profile to fix Performance - Part 4Systems @Scale 2019 - Disaster Recovery at Facebook ScaleHow Do Your Packets Flow? - SRECon2017