feat(zfile): add Intel QAT hardware acceleration for LZ4 decompression#406
Open
xiecl666 wants to merge 1 commit into
Open
feat(zfile): add Intel QAT hardware acceleration for LZ4 decompression#406xiecl666 wants to merge 1 commit into
xiecl666 wants to merge 1 commit into
Conversation
xiecl666
force-pushed
the
feature/support_qat
branch
from
cd3b97b to
c3a0506
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds Intel QAT (QuickAssist Technology) integration to accelerate LZ4 decompression in the zfile module, primarily by enabling batched decompression and introducing a QAT-backed LZ4 decompressor implementation with build-time auto-detection.
Changes:
- Add batched decompression in
CompressionFile::pread()to amortize per-block overhead and better utilize QAT/CPU batch APIs. - Introduce a new QAT LZ4 backend (
lz4-qat.cpp/.h) and wire it into the LZ4 compressor with process-wide availability caching. - Update build logic to auto-disable QAT when headers/libs are missing, and document QAT usage constraints in the root README.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/overlaybd/zfile/zfile.cpp | Adds batched decompression path in pread() and modifies zfile_decompress() read chunk sizing. |
| src/overlaybd/zfile/lz4/lz4-qat.h | Replaces prior header with a zfile-specific QAT LZ4 interface and param struct. |
| src/overlaybd/zfile/lz4/lz4-qat.cpp | New QAT-backed stateless LZ4 batch decompression implementation with a shared DMA pool. |
| src/overlaybd/zfile/lz4/lz4-qat.c | Removes the prior stub C implementation. |
| src/overlaybd/zfile/compressor.cpp | Adds QAT probing/caching and routes LZ4 decompression through QAT with CPU fallback. |
| src/overlaybd/zfile/CMakeLists.txt | Adds QAT header/lib auto-detection, conditional compilation, and links qat/usdm. |
| README.md | Documents QAT enablement and current decompression-only integration (with a minor wording issue). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
xiecl666
force-pushed
the
feature/support_qat
branch
from
c3a0506 to
c11cc93
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces Intel QAT (QuickAssist Technology) hardware acceleration for LZ4 decompression in the zfile module.
In overlaybd, zfile performs on-the-fly block-level decompression (typically 4KB blocks) during container image pulling and mounting. For highly compressible data, CPU-based decompression can become a bottleneck, consuming significant CPU cycles and increasing container startup latency. Offloading this task to QAT hardware can significantly reduce CPU usage, free up CPU resources for other workloads, and improve overall I/O throughput.