[CELEBORN-2241] Memory swap instead of memory copy

[mcdull-zhang]

What changes were proposed in this pull request?

Improve performance by using memory swapping instead of memory copying.

Optimization results: The table below shows the shuffle write time for TPCH 1TB.

	SortBasedShuffleWriter	HashBasedShuffleWriter
Before	615.44W ms	502.31W ms
After	442.47W ms	434.77W ms

Note: This is the result of multiple optimizations combined;
other optimizations will be submitted as pull requests in the future.

Why are the changes needed?

Improve performance by reducing memory copying.

Does this PR resolve a correctness bug?

NO

Does this PR introduce any user-facing change?

NO

How was this patch tested?

Existing Unit Tests.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.21%. Comparing base (ffff5bb) to head (ef3c308).
⚠️ Report is 40 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3572      +/-   ##
==========================================
+ Coverage   67.05%   67.21%   +0.17%     
==========================================
  Files         357      357              
  Lines       21779    21847      +68     
  Branches     1930     1938       +8     
==========================================
+ Hits        14602    14683      +81     
+ Misses       6160     6144      -16     
- Partials     1017     1020       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[FMX]

SortBasedShuffleWriter	HashBasedShuffleWriter
Before	615.44W ms
After	442.47W ms

You mean that the running time dropped from 6000 seconds to 4000 seconds, that's very impressive.

[mcdull-zhang]

SortBasedShuffleWriter HashBasedShuffleWriter
Before 615.44W ms
After 442.47W ms
You mean that the running time dropped from 6000 seconds to 4000 seconds, that's very impressive.

@FMX It is the sum of shuffle write block time for 22 queries, not the end-to-end query duration;
And this is the total effect of multiple PRs;

[github-actions]

This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days.