[CORE] DynamicOffHeapSizingMemoryTarget should warn when -Xms equals ...
...DynamicOffHeapSizingMemoryTarget relies on JVM heap shrinking (returning committed heap pages to the OS) to reclaim memory for off-heap usage. The static initializer already checks for -XX:+ExplicitGCInvokesConcurrent and -XX:+DisableExplicitGC, which would prevent shrinking from working....
https://github.com/apache/gluten/issues/12479
Home | Apache Gluten
...Apache Gluten is a middle layer responsible for offloading JVM-based SQL engines? execution to native engines. Apache Spark is a stable, mature project that has been developed for many years. It is one of the best frameworks to scale out for processing petabyte-scale datasets....
https://gluten.apache.org/
gluten/gluten-core/src/main/java/org/apache/gluten/memory/memtarget ...
...Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines. - gluten/gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java at main · apache/gluten...
https://github.com/apache/gluten/blob/main/gluten-core/src/main/java/org/apache/gluten/memory/memtarget/DynamicOffHeapSizingMemoryTarget.java
Home | gluten - nightlies.apache.org
...The basic rule of Gluten?s design is that we would reuse spark?s whole control flow and as many JVM code as possible but offload the compute-intensive data processing part to native code....
https://nightlies.apache.org/gluten/docs/_site/
gluten | Gluten is a middle layer responsible for offloading JVM-based ...
...Gluten use off-heap memory for certain operations. The absolute amount of memory which can be used for off-heap allocation, in bytes unless otherwise specified. Note: Gluten Plugin will leverage this setting to allocate memory space for native usage even offHeap is disabled....
https://apache.github.io/gluten/Configuration.html
Configuration | Apache Gluten
...Gluten use off-heap memory for certain operations. The absolute amount of memory which can be used for off-heap allocation, in bytes unless otherwise specified. Note: Gluten Plugin will leverage this setting to allocate memory space for native usage even offHeap is disabled....
https://gluten.apache.org/docs/v1.5.0/Configuration.html
Memory Management | apache/incubator-gluten | DeepWiki
...This document covers the memory management system within the Velox backend of Apache Gluten. It details how native memory allocation is coordinated between the JVM (Spark) and native execution engines...
https://deepwiki.com/apache/incubator-gluten/2.5-memory-management
nightlies.apache.org
...The basic rule of Gluten's design is that we would reuse spark's whole control flow and as many JVM code as possible but offload the compute-intensive data processing part to native code....
https://nightlies.apache.org/gluten/docs/index.md
Apache Gluten (Incubating): A Middle Layer for Offloading JVM-based SQL ...
...Gluten currently supports multiple backends, including Velox and ClickHouse. Each backend has its own characteristics?for example, ClickHouse tends to deliver better performance for smaller queries, while Velox excels in handling longer or more complex queries....
https://intel.github.io/optimization-zone/software/gluten/
IBM Community Group Library - IBM Community
...Two new features significantly enhance the usability and efficiency of Apache Gluten by automating complex memory management. Dynamic Off-Heap Sizing simplifies configuration by allowing users to set a single memory size, which Gluten then intelligently manages for its native off-heap operations....
https://community.ibm.com/community/user/viewdocument/4227-gluten-memory-management?CommunityKey=8c64553a-86a9-4af3-a2e6-55826c69b4e2