MORPH
SIGN IN SIGN UP
ray-project / ray UNCLAIMED

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

0 0 1 Python
Normal View History Raw
[core] Bind jemalloc to raylet and GCS (#38644) The default malloc failed to do a good job in managing the memory. For example, the memory fragmentation is high after running the server for a while. jemalloc mitigated this issue.
2023-08-23 13:29:23 -07:00
load("@rules_foreign_cc//foreign_cc:configure.bzl", "configure_make")
[ci] Shorten bazel directory paths to avoid failures on Windows (#55525) - Shortening bazel workspace name: `s/com_github_ray_project_ray/io_ray` - Set `--output_base=c:/bzl` instead of `--output_user_root=c:/raytmp`, which saves ~20 characters in the path. --------- Signed-off-by: Edward Oakes <ed.nmi.oakes@gmail.com>
2025-08-12 21:17:48 -04:00
load("@io_ray//bazel:ray.bzl", "filter_files_with_suffix")
[core] Bind jemalloc to raylet and GCS (#38644) The default malloc failed to do a good job in managing the memory. For example, the memory fragmentation is high after running the server for a while. jemalloc mitigated this issue.
2023-08-23 13:29:23 -07:00
filegroup(
name = "all",
srcs = glob(["**"]),
)
configure_make(
name = "libjemalloc",
lib_source = ":all",
linkopts = ["-ldl"],
copts = ["-fPIC"],
args = ["-j"],
[core] Fix the GCS crash when using LD_PRELOAD and make jemalloc work with worker as well (#39192) ## Why are these changes needed? `libjemalloc` was built into gcs and raylet in [this](https://github.com/ray-project/ray/pull/38644) PR. While it fixed the memory issues it brings some issues. It'll break the workload if the user start ray with tcmalloc. It doesn't work with core worker and for some workload, we need this. This PR make the jemalloc a shared library and if there is LD_PRELOAD, it won't load jemalloc.so automatically. ## Related issue number Closes #39135
2023-09-02 16:29:20 -07:00
out_shared_libs = ["libjemalloc.so"],
Make Ray work on GH200 (#40816) Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
2023-11-16 18:01:02 -08:00
# See https://salsa.debian.org/debian/jemalloc/-/blob/c0a88c37a551be7d12e4863435365c9a6a51525f/debian/rules#L8-23
# for why we are setting "--with-lg-page" on non x86 hardware here.
[core] add --enable-prof-libunwind to jemalloc build (#48671) This fixes a jemalloc profiling deadlock, as hit by clickhouse https://github.com/ClickHouse/ClickHouse/pull/66346 Signed-off-by: Ruiyang Wang <rywang014@gmail.com>
2024-11-12 19:22:05 -08:00
configure_options = ["--disable-static", "--enable-prof", "--enable-prof-libunwind"] +
Make Ray work on GH200 (#40816) Signed-off-by: Philipp Moritz <pcmoritz@gmail.com>
2023-11-16 18:01:02 -08:00
select({
"@platforms//cpu:x86_64": [],
"//conditions:default": ["--with-lg-page=16"],
}),
[core] Fix the GCS crash when using LD_PRELOAD and make jemalloc work with worker as well (#39192) ## Why are these changes needed? `libjemalloc` was built into gcs and raylet in [this](https://github.com/ray-project/ray/pull/38644) PR. While it fixed the memory issues it brings some issues. It'll break the workload if the user start ray with tcmalloc. It doesn't work with core worker and for some workload, we need this. This PR make the jemalloc a shared library and if there is LD_PRELOAD, it won't load jemalloc.so automatically. ## Related issue number Closes #39135
2023-09-02 16:29:20 -07:00
visibility = ["//visibility:public"],
)
filter_files_with_suffix(
name = "shared",
srcs = ["@jemalloc//:libjemalloc"],
suffix = ".so",
[core] Bind jemalloc to raylet and GCS (#38644) The default malloc failed to do a good job in managing the memory. For example, the memory fragmentation is high after running the server for a while. jemalloc mitigated this issue.
2023-08-23 13:29:23 -07:00
visibility = ["//visibility:public"],
)
Reference in New Issue Copy Permalink