TheRock shared libpython for Nuitka/rocprofiler-compute

therock-shared-python-for-nuitka.md

TheRock Already Provides Shared libpython for Nuitka/Embedding

Summary

Yes, TheRock already builds Python with --enable-shared for embedding use cases. The infrastructure exists and is used by rocgdb. rocprofiler-compute just needs to wire into it.

Current Infrastructure

1. install_shared_pythons.sh builds Python with shared library support

PYTHON_VERSIONS=(
  "3.10.16"
  "3.11.11"
  "3.12.9"   # <-- The version needed
  "3.13.2"
  "3.14.0"
)

# Each version is built with:
./configure --prefix=/opt/python-shared/cp312-cp312 \
    --enable-shared \
    LDFLAGS="-Wl,-rpath,${PREFIX}/lib"

2. Installed location in manylinux image

/opt/python-shared/cp312-cp312/
├── bin/python3.12
└── lib/libpython3.12.so.1.0   # <-- The library Nuitka needs

3. rocgdb already uses this pattern

The CMake infrastructure includes:

• THEROCK_SHARED_PYTHON_EXECUTABLES - CMake variable pointing to shared Python interpreters
• therock_detect_shared_python_executables() - CMake function to verify libpython availability (in cmake/therock_detect_python_versions.cmake)

rocgdb iterates over these shared Python builds and creates a variant linked against each version's libpython.

For rocprofiler-compute / Nuitka

The team should:

Option A: Use the shared Python directly

# Python executable with shared library support
/opt/python-shared/cp312-cp312/bin/python3.12

# Link flags for Nuitka/CMake
-L/opt/python-shared/cp312-cp312/lib -lpython3.12
-Wl,-rpath,/opt/python-shared/cp312-cp312/lib

Option B: Leverage the CMake infrastructure

Use the same pattern as rocgdb - the THEROCK_SHARED_PYTHON_EXECUTABLES variable is populated from the shared Python builds and passed to components that need libpython embedding.

See:

• dockerfiles/install_shared_pythons.sh
• cmake/therock_detect_python_versions.cmake
• debug-tools/rocgdb/CMakeLists.txt

Conclusion

The infrastructure is already there. rocprofiler-compute should use /opt/python-shared/cp312-cp312/ rather than building its own CPython. No changes to TheRock's build system are required.

---

Original Question

Can TheRock build system provide libpython3.12.so for linking?

cc Bonnell, Jason who is helping build rocprofiler-compute as part of TheRock

In rocprofiler-compute, we would like to use Nuitka to create a standalone binary for our Python application. Nuikta basically ports Python into C and compiles it down to machine code such that we don't need Python and Python libraries (like pandas, numpy etc...) to run the standalone binary. This has been implemented in rocprofiler-compute cmake file: [rocprofiler-compute] Pre-requisites for including standalone binary with TheRock by vedithal-amd ·…

However, TheRock manylinux docker image is unable to run Nuitka command since libpython is not available for the Python3.12 available in the image.

/opt/rh/gcc-toolset-13/root/usr/libexec/gcc/x86_64-redhat-linux/13/ld: cannot find -lpython3.12: No such file or directory
collect2: error: ld returned 1 exit status
scons: *** [/therock/src/rocm-systems/projects/rocprofiler-compute/rocprof-compute.dist/rocprof-compute.bin] Error 1

This is because manylinux images provide python without shared library libpython*.so To get around this we would have to build our own CPython3.12 with --enable-share flag to created libpython3.12.so using steps mentioned below:

PYVER=3.12.10
PREFIX=/opt/cpython-3.12-shared
curl -fsSL https://www.python.org/ftp/python/${PYVER}/Python-${PYVER}.tgz | tar -xz
cd Python-${PYVER}
./configure --enable-shared --prefix=${PREFIX} \
  LDFLAGS="-Wl,-rpath,${PREFIX}/lib"
make -j"$(nproc)"
make altinstall

The questions is, can TheRock build systems take care of building libpython3.12.so for rocprofiler-compute instead of the component taking care of it?

---

Generated with Claude Code from prompt "Different topic. Need to answer this question. Look in the manylinux docker file and how rocgdb is iterating over the python versions."