(partial) EasyBuild log for failed build of /dev/shm/easybuild-tmp/eb-1j4egck1/files_pr24926/p/PyTorch/PyTorch-2.9.1-foss-2025b-CUDA-12.9.1.eb (PR(s) #24926)

PyTorch-2.9.1-foss-2025b-CUDA-12.9.1_partial.log

== 2025-12-20 08:06:18,027 easyblock.py:374 INFO This is EasyBuild 5.1.3.dev0-rcee3750fc6f956cbe93f8f9a0059d04072aaa9c3 (framework: 5.1.3.dev0-r8ca8af8ef438b4b05054338dd2d33e78a9cca516, easyblocks: 5.1.3.dev0-rcee3750fc6f956cbe93f8f9a0059d04072aaa9c3) on host n1043.barnard.hpc.tu-dresden.de.
== 2025-12-20 08:06:18,027 easyblock.py:380 INFO This is easyblock EB_PyTorch from module easybuild.easyblocks.pytorch (/home/s3248973/.local/EasyBuildDev/easybuild-easyblocks/easybuild/easyblocks/p/pytorch.py)
== 2025-12-20 08:06:18,028 easyblock.py:1178 INFO Build dir set to /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1
== 2025-12-20 08:06:18,028 easyblock.py:1241 INFO Software install dir set to /data/horse/ws/s3248973-EasyBuild/easybuild-rapids/software/PyTorch/2.9.1-foss-2025b-CUDA-12.9.1
== 2025-12-20 08:06:18,028 easyblock.py:1246 INFO Module install dir set to /data/horse/ws/s3248973-EasyBuild/easybuild-rapids/modules/all
== 2025-12-20 08:06:18,028 easyblock.py:348 INFO Init completed for application name PyTorch version 2.9.1
== 2025-12-20 08:06:18,028 pythonpackage.py:473 INFO Using default value for expected module name (lowercase software name): 'pytorch'
== 2025-12-20 08:06:18,028 pythonpackage.py:554 INFO Using '%(python)s setup.py %(install_target)s --prefix=%(prefix)s %(installopts)s' as install command
== 2025-12-20 08:06:18,028 environment.py:95 INFO Environment variable PYTHONNOUSERSITE set to 1 (previously undefined)
== 2025-12-20 08:06:18,028 environment.py:95 INFO Environment variable PIP_REQUIRE_VIRTUALENV set to false (previously undefined)
== 2025-12-20 08:06:18,028 environment.py:95 INFO Environment variable PIP_DISABLE_PIP_VERSION_CHECK set to true (previously undefined)
== 2025-12-20 08:06:18,028 environment.py:95 INFO Environment variable XDG_CACHE_HOME set to /dev/shm/easybuild-tmp/eb-1j4egck1/xdg-cache-home (previously undefined)
== 2025-12-20 08:06:18,028 python.py:273 INFO Using /dev/shm/easybuild-tmp/eb-1j4egck1/xdg-cache-home as pip cache directory
== 2025-12-20 08:06:18,028 pytorch.py:291 INFO Auto-enabling use of pip to install PyTorch >= 2.0, since 'use_pip' is not set
== 2025-12-20 08:06:18,028 pythonpackage.py:526 INFO Using pip with --no-deps option
== 2025-12-20 08:06:18,028 pythonpackage.py:554 INFO Using '%(python)s -m pip install --prefix=%(prefix)s %(installopts)s %(loc)s' as install command
== 2025-12-20 08:06:18,028 easyblock.py:5083 INFO Obtained application instance for PyTorch (easyblock: None)
== 2025-12-20 08:06:18,028 easyconfig.py:1814 INFO Generating template values...
== 2025-12-20 08:06:18,028 mpi.py:123 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'OpenMPI'
== 2025-12-20 08:06:18,029 mpi.py:299 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'nr_ranks': 1, 'cmd': 'xxx_command_xxx'})
== 2025-12-20 08:06:18,029 easyconfig.py:1833 INFO Template values: arch='x86_64', bitbucket_account='pytorch', cuda_cc_cmake='80;70;61', cuda_cc_nvhpc='cc80,cc70,cc61', cuda_cc_semicolon_sep='8.0;7.0;6.1', cuda_cc_space_sep='8.0 7.0 6.1', cuda_cc_space_sep_no_period='80 70 61', cuda_compute_capabilities='8.0,7.0,6.1', cuda_int_comma_sep='80,70,61', cuda_int_semicolon_sep='80;70;61', cuda_int_space_sep='80 70 61', cuda_sm_comma_sep='sm_80,sm_70,sm_61', cuda_sm_space_sep='sm_80 sm_70 sm_61', cudamajver='12', cudaminver='9', cudashortver='12.9', cudaver='12.9.1', github_account='pytorch', module_name='PyTorch/2.9.1-foss-2025b-CUDA-12.9.1', mpi_cmd_prefix='mpirun -n 1', name='PyTorch', nameletter='P', nameletterlower='p', namelower='pytorch', pymajver='3', pyminver='13', pyshortver='3.13', pyver='3.13.5', rpath_enabled='false', software_commit='', sysroot='', toolchain_name='foss', toolchain_version='2025b', version='2.9.1', version_major='2', version_major_minor='2.9', version_major_minor_patch='2.9.1', version_minor='9', version_minor_patch='9.1', version_patch='1', versionprefix='', versionsuffix='-CUDA-12.9.1'
== 2025-12-20 08:06:18,031 one.py:179 INFO Skipping reformatting value for parameter 'toolchain'
== 2025-12-20 08:06:18,034 filetools.py:2081 INFO Creating directory /dev/shm/easybuild-tmplog/reprod_20251220080618_604372 (parents: True, set_gid: False, sticky: False)
== 2025-12-20 08:06:18,034 easyblock.py:5364 INFO Dumped easyconfig instance to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/PyTorch-2.9.1-foss-2025b-CUDA-12.9.1.eb
== 2025-12-20 08:06:18,035 filetools.py:2081 INFO Creating directory /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/easyblocks (parents: True, set_gid: False, sticky: False)
== 2025-12-20 08:06:18,038 filetools.py:2565 INFO /home/s3248973/.local/EasyBuildDev/easybuild-easyblocks/easybuild/easyblocks/p/pytorch.py copied to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/easyblocks/pytorch.py
== 2025-12-20 08:06:18,038 easyblock.py:5344 INFO Dumped easyblock pytorch.py required for reproduction to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/easyblocks
== 2025-12-20 08:06:18,038 filetools.py:2565 INFO /home/s3248973/.local/EasyBuildDev/easybuild-easyblocks/easybuild/easyblocks/generic/pythonpackage.py copied to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/easyblocks/pythonpackage.py
== 2025-12-20 08:06:18,038 easyblock.py:5344 INFO Dumped easyblock pythonpackage.py required for reproduction to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/easyblocks
== 2025-12-20 08:06:18,038 filetools.py:2081 INFO Creating directory /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/hooks (parents: True, set_gid: False, sticky: False)
== 2025-12-20 08:06:18,039 filetools.py:2565 INFO /home/s3248973/.local/EasyBuildDev/hooks.py copied to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/hooks/hooks.py
== 2025-12-20 08:06:18,039 easyblock.py:5376 INFO Dumped hooks file /home/s3248973/.local/EasyBuildDev/hooks.py which is (potentially) required for reproduction to /dev/shm/easybuild-tmplog/reprod_20251220080618_604372/hooks/hooks.py
== 2025-12-20 08:06:18,040 filetools.py:1933 INFO Adjusting permissions recursively for /data/horse/ws/s3248973-EasyBuild/easybuild-rapids/software/PyTorch/2.9.1-foss-2025b-CUDA-12.9.1
== 2025-12-20 08:06:18,041 easyblock.py:2499 INFO Number of iterations to perform for central part of installation procedure: 1
== 2025-12-20 08:06:18,041 build_log.py:329 INFO building and installing PyTorch/2.9.1-foss-2025b-CUDA-12.9.1...
== 2025-12-20 08:06:18,043 filetools.py:2142 INFO Lock /data/horse/ws/s3248973-EasyBuild/easybuild-rapids/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rapids_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock exists!
== 2025-12-20 13:06:35,414 build_log.py:233 ERROR EasyBuild encountered an error: Maximum wait time for lock /data/horse/ws/s3248973-EasyBuild/easybuild-rapids/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rapids_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock to be released reached: 18000 sec >= 18000 sec (at easybuild/tools/filetools.py:2158 in check_lock)
Callstack:
	easybuild/tools/filetools.py:2158 in check_lock
	easybuild/framework/easyblock.py:4917 in run_all_steps
	easybuild/framework/easyblock.py:5127 in build_and_install_one
	easybuild/main.py:178 in build_and_install_software
	easybuild/main.py:611 in process_eb_args
	easybuild/main.py:795 in main
	easybuild/main.py:844 in main_with_hooks
	easybuild/main.py:859 in <module>
== 2025-12-20 13:06:35,414 easyblock.py:392 INFO Closing log for application name PyTorch version 2.9.1