(partial) EasyBuild log for failed build of /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.1-foss-2025b-CUDA-12.9.1.eb (PR(s) #24926)

PyTorch-2.9.1-foss-2025b-CUDA-12.9.1_partial.log

== 2025-12-28 20:36:34,612 easyblock.py:4765 INFO Running method extract_step part of step extract
== 2025-12-28 20:36:34,612 easyblock.py:2875 INFO Unpacking source pytorch-v2.9.1.tar.gz
== 2025-12-28 20:36:34,625 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'tar ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/tar-kn7xyxah/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/tar-kn7xyxah/out.txt
== 2025-12-28 20:36:34,626 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:34,626 run.py:518 INFO Running 'tar ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1:
	tar xzf /data/horse/ws/s3248973-EasyBuild/sources/p/PyTorch/pytorch-v2.9.1.tar.gz
== 2025-12-28 20:36:43,213 run.py:632 INFO 'tar ...' shell command completed successfully
== 2025-12-28 20:36:43,214 run.py:634 INFO Output of 'tar ...' shell command (stdout + stderr):

== 2025-12-28 20:36:43,215 easyblock.py:2875 INFO Unpacking source six-1.11.0.tar.gz
== 2025-12-28 20:36:43,228 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'tar ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/tar-ck2a4wm5/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/tar-ck2a4wm5/out.txt
== 2025-12-28 20:36:43,229 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,229 run.py:518 INFO Running 'tar ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1:
	tar xzf /data/horse/ws/s3248973-EasyBuild/sources/p/PyTorch/six-1.11.0.tar.gz
== 2025-12-28 20:36:43,250 run.py:632 INFO 'tar ...' shell command completed successfully
== 2025-12-28 20:36:43,251 run.py:634 INFO Output of 'tar ...' shell command (stdout + stderr):

== 2025-12-28 20:36:43,252 build_log.py:329 INFO ... (took 8 secs)
== 2025-12-28 20:36:43,252 build_log.py:329 INFO patching...
== 2025-12-28 20:36:43,252 easyblock.py:4754 INFO Starting patch step
== 2025-12-28 20:36:43,252 easyconfig.py:1814 INFO Generating template values...
== 2025-12-28 20:36:43,253 mpi.py:123 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'OpenMPI'
== 2025-12-28 20:36:43,253 mpi.py:299 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'nr_ranks': 1, 'cmd': 'xxx_command_xxx'})
== 2025-12-28 20:36:43,253 easyconfig.py:1833 INFO Template values: arch='x86_64', bitbucket_account='pytorch', builddir='/dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1', cuda_cc_cmake='80;70', cuda_cc_nvhpc='cc80,cc70', cuda_cc_semicolon_sep='8.0;7.0', cuda_cc_space_sep='8.0 7.0', cuda_cc_space_sep_no_period='80 70', cuda_compute_capabilities='8.0,7.0', cuda_int_comma_sep='80,70', cuda_int_semicolon_sep='80;70', cuda_int_space_sep='80 70', cuda_sm_comma_sep='sm_80,sm_70', cuda_sm_space_sep='sm_80 sm_70', cudamajver='12', cudaminver='9', cudashortver='12.9', cudaver='12.9.1', github_account='pytorch', installdir='/data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/PyTorch/2.9.1-foss-2025b-CUDA-12.9.1', module_name='PyTorch/2.9.1-foss-2025b-CUDA-12.9.1', mpi_cmd_prefix='mpirun -n 1', name='PyTorch', nameletter='P', nameletterlower='p', namelower='pytorch', parallel='64', pymajver='3', pyminver='13', pyshortver='3.13', pyver='3.13.5', rpath_enabled='false', software_commit='', start_dir='None', sysroot='', toolchain_name='foss', toolchain_version='2025b', version='2.9.1', version_major='2', version_major_minor='2.9', version_major_minor_patch='2.9.1', version_minor='9', version_minor_patch='9.1', version_patch='1', versionprefix='', versionsuffix='-CUDA-12.9.1'
== 2025-12-28 20:36:43,253 easyblock.py:4765 INFO Running method patch_step part of step patch
== 2025-12-28 20:36:43,253 easyblock.py:2892 INFO Applying patch PyTorch-1.12.1_add-hypothesis-suppression.patch
== 2025-12-28 20:36:43,266 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-dhrb77gu/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-dhrb77gu/out.txt
== 2025-12-28 20:36:43,267 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,268 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-1.12.1_add-hypothesis-suppression.patch
== 2025-12-28 20:36:43,285 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,285 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/quantization/core/test_quantized_op.py
Hunk #1 succeeded at 8362 (offset 2793 lines).

== 2025-12-28 20:36:43,286 easyblock.py:2892 INFO Applying patch PyTorch-1.7.0_disable-dev-shm-test.patch
== 2025-12-28 20:36:43,301 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-47icyuj6/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-47icyuj6/out.txt
== 2025-12-28 20:36:43,302 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,302 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-1.7.0_disable-dev-shm-test.patch
== 2025-12-28 20:36:43,315 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,316 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_multiprocessing.py
Hunk #1 succeeded at 266 (offset 64 lines).

== 2025-12-28 20:36:43,316 easyblock.py:2892 INFO Applying patch PyTorch-2.0.1_skip-tests-skipped-in-subprocess.patch
== 2025-12-28 20:36:43,334 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-gji2bueq/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-gji2bueq/out.txt
== 2025-12-28 20:36:43,335 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,335 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.0.1_skip-tests-skipped-in-subprocess.patch
== 2025-12-28 20:36:43,346 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,347 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/testing/_internal/common_distributed.py
Hunk #1 succeeded at 984 with fuzz 1 (offset 156 lines).
Hunk #2 succeeded at 1400 with fuzz 1 (offset 277 lines).

== 2025-12-28 20:36:43,347 easyblock.py:2892 INFO Applying patch PyTorch-2.1.0_remove-test-requiring-online-access.patch
== 2025-12-28 20:36:43,361 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-q17zzy88/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-q17zzy88/out.txt
== 2025-12-28 20:36:43,362 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,362 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.1.0_remove-test-requiring-online-access.patch
== 2025-12-28 20:36:43,375 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,375 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_cuda.py
Hunk #1 succeeded at 4081 with fuzz 1 (offset 709 lines).

== 2025-12-28 20:36:43,376 easyblock.py:2892 INFO Applying patch PyTorch-2.6.0_show-test-duration.patch
== 2025-12-28 20:36:43,391 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-8wlqzliu/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-8wlqzliu/out.txt
== 2025-12-28 20:36:43,392 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,392 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.6.0_show-test-duration.patch
== 2025-12-28 20:36:43,404 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,405 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/run_test.py
Hunk #1 succeeded at 1787 (offset -171 lines).
Hunk #2 succeeded at 1812 (offset -171 lines).

== 2025-12-28 20:36:43,405 easyblock.py:2892 INFO Applying patch PyTorch-2.6.0_skip-test_segfault.patch
== 2025-12-28 20:36:43,422 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-wuv2qzyr/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-wuv2qzyr/out.txt
== 2025-12-28 20:36:43,423 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,423 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.6.0_skip-test_segfault.patch
== 2025-12-28 20:36:43,436 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,437 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_dataloader.py
Hunk #1 succeeded at 1407 with fuzz 2 (offset 19 lines).

== 2025-12-28 20:36:43,437 easyblock.py:2892 INFO Applying patch PyTorch-2.7.0_avoid_caffe2_test_cpp_jit.patch
== 2025-12-28 20:36:43,452 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-k23wdw_b/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-k23wdw_b/out.txt
== 2025-12-28 20:36:43,453 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,453 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.0_avoid_caffe2_test_cpp_jit.patch
== 2025-12-28 20:36:43,464 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,465 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/testing/_internal/torchbind_impls.py
Hunk #1 succeeded at 175 (offset 59 lines).

== 2025-12-28 20:36:43,465 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_avoid-caffe2-sandcastle-test-lib.patch
== 2025-12-28 20:36:43,479 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-se0ltv1t/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-se0ltv1t/out.txt
== 2025-12-28 20:36:43,480 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,480 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_avoid-caffe2-sandcastle-test-lib.patch
== 2025-12-28 20:36:43,494 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,494 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_aot_inductor_custom_ops.py
Hunk #1 succeeded at 473 (offset 93 lines).

== 2025-12-28 20:36:43,495 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_data_parallel_rnn.patch
== 2025-12-28 20:36:43,510 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-qf8xjcqm/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-qf8xjcqm/out.txt
== 2025-12-28 20:36:43,511 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,512 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_data_parallel_rnn.patch
== 2025-12-28 20:36:43,524 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,525 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_data_parallel.py

== 2025-12-28 20:36:43,525 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_gds_fails_in_ci.patch
== 2025-12-28 20:36:43,539 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-39pqeg0v/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-39pqeg0v/out.txt
== 2025-12-28 20:36:43,540 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,540 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_gds_fails_in_ci.patch
== 2025-12-28 20:36:43,552 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,553 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_cuda.py
Hunk #1 succeeded at 3986 (offset 353 lines).

== 2025-12-28 20:36:43,553 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_mixed_mm_exhaustive_dtypes.patch
== 2025-12-28 20:36:43,567 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-7flinxux/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-7flinxux/out.txt
== 2025-12-28 20:36:43,568 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,568 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_mixed_mm_exhaustive_dtypes.patch
== 2025-12-28 20:36:43,583 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,584 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pattern_matcher.py
Hunk #1 succeeded at 382 (offset -7 lines).

== 2025-12-28 20:36:43,584 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-tests-requiring-SM90.patch
== 2025-12-28 20:36:43,601 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-awwv8_h5/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-awwv8_h5/out.txt
== 2025-12-28 20:36:43,601 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,602 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-tests-requiring-SM90.patch
== 2025-12-28 20:36:43,614 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,615 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py
Hunk #1 succeeded at 3350 with fuzz 1 (offset -17 lines).
Hunk #2 succeeded at 3358 (offset -17 lines).

== 2025-12-28 20:36:43,615 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_suport-64bit-BARs.patch
== 2025-12-28 20:36:43,629 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-2wnl6icx/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-2wnl6icx/out.txt
== 2025-12-28 20:36:43,630 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,630 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_suport-64bit-BARs.patch
== 2025-12-28 20:36:43,642 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,642 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file third_party/tensorpipe/tensorpipe/channel/cuda_gdr/context_impl.cc

== 2025-12-28 20:36:43,642 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_tolerance-test_partial_flat_weights.patch
== 2025-12-28 20:36:43,656 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-f9z0sojt/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-f9z0sojt/out.txt
== 2025-12-28 20:36:43,657 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,657 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.7.1_tolerance-test_partial_flat_weights.patch
== 2025-12-28 20:36:43,669 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,670 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_nn.py
Hunk #1 succeeded at 4321 (offset 22 lines).

== 2025-12-28 20:36:43,670 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_disable-test_nan_assert.patch
== 2025-12-28 20:36:43,685 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-hc598moe/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-hc598moe/out.txt
== 2025-12-28 20:36:43,685 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,686 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_disable-test_nan_assert.patch
== 2025-12-28 20:36:43,697 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,698 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py

== 2025-12-28 20:36:43,698 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_enable-symbolizer-in-test_workspace_allocation_error.patch
== 2025-12-28 20:36:43,712 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-lr5d8f1u/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-lr5d8f1u/out.txt
== 2025-12-28 20:36:43,713 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,713 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_enable-symbolizer-in-test_workspace_allocation_error.patch
== 2025-12-28 20:36:43,725 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,725 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_cudagraph_trees.py
Hunk #2 succeeded at 1674 (offset 130 lines).

== 2025-12-28 20:36:43,726 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-attention-squeeze.patch
== 2025-12-28 20:36:43,736 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-45b37a80/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-45b37a80/out.txt
== 2025-12-28 20:36:43,737 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,737 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-attention-squeeze.patch
== 2025-12-28 20:36:43,748 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,749 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/distributed/tensor/experimental/_attention.py
Hunk #1 succeeded at 152 (offset 18 lines).
Hunk #2 succeeded at 160 (offset 18 lines).
Hunk #3 succeeded at 225 (offset 18 lines).

== 2025-12-28 20:36:43,749 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-FP16-CPU-tests-in-test_torchinductor_opinfo.patch
== 2025-12-28 20:36:43,763 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-7t4l3q64/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-7t4l3q64/out.txt
== 2025-12-28 20:36:43,764 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,764 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-FP16-CPU-tests-in-test_torchinductor_opinfo.patch
== 2025-12-28 20:36:43,776 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,777 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_torchinductor_opinfo.py

== 2025-12-28 20:36:43,777 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-nccl-test-env.patch
== 2025-12-28 20:36:43,791 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-kp_b9uh9/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-kp_b9uh9/out.txt
== 2025-12-28 20:36:43,792 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,792 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-nccl-test-env.patch
== 2025-12-28 20:36:43,804 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,805 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py
Hunk #1 succeeded at 3087 (offset -58 lines).

== 2025-12-28 20:36:43,805 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-test_exclude_padding.patch
== 2025-12-28 20:36:43,819 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ycorvpkc/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ycorvpkc/out.txt
== 2025-12-28 20:36:43,820 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,820 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-test_exclude_padding.patch
== 2025-12-28 20:36:43,831 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,832 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pad_mm.py

== 2025-12-28 20:36:43,832 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-test_version_error.patch
== 2025-12-28 20:36:43,845 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-j09rakr4/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-j09rakr4/out.txt
== 2025-12-28 20:36:43,846 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,846 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-test_version_error.patch
== 2025-12-28 20:36:43,859 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,860 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/export/test_serialize.py

== 2025-12-28 20:36:43,860 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_honor-XDG_CACHE_HOME.patch
== 2025-12-28 20:36:43,873 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-opqs3251/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-opqs3251/out.txt
== 2025-12-28 20:36:43,873 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,874 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_honor-XDG_CACHE_HOME.patch
== 2025-12-28 20:36:43,885 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,886 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/csrc/distributed/c10d/FlightRecorder.cpp

== 2025-12-28 20:36:43,886 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_increase-tolerance-in-test_transformers.patch
== 2025-12-28 20:36:43,903 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-fjef8n3u/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-fjef8n3u/out.txt
== 2025-12-28 20:36:43,904 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,904 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_increase-tolerance-in-test_transformers.patch
== 2025-12-28 20:36:43,920 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,921 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_transformers.py

== 2025-12-28 20:36:43,921 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_remove-faulty-close.patch
== 2025-12-28 20:36:43,935 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ddv49a9p/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ddv49a9p/out.txt
== 2025-12-28 20:36:43,936 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,936 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_remove-faulty-close.patch
== 2025-12-28 20:36:43,947 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,947 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/_inductor/autotune_process.py
Hunk #1 succeeded at 864 (offset -18 lines).
Hunk #2 succeeded at 913 (offset -18 lines).

== 2025-12-28 20:36:43,948 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_revert-pybind11-3-change.patch
== 2025-12-28 20:36:43,962 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-gmoccs2b/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-gmoccs2b/out.txt
== 2025-12-28 20:36:43,962 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,963 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_revert-pybind11-3-change.patch
== 2025-12-28 20:36:43,974 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:43,975 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_cpu_cpp_wrapper.py
patching file torch/_inductor/codegen/cpp_wrapper_cpu.py
patching file torch/_inductor/codegen/cpp_wrapper_cpu_array_ref.py
patching file torch/csrc/inductor/cpp_wrapper/common.h

== 2025-12-28 20:36:43,975 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_benchmark_on_non_zero_device.patch
== 2025-12-28 20:36:43,989 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-o60p0hts/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-o60p0hts/out.txt
== 2025-12-28 20:36:43,990 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:43,990 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_benchmark_on_non_zero_device.patch
== 2025-12-28 20:36:44,002 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,002 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_benchmark_fusion.py

== 2025-12-28 20:36:44,002 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_convolution1-on-H100.patch
== 2025-12-28 20:36:44,016 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-f7jtx9ef/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-f7jtx9ef/out.txt
== 2025-12-28 20:36:44,017 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,017 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_convolution1-on-H100.patch
== 2025-12-28 20:36:44,028 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,029 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_select_algorithm.py

== 2025-12-28 20:36:44,029 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_inductor_all_gather_into_tensor_coalesced.patch
== 2025-12-28 20:36:44,046 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-pxwxrd6d/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-pxwxrd6d/out.txt
== 2025-12-28 20:36:44,046 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,047 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_inductor_all_gather_into_tensor_coalesced.patch
== 2025-12-28 20:36:44,059 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,059 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_functional_native.py

== 2025-12-28 20:36:44,060 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_original_aten_preserved_pad_mm.patch
== 2025-12-28 20:36:44,074 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-r05i6029/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-r05i6029/out.txt
== 2025-12-28 20:36:44,074 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,075 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_original_aten_preserved_pad_mm.patch
== 2025-12-28 20:36:44,093 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,094 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pad_mm.py
Hunk #1 succeeded at 511 (offset 3 lines).

== 2025-12-28 20:36:44,094 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_override-without-CUDA.patch
== 2025-12-28 20:36:44,108 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-13kx35da/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-13kx35da/out.txt
== 2025-12-28 20:36:44,109 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,109 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_override-without-CUDA.patch
== 2025-12-28 20:36:44,121 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,122 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_overrides.py

== 2025-12-28 20:36:44,122 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_unbacked_reduction.patch
== 2025-12-28 20:36:44,138 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-5z2o92sx/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-5z2o92sx/out.txt
== 2025-12-28 20:36:44,139 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,140 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_unbacked_reduction.patch
== 2025-12-28 20:36:44,153 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,153 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_torchinductor_dynamic_shapes.py

== 2025-12-28 20:36:44,154 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-tests-requiring-CUDA-12.8.patch
== 2025-12-28 20:36:44,168 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ci0ytyhk/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-ci0ytyhk/out.txt
== 2025-12-28 20:36:44,169 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,169 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-tests-requiring-CUDA-12.8.patch
== 2025-12-28 20:36:44,179 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,179 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_aot_inductor.py
patching file test/inductor/test_aot_inductor_package.py
patching file torch/testing/_internal/common_cuda.py

== 2025-12-28 20:36:44,180 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-unexpected-success-in-test_fake_export.patch
== 2025-12-28 20:36:44,195 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-sagr3a3x/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-sagr3a3x/out.txt
== 2025-12-28 20:36:44,196 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,196 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-unexpected-success-in-test_fake_export.patch
== 2025-12-28 20:36:44,207 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,208 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/export/test_export_opinfo.py

== 2025-12-28 20:36:44,208 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_ignore-warning-incompatible-pointer-types.patch
== 2025-12-28 20:36:44,222 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-fvacuj6g/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-fvacuj6g/out.txt
== 2025-12-28 20:36:44,223 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,223 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.1_ignore-warning-incompatible-pointer-types.patch
== 2025-12-28 20:36:44,235 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,235 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file CMakeLists.txt

== 2025-12-28 20:36:44,236 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_skip-RingFlexAttentionTest.patch
== 2025-12-28 20:36:44,250 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-cyfhx48j/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-cyfhx48j/out.txt
== 2025-12-28 20:36:44,250 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,251 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.1_skip-RingFlexAttentionTest.patch
== 2025-12-28 20:36:44,262 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:36:44,263 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/tensor/test_attention.py

== 2025-12-28 20:36:44,263 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_workaround-GCC-14-error.patch
== 2025-12-28 20:36:44,278 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-k6zx8e5o/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-pfc5is5u/run-shell-cmd-output/patch-k6zx8e5o/out.txt
== 2025-12-28 20:36:44,279 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:36:44,279 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.1_workaround-GCC-14-error.patch
== 2025-12-28 20:36:44,289 run.py:636 WARNING 'patch ...' shell command FAILED (exit code 1)
== 2025-12-28 20:36:44,290 run.py:637 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h.rej
patching file aten/src/ATen/native/cpu/Activation.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Activation.cpp.rej
patching file aten/src/ATen/native/cpu/Unfold2d.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Unfold2d.cpp.rej

== 2025-12-28 20:36:44,487 build_log.py:233 ERROR EasyBuild encountered an error: Couldn't apply patch file /tmp/easybuild-tmp/eb-pfc5is5u/files_pr24926/p/PyTorch/PyTorch-2.9.1_workaround-GCC-14-error.patch. Process exited with code 1: patching file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h.rej
patching file aten/src/ATen/native/cpu/Activation.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Activation.cpp.rej
patching file aten/src/ATen/native/cpu/Unfold2d.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Unfold2d.cpp.rej
 (at easybuild/tools/filetools.py:1773 in apply_patch)
Callstack:
	easybuild/tools/filetools.py:1773 in apply_patch
	easybuild/framework/easyblock.py:2925 in patch_step
	easybuild/framework/easyblock.py:4781 in run_step
	easybuild/framework/easyblock.py:4940 in run_all_steps
	easybuild/framework/easyblock.py:5127 in build_and_install_one
	easybuild/main.py:178 in build_and_install_software
	easybuild/main.py:611 in process_eb_args
	easybuild/main.py:795 in main
	easybuild/main.py:844 in main_with_hooks
	easybuild/main.py:859 in <module>
== 2025-12-28 20:36:44,487 build_log.py:329 INFO ... (took 1 secs)
== 2025-12-28 20:36:44,487 filetools.py:2172 INFO Removing lock /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock...
== 2025-12-28 20:36:44,490 filetools.py:423 INFO Path /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock successfully removed.
== 2025-12-28 20:36:44,491 filetools.py:2176 INFO Lock removed: /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock
== 2025-12-28 20:36:44,491 easyblock.py:392 INFO Closing log for application name PyTorch version 2.9.1