(partial) EasyBuild log for failed build of /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.1-foss-2025b-CUDA-12.9.1.eb (PR(s) #24926)

PyTorch-2.9.1-foss-2025b-CUDA-12.9.1_partial.log

== 2025-12-28 20:47:08,027 easyblock.py:4765 INFO Running method extract_step part of step extract
== 2025-12-28 20:47:08,027 easyblock.py:2875 INFO Unpacking source pytorch-v2.9.1.tar.gz
== 2025-12-28 20:47:08,040 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'tar ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/tar-t92gx4py/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/tar-t92gx4py/out.txt
== 2025-12-28 20:47:08,041 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:08,041 run.py:518 INFO Running 'tar ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1:
	tar xzf /data/horse/ws/s3248973-EasyBuild/sources/p/PyTorch/pytorch-v2.9.1.tar.gz
== 2025-12-28 20:47:16,593 run.py:632 INFO 'tar ...' shell command completed successfully
== 2025-12-28 20:47:16,593 run.py:634 INFO Output of 'tar ...' shell command (stdout + stderr):

== 2025-12-28 20:47:16,594 easyblock.py:2875 INFO Unpacking source six-1.11.0.tar.gz
== 2025-12-28 20:47:16,607 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'tar ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/tar-4rxu_cey/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/tar-4rxu_cey/out.txt
== 2025-12-28 20:47:16,608 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,608 run.py:518 INFO Running 'tar ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1:
	tar xzf /data/horse/ws/s3248973-EasyBuild/sources/p/PyTorch/six-1.11.0.tar.gz
== 2025-12-28 20:47:16,630 run.py:632 INFO 'tar ...' shell command completed successfully
== 2025-12-28 20:47:16,630 run.py:634 INFO Output of 'tar ...' shell command (stdout + stderr):

== 2025-12-28 20:47:16,631 build_log.py:329 INFO ... (took 8 secs)
== 2025-12-28 20:47:16,631 build_log.py:329 INFO patching...
== 2025-12-28 20:47:16,632 easyblock.py:4754 INFO Starting patch step
== 2025-12-28 20:47:16,632 easyconfig.py:1814 INFO Generating template values...
== 2025-12-28 20:47:16,632 mpi.py:123 INFO Using template MPI command 'mpirun -n %(nr_ranks)s %(cmd)s' for MPI family 'OpenMPI'
== 2025-12-28 20:47:16,632 mpi.py:299 INFO Using MPI command template 'mpirun -n %(nr_ranks)s %(cmd)s' (params: {'nr_ranks': 1, 'cmd': 'xxx_command_xxx'})
== 2025-12-28 20:47:16,633 easyconfig.py:1833 INFO Template values: arch='x86_64', bitbucket_account='pytorch', builddir='/dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1', cuda_cc_cmake='80;70', cuda_cc_nvhpc='cc80,cc70', cuda_cc_semicolon_sep='8.0;7.0', cuda_cc_space_sep='8.0 7.0', cuda_cc_space_sep_no_period='80 70', cuda_compute_capabilities='8.0,7.0', cuda_int_comma_sep='80,70', cuda_int_semicolon_sep='80;70', cuda_int_space_sep='80 70', cuda_sm_comma_sep='sm_80,sm_70', cuda_sm_space_sep='sm_80 sm_70', cudamajver='12', cudaminver='9', cudashortver='12.9', cudaver='12.9.1', github_account='pytorch', installdir='/data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/PyTorch/2.9.1-foss-2025b-CUDA-12.9.1', module_name='PyTorch/2.9.1-foss-2025b-CUDA-12.9.1', mpi_cmd_prefix='mpirun -n 1', name='PyTorch', nameletter='P', nameletterlower='p', namelower='pytorch', parallel='64', pymajver='3', pyminver='13', pyshortver='3.13', pyver='3.13.5', rpath_enabled='false', software_commit='', start_dir='None', sysroot='', toolchain_name='foss', toolchain_version='2025b', version='2.9.1', version_major='2', version_major_minor='2.9', version_major_minor_patch='2.9.1', version_minor='9', version_minor_patch='9.1', version_patch='1', versionprefix='', versionsuffix='-CUDA-12.9.1'
== 2025-12-28 20:47:16,633 easyblock.py:4765 INFO Running method patch_step part of step patch
== 2025-12-28 20:47:16,633 easyblock.py:2892 INFO Applying patch PyTorch-1.12.1_add-hypothesis-suppression.patch
== 2025-12-28 20:47:16,647 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-vblwr6dn/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-vblwr6dn/out.txt
== 2025-12-28 20:47:16,648 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,648 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-1.12.1_add-hypothesis-suppression.patch
== 2025-12-28 20:47:16,663 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,664 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/quantization/core/test_quantized_op.py
Hunk #1 succeeded at 8362 (offset 2793 lines).

== 2025-12-28 20:47:16,664 easyblock.py:2892 INFO Applying patch PyTorch-1.7.0_disable-dev-shm-test.patch
== 2025-12-28 20:47:16,677 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-daomx2su/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-daomx2su/out.txt
== 2025-12-28 20:47:16,678 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,678 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-1.7.0_disable-dev-shm-test.patch
== 2025-12-28 20:47:16,688 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,688 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_multiprocessing.py
Hunk #1 succeeded at 266 (offset 64 lines).

== 2025-12-28 20:47:16,688 easyblock.py:2892 INFO Applying patch PyTorch-2.0.1_skip-tests-skipped-in-subprocess.patch
== 2025-12-28 20:47:16,700 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-nqwwirh9/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-nqwwirh9/out.txt
== 2025-12-28 20:47:16,701 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,701 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.0.1_skip-tests-skipped-in-subprocess.patch
== 2025-12-28 20:47:16,712 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,712 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/testing/_internal/common_distributed.py
Hunk #1 succeeded at 984 with fuzz 1 (offset 156 lines).
Hunk #2 succeeded at 1400 with fuzz 1 (offset 277 lines).

== 2025-12-28 20:47:16,713 easyblock.py:2892 INFO Applying patch PyTorch-2.1.0_remove-test-requiring-online-access.patch
== 2025-12-28 20:47:16,724 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-fgev0ob6/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-fgev0ob6/out.txt
== 2025-12-28 20:47:16,725 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,725 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.1.0_remove-test-requiring-online-access.patch
== 2025-12-28 20:47:16,736 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,737 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_cuda.py
Hunk #1 succeeded at 4081 with fuzz 1 (offset 709 lines).

== 2025-12-28 20:47:16,737 easyblock.py:2892 INFO Applying patch PyTorch-2.6.0_show-test-duration.patch
== 2025-12-28 20:47:16,749 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-gs_ei7xe/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-gs_ei7xe/out.txt
== 2025-12-28 20:47:16,750 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,750 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.6.0_show-test-duration.patch
== 2025-12-28 20:47:16,761 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,762 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/run_test.py
Hunk #1 succeeded at 1787 (offset -171 lines).
Hunk #2 succeeded at 1812 (offset -171 lines).

== 2025-12-28 20:47:16,762 easyblock.py:2892 INFO Applying patch PyTorch-2.6.0_skip-test_segfault.patch
== 2025-12-28 20:47:16,775 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-mm23_5hi/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-mm23_5hi/out.txt
== 2025-12-28 20:47:16,775 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,776 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.6.0_skip-test_segfault.patch
== 2025-12-28 20:47:16,785 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,786 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_dataloader.py
Hunk #1 succeeded at 1407 with fuzz 2 (offset 19 lines).

== 2025-12-28 20:47:16,786 easyblock.py:2892 INFO Applying patch PyTorch-2.7.0_avoid_caffe2_test_cpp_jit.patch
== 2025-12-28 20:47:16,798 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-p7t1zd06/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-p7t1zd06/out.txt
== 2025-12-28 20:47:16,799 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,799 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.0_avoid_caffe2_test_cpp_jit.patch
== 2025-12-28 20:47:16,808 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,809 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/testing/_internal/torchbind_impls.py
Hunk #1 succeeded at 175 (offset 59 lines).

== 2025-12-28 20:47:16,809 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_avoid-caffe2-sandcastle-test-lib.patch
== 2025-12-28 20:47:16,821 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-ew4va3zf/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-ew4va3zf/out.txt
== 2025-12-28 20:47:16,821 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,822 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_avoid-caffe2-sandcastle-test-lib.patch
== 2025-12-28 20:47:16,832 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,832 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_aot_inductor_custom_ops.py
Hunk #1 succeeded at 473 (offset 93 lines).

== 2025-12-28 20:47:16,833 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_data_parallel_rnn.patch
== 2025-12-28 20:47:16,844 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-xzvunna7/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-xzvunna7/out.txt
== 2025-12-28 20:47:16,845 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,845 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_data_parallel_rnn.patch
== 2025-12-28 20:47:16,854 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,855 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_data_parallel.py

== 2025-12-28 20:47:16,855 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_gds_fails_in_ci.patch
== 2025-12-28 20:47:16,866 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-t_yiwcl7/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-t_yiwcl7/out.txt
== 2025-12-28 20:47:16,867 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,867 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_gds_fails_in_ci.patch
== 2025-12-28 20:47:16,878 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,878 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_cuda.py
Hunk #1 succeeded at 3986 (offset 353 lines).

== 2025-12-28 20:47:16,878 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-test_mixed_mm_exhaustive_dtypes.patch
== 2025-12-28 20:47:16,891 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-adckm8lv/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-adckm8lv/out.txt
== 2025-12-28 20:47:16,892 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,892 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-test_mixed_mm_exhaustive_dtypes.patch
== 2025-12-28 20:47:16,902 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,902 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pattern_matcher.py
Hunk #1 succeeded at 382 (offset -7 lines).

== 2025-12-28 20:47:16,902 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_skip-tests-requiring-SM90.patch
== 2025-12-28 20:47:16,914 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-eesyv8_1/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-eesyv8_1/out.txt
== 2025-12-28 20:47:16,915 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,915 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_skip-tests-requiring-SM90.patch
== 2025-12-28 20:47:16,926 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,926 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py
Hunk #1 succeeded at 3350 with fuzz 1 (offset -17 lines).
Hunk #2 succeeded at 3358 (offset -17 lines).

== 2025-12-28 20:47:16,926 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_suport-64bit-BARs.patch
== 2025-12-28 20:47:16,939 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-ucjtehc_/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-ucjtehc_/out.txt
== 2025-12-28 20:47:16,939 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,940 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_suport-64bit-BARs.patch
== 2025-12-28 20:47:16,949 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,950 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file third_party/tensorpipe/tensorpipe/channel/cuda_gdr/context_impl.cc

== 2025-12-28 20:47:16,950 easyblock.py:2892 INFO Applying patch PyTorch-2.7.1_tolerance-test_partial_flat_weights.patch
== 2025-12-28 20:47:16,962 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-jg3r43cm/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-jg3r43cm/out.txt
== 2025-12-28 20:47:16,962 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,963 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.7.1_tolerance-test_partial_flat_weights.patch
== 2025-12-28 20:47:16,974 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,974 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_nn.py
Hunk #1 succeeded at 4321 (offset 22 lines).

== 2025-12-28 20:47:16,975 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_disable-test_nan_assert.patch
== 2025-12-28 20:47:16,986 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-lqui12jv/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-lqui12jv/out.txt
== 2025-12-28 20:47:16,987 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:16,987 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_disable-test_nan_assert.patch
== 2025-12-28 20:47:16,997 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:16,998 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py

== 2025-12-28 20:47:16,998 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_enable-symbolizer-in-test_workspace_allocation_error.patch
== 2025-12-28 20:47:17,010 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-55cgtwrw/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-55cgtwrw/out.txt
== 2025-12-28 20:47:17,011 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,011 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_enable-symbolizer-in-test_workspace_allocation_error.patch
== 2025-12-28 20:47:17,022 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,022 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_cudagraph_trees.py
Hunk #2 succeeded at 1674 (offset 130 lines).

== 2025-12-28 20:47:17,023 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-attention-squeeze.patch
== 2025-12-28 20:47:17,036 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-csp5104e/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-csp5104e/out.txt
== 2025-12-28 20:47:17,037 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,037 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-attention-squeeze.patch
== 2025-12-28 20:47:17,048 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,048 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/distributed/tensor/experimental/_attention.py
Hunk #1 succeeded at 152 (offset 18 lines).
Hunk #2 succeeded at 160 (offset 18 lines).
Hunk #3 succeeded at 225 (offset 18 lines).

== 2025-12-28 20:47:17,049 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-FP16-CPU-tests-in-test_torchinductor_opinfo.patch
== 2025-12-28 20:47:17,061 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-__7n4t1b/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-__7n4t1b/out.txt
== 2025-12-28 20:47:17,062 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,062 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-FP16-CPU-tests-in-test_torchinductor_opinfo.patch
== 2025-12-28 20:47:17,071 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,072 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_torchinductor_opinfo.py

== 2025-12-28 20:47:17,072 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-nccl-test-env.patch
== 2025-12-28 20:47:17,083 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-1ph5pv3h/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-1ph5pv3h/out.txt
== 2025-12-28 20:47:17,084 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,084 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-nccl-test-env.patch
== 2025-12-28 20:47:17,095 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,095 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_nccl.py
Hunk #1 succeeded at 3087 (offset -58 lines).

== 2025-12-28 20:47:17,096 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-test_exclude_padding.patch
== 2025-12-28 20:47:17,107 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-fyudlhwn/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-fyudlhwn/out.txt
== 2025-12-28 20:47:17,108 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,108 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-test_exclude_padding.patch
== 2025-12-28 20:47:17,118 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,119 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pad_mm.py

== 2025-12-28 20:47:17,119 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_fix-test_version_error.patch
== 2025-12-28 20:47:17,131 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-i2ev80ge/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-i2ev80ge/out.txt
== 2025-12-28 20:47:17,132 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,132 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_fix-test_version_error.patch
== 2025-12-28 20:47:17,143 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,143 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/export/test_serialize.py

== 2025-12-28 20:47:17,144 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_honor-XDG_CACHE_HOME.patch
== 2025-12-28 20:47:17,155 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-sfkxq6gh/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-sfkxq6gh/out.txt
== 2025-12-28 20:47:17,155 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,155 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_honor-XDG_CACHE_HOME.patch
== 2025-12-28 20:47:17,166 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,166 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/csrc/distributed/c10d/FlightRecorder.cpp

== 2025-12-28 20:47:17,166 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_increase-tolerance-in-test_transformers.patch
== 2025-12-28 20:47:17,178 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-d_o9uzpq/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-d_o9uzpq/out.txt
== 2025-12-28 20:47:17,179 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,179 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_increase-tolerance-in-test_transformers.patch
== 2025-12-28 20:47:17,190 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,190 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_transformers.py

== 2025-12-28 20:47:17,191 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_remove-faulty-close.patch
== 2025-12-28 20:47:17,206 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-9kxtv0yy/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-9kxtv0yy/out.txt
== 2025-12-28 20:47:17,207 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,207 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_remove-faulty-close.patch
== 2025-12-28 20:47:17,221 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,221 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file torch/_inductor/autotune_process.py
Hunk #1 succeeded at 864 (offset -18 lines).
Hunk #2 succeeded at 913 (offset -18 lines).

== 2025-12-28 20:47:17,221 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_revert-pybind11-3-change.patch
== 2025-12-28 20:47:17,236 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-rqp641ts/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-rqp641ts/out.txt
== 2025-12-28 20:47:17,237 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,238 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_revert-pybind11-3-change.patch
== 2025-12-28 20:47:17,251 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,252 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_cpu_cpp_wrapper.py
patching file torch/_inductor/codegen/cpp_wrapper_cpu.py
patching file torch/_inductor/codegen/cpp_wrapper_cpu_array_ref.py
patching file torch/csrc/inductor/cpp_wrapper/common.h

== 2025-12-28 20:47:17,252 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_benchmark_on_non_zero_device.patch
== 2025-12-28 20:47:17,266 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-6w2w6kvv/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-6w2w6kvv/out.txt
== 2025-12-28 20:47:17,267 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,267 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_benchmark_on_non_zero_device.patch
== 2025-12-28 20:47:17,279 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,279 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_benchmark_fusion.py

== 2025-12-28 20:47:17,280 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_convolution1-on-H100.patch
== 2025-12-28 20:47:17,296 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-veluwpdw/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-veluwpdw/out.txt
== 2025-12-28 20:47:17,297 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,297 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_convolution1-on-H100.patch
== 2025-12-28 20:47:17,308 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,309 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_select_algorithm.py

== 2025-12-28 20:47:17,309 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_inductor_all_gather_into_tensor_coalesced.patch
== 2025-12-28 20:47:17,324 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-hkd9ym5n/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-hkd9ym5n/out.txt
== 2025-12-28 20:47:17,325 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,325 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_inductor_all_gather_into_tensor_coalesced.patch
== 2025-12-28 20:47:17,337 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,338 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/test_c10d_functional_native.py

== 2025-12-28 20:47:17,338 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_original_aten_preserved_pad_mm.patch
== 2025-12-28 20:47:17,351 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-_9_p2oya/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-_9_p2oya/out.txt
== 2025-12-28 20:47:17,352 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,352 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_original_aten_preserved_pad_mm.patch
== 2025-12-28 20:47:17,363 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,363 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_pad_mm.py
Hunk #1 succeeded at 511 (offset 3 lines).

== 2025-12-28 20:47:17,364 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_override-without-CUDA.patch
== 2025-12-28 20:47:17,379 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-xltnrano/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-xltnrano/out.txt
== 2025-12-28 20:47:17,379 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,380 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_override-without-CUDA.patch
== 2025-12-28 20:47:17,390 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,390 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/test_overrides.py

== 2025-12-28 20:47:17,390 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-test_unbacked_reduction.patch
== 2025-12-28 20:47:17,403 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-cg1hjoyk/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-cg1hjoyk/out.txt
== 2025-12-28 20:47:17,404 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,404 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-test_unbacked_reduction.patch
== 2025-12-28 20:47:17,414 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,414 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_torchinductor_dynamic_shapes.py

== 2025-12-28 20:47:17,415 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-tests-requiring-CUDA-12.8.patch
== 2025-12-28 20:47:17,427 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-czxdd0sa/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-czxdd0sa/out.txt
== 2025-12-28 20:47:17,427 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,428 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-tests-requiring-CUDA-12.8.patch
== 2025-12-28 20:47:17,438 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,439 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/inductor/test_aot_inductor.py
patching file test/inductor/test_aot_inductor_package.py
patching file torch/testing/_internal/common_cuda.py

== 2025-12-28 20:47:17,439 easyblock.py:2892 INFO Applying patch PyTorch-2.9.0_skip-unexpected-success-in-test_fake_export.patch
== 2025-12-28 20:47:17,451 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-3e0k8ag5/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-3e0k8ag5/out.txt
== 2025-12-28 20:47:17,452 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,452 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.0_skip-unexpected-success-in-test_fake_export.patch
== 2025-12-28 20:47:17,462 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,462 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/export/test_export_opinfo.py

== 2025-12-28 20:47:17,463 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_ignore-warning-incompatible-pointer-types.patch
== 2025-12-28 20:47:17,474 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-471oymhw/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-471oymhw/out.txt
== 2025-12-28 20:47:17,475 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,475 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.1_ignore-warning-incompatible-pointer-types.patch
== 2025-12-28 20:47:17,485 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,485 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file CMakeLists.txt

== 2025-12-28 20:47:17,486 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_skip-RingFlexAttentionTest.patch
== 2025-12-28 20:47:17,498 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-n2zghmfu/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-n2zghmfu/out.txt
== 2025-12-28 20:47:17,499 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,500 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.1_skip-RingFlexAttentionTest.patch
== 2025-12-28 20:47:17,509 run.py:632 INFO 'patch ...' shell command completed successfully
== 2025-12-28 20:47:17,509 run.py:634 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file test/distributed/tensor/test_attention.py

== 2025-12-28 20:47:17,510 easyblock.py:2892 INFO Applying patch PyTorch-2.9.1_workaround-GCC-14-error.patch
== 2025-12-28 20:47:17,522 run.py:478 INFO run_shell_cmd: Script to start debug shell for 'patch ...' shell command
	will be saved to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-n50zi5ps/cmd.sh
	output will be logged to /tmp/easybuild-tmp/eb-y0i1enc4/run-shell-cmd-output/patch-n50zi5ps/out.txt
== 2025-12-28 20:47:17,523 run.py:504 INFO Path to bash that will be used to run shell commands: /usr/bin/bash
== 2025-12-28 20:47:17,523 run.py:518 INFO Running 'patch ...' shell command in /dev/shm/s3248973-EasyBuild/PyTorch/2.9.1/foss-2025b-CUDA-12.9.1/pytorch-v2.9.1:
	patch  -p1 -i /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.1_workaround-GCC-14-error.patch
== 2025-12-28 20:47:17,533 run.py:636 WARNING 'patch ...' shell command FAILED (exit code 1)
== 2025-12-28 20:47:17,533 run.py:637 INFO Output of 'patch ...' shell command (stdout + stderr):
patching file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h.rej
patching file aten/src/ATen/native/cpu/Activation.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Activation.cpp.rej
patching file aten/src/ATen/native/cpu/Unfold2d.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Unfold2d.cpp.rej

== 2025-12-28 20:47:17,727 build_log.py:233 ERROR EasyBuild encountered an error: Couldn't apply patch file /tmp/easybuild-tmp/eb-y0i1enc4/files_pr24926/p/PyTorch/PyTorch-2.9.1_workaround-GCC-14-error.patch. Process exited with code 1: patching file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/cpu/vec/sve/vec_bfloat16.h.rej
patching file aten/src/ATen/native/cpu/Activation.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Activation.cpp.rej
patching file aten/src/ATen/native/cpu/Unfold2d.cpp
Reversed (or previously applied) patch detected!  Assume -R? [n] 
Apply anyway? [n] 
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file aten/src/ATen/native/cpu/Unfold2d.cpp.rej
 (at easybuild/tools/filetools.py:1773 in apply_patch)
Callstack:
	easybuild/tools/filetools.py:1773 in apply_patch
	easybuild/framework/easyblock.py:2925 in patch_step
	easybuild/framework/easyblock.py:4781 in run_step
	easybuild/framework/easyblock.py:4940 in run_all_steps
	easybuild/framework/easyblock.py:5127 in build_and_install_one
	easybuild/main.py:178 in build_and_install_software
	easybuild/main.py:611 in process_eb_args
	easybuild/main.py:795 in main
	easybuild/main.py:844 in main_with_hooks
	easybuild/main.py:859 in <module>
== 2025-12-28 20:47:17,728 build_log.py:329 INFO ... (took 1 secs)
== 2025-12-28 20:47:17,728 filetools.py:2172 INFO Removing lock /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock...
== 2025-12-28 20:47:17,731 filetools.py:423 INFO Path /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock successfully removed.
== 2025-12-28 20:47:17,732 filetools.py:2176 INFO Lock removed: /data/horse/ws/s3248973-EasyBuild/easybuild-rome/software/.locks/_data_horse_ws_s3248973-EasyBuild_easybuild-rome_software_PyTorch_2.9.1-foss-2025b-CUDA-12.9.1.lock
== 2025-12-28 20:47:17,732 easyblock.py:392 INFO Closing log for application name PyTorch version 2.9.1