Fortran runtime error: Index '143' of dimension 1 of array 'cpu_use' above upper bound of 64

[LMurphy186232]

I recently compiled ED2 for use on a new-to-me HPC system. I got the following error at runtime:

At line 200 of file edmain.F90 Fortran runtime error: Index '143' of dimension 1 of array 'cpu_use' above upper bound of 64
with an unhelpful backtrace. I investigated this error, and I think I know what is happening.

This is the code in question in edmain.F90, which starts at line 178:

!---------------------------------------------------------------------------------------!
! Check OMP thread and processor use and availability. This is done outside the !
! preprocessor "if block" because it is possible to activate multithread without !
! compiling with mpif90. !
! !
! Note: One could use omp_get_num_threads() in loop, but that would depend on how many !
! threads were open at the time of its call. !
!---------------------------------------------------------------------------------------!
max_threads = 1
num_procs = 1
thread = 1
cpu = 1
thread_use(:) = 0
cpu_use(:) = 0
!$ max_threads = omp_get_max_threads()
!$ num_procs = omp_get_num_procs()

!$OMP PARALLEL DO DEFAULT(SHARED) PRIVATE(thread,cpu)
do n = 1,max_threads
!$ thread = omp_get_thread_num() + 1
!$ cpu = findmycpu() + 1
thread_use(thread) = 1
cpu_use(cpu) = 1
end do
!$OMP END PARALLEL DO
!---------------------------------------------------------------------------------------!

The line "cpu = findmycpu() + 1" appears to be returning a CPU ID number, which may be larger than the "cpu_use" max array length of 64 on an HPC system. The array overrun is causing an error to be thrown.

I do not have the OMP coding chops to write a pull request to fix this, unfortunately, so I am filing a bug report instead. This appears to be needed for an informational banner (only?), so I just commented it out in my version, and I am back up and running.

[xiangtaoxu]

Thanks, Lora! Have requested more than 64 cores for your job? Are you able to run without error if you request <64 cores in your HPC system?

[mpaiao]

Thanks for flagging this @LMurphy186232. If I recall correctly, this OMP code was written by @rgknox a long time ago. Looking back at it, I wonder if the solution should be simply to turn both thread_use and cpu_use into allocatable arrays.

I drafted a quick PR, @LMurphy186232 would you mind trying it to see if this solves the issue? Thanks!

[LMurphy186232]

The current approach is to build a vector indexed by CPU ID, with 1s for used CPUs and 0s for unused slots. Summing the vector then counts the number of CPUs in use. I think the original assumption for the code would be that the CPU IDs returned would start at 0 and count up from there. But on my system they do not. So I may request a single core, the code is allocating an array of length 1 and expecting a value of 1, but the core ID is 143, which overruns the array.

I think the better approach would be to build a vector of CPU IDs, and then count the unique values. Since this doesn't end up being an OMP issue at all as I originally thought, I can tackle a fix for this since my system is the one that reliably crashes with this bug.

[mpaiao]

I'd say it would still be good to get rid of the hard-coded 64 and make the vectors allocatable (but initialising them after they are allocated, I pushed a fix for it). But then I agree with you that we should count unique IDs.

[LMurphy186232]

100% agree, I will keep the dynamic allocation.