concat.sbatch issue - CONFIG

[manuparra]

Hello @Jordatious @Kitchi , I have already managed to adapt the pipeline to our infrastructure and in general to any other following the PR, but after several tests, I see that there is a problem in the concatenation phase:

Inside the submission file, this part that is executed:

...
#concat.sbatch
allSPWIDs+=,$(sbatch -d afterany:${IDs//,/:} concat.sbatch | cut -d ' ' -f4)
....

When checking for errors with findErrors.sh, I see the following:

• All the above processes terminate correctly, but just at the AllSPW part:

....
All SPWs: /mnt/slurm-jobs/pipelines
logs/partition-2487_0.err  logs/partition-2487_10.err  logs/partition-2487_3.err  logs/partition-2487_5.err  logs/partition-2487_7.err	logs/partition-2487_9.err
logs/partition-2487_0.out  logs/partition-2487_10.out  logs/partition-2487_3.out  logs/partition-2487_5.out  logs/partition-2487_7.out	logs/partition-2487_9.out
logs/partition-2487_1.err  logs/partition-2487_2.err   logs/partition-2487_4.err  logs/partition-2487_6.err  logs/partition-2487_8.err
logs/partition-2487_1.out  logs/partition-2487_2.out   logs/partition-2487_4.out  logs/partition-2487_6.out  logs/partition-2487_8.out
logs/concat-2599.casa  logs/concat-2599.err  logs/concat-2599.out
AttributeError: module 'processMeerKAT' has no attribute 'CONFIG'
srun: error: espsrc-slurm-01: task 0: Exited with exit code 1

Checking the log specifically I see:

cat  logs/concat-2599.err
/var/spool/slurmd/job02599/slurm_script: line 15: module: command not found
Traceback (most recent call last):
  File "/mnt/slurm-jobs/pipelines/processMeerKAT/aux_scripts/concat.py", line 173, in <module>
    bookkeeping.run_script(main,logfile)
  File "/mnt/slurm-jobs/pipelines/processMeerKAT/bookkeeping.py", line 330, in run_script
    args = config_parser.parse_args()
  File "/mnt/slurm-jobs/pipelines/processMeerKAT/config_parser.py", line 16, in parse_args
    parser.add_argument('-C','--config', default=processMeerKAT.CONFIG, required=False, help='Name of the input config file')
AttributeError: module 'processMeerKAT' has no attribute 'CONFIG'
srun: error: espsrc-slurm-01: task 0: Exited with exit code 1

It looks like an issue with paths or access to the config file. The previous steps for each SPW worked well without errors.
Any ideas?

Regards,
Manu.

[mb010]

I think it should be referring to this CONFIG variable here. I'm not sure why it wouldn't. Check your paths and imports to see if CONFIG can be read in, otherwise you could probably get away with changing this line to a file path with the default name.