open-discussion
open-discussion > RE: Restart job if files_out are missing
Feb 27, 2017 04:02 AM | Pierre Bellec
RE: Restart job if files_out are missing
I somehow missed this post, sorry for the very late answer. PSOM
does not automatically reprocess jobs with missing outputs. The
main reason is that PSOM allows for "clean up" jobs, i.e. jobs that
remove files. If you re-run a pipeline to change a job at the end,
you will not expect to re-run all jobs that generate intermediate
outputs, unless those are necessary to run the jobs that have
changed.
So, when re-running pipelines, any change in options or input/output file names will trigger a re-start of a job. If this job requires inputs that are missing, and that can be generated by other jobs, those will also be restarted (and that behavior is recursive).
You can force to restart the job called 'job' with the following option for `psom_run_pipeline`:
opt.restart = { 'job' };
There is no flag to restart jobs with missing outputs. Would not be hard to implement though, if you think this is useful.
Best,
Pierre
So, when re-running pipelines, any change in options or input/output file names will trigger a re-start of a job. If this job requires inputs that are missing, and that can be generated by other jobs, those will also be restarted (and that behavior is recursive).
You can force to restart the job called 'job' with the following option for `psom_run_pipeline`:
opt.restart = { 'job' };
There is no flag to restart jobs with missing outputs. Would not be hard to implement though, if you think this is useful.
Best,
Pierre
Threaded View
Title | Author | Date |
---|---|---|
Joel Parker | Jan 24, 2017 | |
Pierre Bellec | Feb 27, 2017 | |
Joel Parker | May 17, 2017 | |