Bug in cwltool results in packed.cwl but no individual cwl files

[jjkoehorst]

Due to a bug in the cwltool (or maybe it is intentional) individual cwl files are missing from the PROV/workflow/ location and only the PROV/workflow/packed.cwl is available. This only happens when you start the workflow with a cwl:tool: argument in the input yaml file.

In theory the packed.cwl should be sufficient or not? Not sure if this is an issue in runcrate but I thought I would let you know.

http://download.systemsbiology.nl/unlock/cwl/issues/PROV_No_CWL.zip (Removed the .DS_Store files).

[mr-c]

@jjkoehorst What is your cwltool version? Can you give me an example I can try out? (I wasn't able to create a reproducer)

[jjkoehorst]

Using the following command and version of cwl

> cwltool --outdir OUT --provenance PROV quality/longread_no_kraken.yaml
INFO /Users/jasperk/mambaforge/bin/cwltool 3.1.20230513155734
INFO [cwltool] /Users/jasperk/mambaforge/bin/cwltool --outdir OUT --provenance PROV quality/longread_no_kraken.yaml
INFO Resolved 'quality/longread_no_kraken.yaml' to 'file:///Volumes/Git/m-unlock/cwl/tests/quality/longread_no_kraken.yaml'

It looks that the workflow finishes nicely but in PROV only packed.cwl is present

}INFO Final process status is success
INFO [provenance] Finalizing Research Object
INFO [provenance] Research Object saved to /Volumes/Git/m-unlock/cwl/tests/PROV
(base) ➜  tests git:(dev) ✗ find PROV | grep cwl
PROV/snapshot/workflow_longread_quality.cwl
PROV/workflow/packed.cwl
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.xml
PROV/metadata/provenance/primary.cwlprov.jsonld
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.jsonld
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.provn
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.nt
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.ttl
PROV/metadata/provenance/primary.cwlprov.json
PROV/metadata/provenance/primary.cwlprov.xml
PROV/metadata/provenance/primary.cwlprov.nt
PROV/metadata/provenance/workflow_20prepare_fasta_db.e8ff364c-ee48-4002-9405-d71de13defa0.cwlprov.json
PROV/metadata/provenance/primary.cwlprov.ttl
PROV/metadata/provenance/primary.cwlprov.provn

The job file:

 cat quality/longread_no_kraken.yaml
cwl:tool: ../../workflows/workflow_longread_quality.cwl
# cwl:tool: https://gitlab.com/m-unlock/cwl/-/raw/master/workflows/workflow_longread_quality.cwl
identifier: TEST
readtype: Nanopore
fastq_rich: true
threads: 6
memory: 4000
longreads:
   - class: File
     location: http://download.systemsbiology.nl/unlock/cwl/test_data/ont_guppy_reads.fq.gz
filter_references:
   - class: File
     location: http://download.systemsbiology.nl/unlock/cwl/test_data/human_small.fa.gz