[Proposition] Results only competitions (with copy to predictions dir)

[ihsaan-ullah]

@Didayolo

Description

Now we have two types of phases:

1. Accepts only result submissions
2. Accepts both result and code submissions

Users can choose this option from yaml and from the interface

YAML documentation is updated here.

Now the submission content is copied to predictions output only when the phase to which the submission is submitted has accepts_only_result_submissions set to True.

I have added some comments to the compute worker code for clarification.

Issues this PR resolves

• Compute worker - Avoid duplicating submission files #1874

A checklist for hand testing

• Play with this new attribute and submit both result and code submissions. Check that files are duplicated only when accepts_only_result_submissions is true and that submissions are executed as expected.
• Check that from the interface and from yaml you can change this attribute
• Check that the comments added make sense and are easy to understand

Checklist

• Code review by me
• Hand tested by me
• I'm proud of my work
• Code review by reviewer
• Hand tested by reviewer
• CircleCi tests are passing
• Ready to merge

[ihsaan-ullah]

Closing this for now. I haven't tested this. I will reopen or create a new one after the tests

[Didayolo]

Related to #1874

[ihsaan-ullah]

@Didayolo This is ready for testing

[Didayolo]

@ihsaan-ullah

So for a competition like Iris that accepts both results and code submissions, won't results submissions fail?

[ihsaan-ullah]

Yes I think result submission will fail. BUT if you want both result and code submission to work, you have to set accepts_only_result_submissions to True.

Maybe we want to change the name of accepts_only_result_submissions to accepts_only_code_submissions and reverse the logic.

[Didayolo]

Post-it: think and discuss about the design. Should we simply allow the scoring program to access the submission?

Then we do not need to copy files, nor to have these flags like "result only competition".