I created a prize for short fiction with a $3500 prize pool, judged entirely by AI.
In short, the first stage of evaluation gets passed through four small models based on 21 literary criteria. If there’s agreement between the models, or if the first pass scores particularly well, the work is promoted to the second stage. Here, seven sub-agents which specialize in groups of the above criteria give two passes on the work. If the same scoring evaluations are met, all of the previous results are added to contexts for larger, thinking models to give a final score.
Currently, submissions are limited to short (2000 to 20,000 words) literary (not genre) fiction.
Once the submission period ends, and prizes are awarded, I’ll be opening it up for anyone to use.
gus_massa · 14h ago
Do you have some tests in case someone send the word banana repeated 10000 times?
Do you have some tests in case someone send Romeo and Juliet?
westernaccess · 14h ago
Of course, the banana example would be scored poorly almost instantly, and any recognitions of canonical work are flagged.
In short, the first stage of evaluation gets passed through four small models based on 21 literary criteria. If there’s agreement between the models, or if the first pass scores particularly well, the work is promoted to the second stage. Here, seven sub-agents which specialize in groups of the above criteria give two passes on the work. If the same scoring evaluations are met, all of the previous results are added to contexts for larger, thinking models to give a final score.
Currently, submissions are limited to short (2000 to 20,000 words) literary (not genre) fiction.
Once the submission period ends, and prizes are awarded, I’ll be opening it up for anyone to use.
Do you have some tests in case someone send Romeo and Juliet?
The prize is nonetheless in good faith.