Fixing handling None filtered score · EvolvingLMMs-Lab/lmms-eval@79514ee (original) (raw)
Navigation Menu
- GitHub Copilot Write better code with AI
- GitHub Models New Manage and compare prompts
- GitHub Advanced Security Find and fix vulnerabilities
- Actions Automate any workflow
- Codespaces Instant dev environments
- Issues Plan and track work
- Code Review Manage code changes
- Discussions Collaborate outside of code
- Code Search Find more, search less
- Explore
- Pricing
Provide feedback
Saved searches
Use saved searches to filter your results more quickly
Appearance settings
Commit 79514ee
Fixing handling None filtered score
File tree
1 file changed
lines changed
- lmms_eval/tasks/wild_vision_bench
1 file changed
lines changed
Lines changed: 3 additions & 0 deletions
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -152,6 +152,9 @@ def wild_vision_process_results(doc, results): | ||
| 152 | 152 | resps, gpt_name = get_chat_response(base64_image, user_prompt) |
| 153 | 153 | score, _ = get_score(resps, pattern=re.compile("\[\[([AB<>=]+)\]\]")) |
| 154 | 154 | |
| 155 | +if score is None: | |
| 156 | +score = resps | |
| 157 | + | |
| 155 | 158 | if "A>B" in score: |
| 156 | 159 | final_score = -1 |
| 157 | 160 | judgement = "Worse" #Baseline better |