Repository navigation

All pull requests

Labels (9)Milestones (6)New pull request

Open

(36)

Closed

(123)

Score MultiPL-E pass@k from a per-run temp dir
#328shaurya416 opened Sep 25, 2026
feat: Add code_x_glue code_to_code task
#317jonleinena opened Dec 11, 20252 of 3 tasks
[Community Submission] Model: Isaac-AI-OS/isaac-20b, Username: petermantell
#316pedroMantAmp opened Oct 15, 2025
1
Fix UnboundLocalError in APPS task evaluation
#310sad-mathematician opened Apr 20, 2025
Updated bigcode-evaluation-harness/leaderboard/README.md
#305zoya-hammad opened Mar 13, 2025
Support for remote inference
#302pawelknes opened Feb 17, 2025
use tokenizer.chat_template by default for instruction type tasks
#301TK-21st opened Jan 25, 2025
Missing comma in MultiPL-E languages
#299hrshtv opened Jan 13, 2025
2
[Pytest] Fix bad import to use relative instead @ module_test
#298ggcr opened Jan 11, 2025
Fix the bugs in the ds1000 sample bash script; Fix typos
#295gameofby opened Dec 10, 2024
Support multiple datasets from MBPP; Fix missing commas in python list; Fix doc typos;
#291gameofby opened Dec 4, 2024
add support for hpu devices
#281envsp opened Oct 25, 2024
1
"," missing in LANGUAGES list
#280ArtemisDicoTiar opened Oct 21, 2024
2
Speedup execute.py: Reuse same manager and dict in
#277michaelfeil opened Oct 1, 2024
Basecodes
#263Abhineetsoccer opened Aug 14, 2024
Add a new benchmark ENAMEL for evaluating the efficiency of LLM-generated code
#260q-rz opened Jul 22, 2024
Fix Max New Tokens in HF's Generation Config
#257mostafaelhoushi opened Jul 18, 2024
2
Fix unnecessary repeated overwrite
#249nielstron opened Jun 29, 2024
Add llama3 instruction prompts
#222TechxGenus opened Apr 19, 2024
1
Leaderboard README improvements
#217nikita1503 opened Apr 14, 2024
4
remove pad tokens added by the accelerator.pad_across_processes
#216IQ17 opened Apr 13, 2024
Ensure generations get saved in generation_only mode
#212Vipitis opened Mar 31, 2024
1
fix apps evaluate error: local variable 'level' referenced before assignment
#206koking0 opened Mar 10, 2024
2
Update README.md
#204AnitaLiu98 opened Mar 2, 2024
4
Fix loading PAL-GSM few-shot examples
#196sxjscience opened Feb 8, 2024
1

Pull requests