ansible/ansible/a26c325bd8f6e2822d9d7e62f77a424c1db4fbf6 · v0f01c69f1e2528b935359cfe578530722bca2c59

Fix for ansible/ansible@a26c325b

Verdict

+168.2%
Lineman cost extra $0.2387
Baseline cost$0.1419
Lineman cost$0.3806
Baseline turns10
Lineman turns22
Baseline duration40.9s
Lineman duration1m 34s
Lineman version0.1.0

Quality - SWE-bench Docker Harness

Baseline
Not yet evaluated
Lineman
Not yet evaluated

Analysis

(narrative skipped)

Baseline

Turns10
Tool calls9
Cache read195,512
Cache write13,383
Output tokens2,205
Total cost$0.1419

Lineman

Turns22+120%
Tool calls21+133%
Cache read523,375+168%
Cache write39,060+192%
Output tokens5,139+133%
Total cost$0.3806

Tool Call Timeline

#Baseline toolCharsLineman toolCharsΔ
1Read/private/var/folders/04/qmf7f1qx7nj2_2mczkd4msth0000gn/T/bench-baseline-instance_ansible__ansible-a26c325bd8f6e2822d9d7e62f77a424c1db4fbf6-v0f01c69f1e2528b935359cfe578530722bca2c59/ISSUE.md2.0klinemanread_file /private/var/folders/04/qmf7f1qx7nj2_2mczkd4msth0000gn/T/bench-lineman-instance_ansible__ansible-a26c325bd8f6e2822d9d7e62f77a424c1db4fbf6-v0f01c69f1e2528b935359cfe578530722bca2c59/ISSUE.md2.4k+24%
2Glob**/uri.py92Grepnetrc45-51%
3Glob**/urls.py67Glob**/uri.py499+645%
4Grepnetrc82Glob**/uri.py92+12%
5Grepnetrc523linemanread_file_context lib/ansible/module_utils/urls.py — netrc authorization header use_netrc48-91%
21edit_file/private/var/folders/04/qmf7f1qx7nj2_2mczkd4msth0000gn/T/bench-lineman-instance_ansible__ansible-a26c325bd8f6e2822d9d7e62f77a424c1db4fbf6-v0f01c69f1e2528b935359cfe578530722bca2c59/lib/ansible/module_utils/urls.py229
Total9.2k28.8k+213%