OSError: [Errno 28] No space left on device: '/var/lib/mock/f39-python-43510926-5227179/buildroot.lock'

in https://koji.fedoraproject.org/koji/taskinfo?taskID=102220773

My last few builds have started failing with similar issues:

https://koji.fedoraproject.org/koji/taskinfo?taskID=102220437
https://koji.fedoraproject.org/koji/taskinfo?taskID=102220450
https://koji.fedoraproject.org/koji/taskinfo?taskID=102220469
https://koji.fedoraproject.org/koji/taskinfo?taskID=102220470
https://koji.fedoraproject.org/koji/taskinfo?taskID=102220520
https://koji.fedoraproject.org/koji/taskinfo?taskID=102220521

Seems to be buildvm-x86-28.iad2.fedoraproject.org

I have disabled the builder

Oddly, it doesn't say it's out of space now (but it may have been eariler?)

Filesystem      Size  Used Avail Use% Mounted on
/dev/vda2       255G   61G  195G  24% /

It does seem to have the epel7 cleanup issue:

Jun 16 19:17:40 buildvm-x86-28.iad2.fedoraproject.org kojid[719]: 2023-06-16 19:17:40,825 [ERROR] {719} koji:517 Unable to remove directory lib: [Errno 39] Directory not empty: 'lib'
Jun 16 19:17:40 buildvm-x86-28.iad2.fedoraproject.org kojid[719]: 2023-06-16 19:17:40,825 [WARNING] {719} koji.build:96 file removal failed for /var/lib/mock/epel7-build-43509800-5226604/root/var

I'll try and investigate more next week.

Closing this issue because it seems to be fixed as it was discussed in the daily.

Reopened. Sorry, this will be investigated, closed by mistake

So, this I think was the btrfs metadata thing. It should be worked around right now...