A run of monitor-gating has just failed in prod for the single-package workflow.
The suspects are 'datagrepper, datagrepper, resultsdb, datagrepper, datagrepper, datagrepper'.
Full log:
01:03:38 - Cloning as packagerbot the git repo: rpms/dummy-test-package-gloster 01:03:47 - Cloning as packagerbot the git repo: rpms/dummy-test-package-gloster [DONE] 01:03:47 - Switching to branch: master 01:03:48 - Switching to branch: master [DONE] 01:03:48 - Bumping release of: dummy-test-package-gloster.spec 01:03:48 - Bumping release of: dummy-test-package-gloster.spec [DONE] 01:03:48 - Commiting changes 01:03:48 - Commiting changes [DONE] 01:03:48 - Getting nevr 01:03:49 - Getting nevr [DONE] 01:03:49 - Upcoming build : dummy-test-package-gloster-0-2530.fc34 [DONE] 01:03:49 - Pushing changes 01:03:54 - Pushing changes [DONE] 01:03:54 - The build should be found on: https://koji.fedoraproject.org/koji/search?match=glob&type=build&terms=dummy-test-package-gloster-0-2530.fc34 [DONE] 01:03:54 - The build should be found in: https://koji.fedoraproject.org/koji/search?match=glob&type=package&terms=dummy-test-package-gloster [DONE] 01:03:54 - Building the package 01:12:58 - Building the package [DONE] 01:12:58 - Retrieving koji tags 01:12:59 - Retrieving koji tags: ['f34-updates-candidate', 'f34-signing-pending'] found while expected ['updates-candidate', 'signing-pending'] [DONE] 01:12:59 - Retrieving update created from https://bodhi.fedoraproject.org/updates/?builds=dummy-test-package-gloster-0-2530.fc34 01:12:59 - Retrieving update created from https://bodhi.fedoraproject.org/updates/?builds=dummy-test-package-gloster-0-2530.fc34 [DONE] 01:12:59 - Update automatically created : https://bodhi.fedoraproject.org/updates/FEDORA-2021-5859ed1d0e [DONE] 01:12:59 - Retrieving koji tags 01:12:59 - Retrieving koji tags: ['f34-updates-candidate', 'f34-signing-pending'] found while expected ['signing-pending', 'testing-pending'] [DONE] 01:12:59 - Checking datagrepper for bodhi to CI messages 01:20:35 - bodhi to CI results in datagrepper returned - ran for: 455s [DONE] 01:20:35 - Checking datagrepper for CI (running) messages 04:21:03 - CI (running) results not found in datagrepper - ran for: 10828s [FAILED] 04:21:03 - Checking datagrepper for CI (complete) messages 04:21:04 - CI (complete) results not found in datagrepper - ran for: 1s [FAILED] 04:21:04 - Retrieving koji tags 04:21:10 - Retrieving koji tags: ['f34-updates-candidate', 'f34-updates-testing-pending'] found while expected ['testing-pending'] [DONE] 04:21:10 - CI results query from https://taskotron.fedoraproject.org/resultsdb_api/api/v2.0/results&page=0 [DONE] 04:21:10 - Checking resultsdb(phx) for CI results 05:21:34 - CI results did not show in resultsdb(phx) for dummy-test-package-gloster-0-2530.fc34 within 3600 minutes since 2021-01-28 04:21:10.691119 - ran for: 3623s [FAILED] 05:21:34 - Checking datagrepper for resultsdb messages 05:21:37 - resultsdb results not found in datagrepper - ran for: 3s [FAILED] 05:21:37 - Checking datagrepper for greenwave messages 05:21:37 - greenwave results not found in datagrepper - ran for: 0s [FAILED] 05:21:37 - Retrieving koji tags 05:21:39 - Retrieving koji tags: ['f34-updates-candidate', 'f34-updates-testing-pending'] found while expected ['testing-pending'] [DONE] 05:21:39 - Waiving test results for bodhi update 05:21:44 - Waiving test results for bodhi update [DONE] 05:21:44 - Checking datagrepper for waiverdb messages 05:21:44 - waiverdb results in datagrepper returned - ran for: 0s [DONE] 05:21:44 - Checking datagrepper for greenwave messages 05:21:44 - greenwave results not found in datagrepper - ran for: 0s [FAILED] 05:21:44 - Retrieving koji tags 05:24:50 - Retrieving koji tags: ['f34'] found while expected f34 [DONE] Ran for 15671 seconds (261.18 minutes)
intermittent issues with greenwave:
zodbot> PROBLEM - rabbitmq01.iad2.fedoraproject.org/Check queue greenwave is WARNING: RABBITMQ_QUEUE WARNING - messages WARNING (36), messages_ready OK (11) messages_unacknowledged OK (25) consumers OK (1) (noc01)
Metadata Update from @asaleh: - Issue status updated to: Closed (was: Open)