From: Petr Štetiar Date: Wed, 21 Jul 2021 19:31:28 +0000 (+0200) Subject: phase1,phase2: improve round robin builds X-Git-Tag: v1~14 X-Git-Url: http://git.openwrt.org/?a=commitdiff_plain;h=87ed6cf749bee5a22ffed22fad248d78a1144352;p=buildbot.git phase1,phase2: improve round robin builds There seems to be some issue with database updates, where database updates are asynchronous and thus the status update of the previous build (complete_at DB column) might take little bit longer, then preparation of the next/current build. This is issue during job prioritization as this might result in the build of the same builder twice in a row, for example: 2021-06-22 02:42:54+0000 [-] : build finished 2021-06-22 02:42:55+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-21 20:17:14+00:00 2021-06-22 03:14:30+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-22 02:42:54+00:00 Build finishes at 02:42:54, scheduler then asks for next build at 02:42:55, but the build still has the old complete_at timestamp 2021-06-21 20:17:14 instead of the correct one 2021-06-22 02:42:54, thus scheduling the build of oldest mipsel_24kc builder one more time. This is so far very promising workaround attempt which checks latest builder complete_at in builds table which seems to be updated faster, thus using greater complete_at value seems to work for now. References: https://github.com/buildbot/buildbot/issues/4592#issuecomment-801163587 Signed-off-by: Petr Štetiar --- diff --git a/phase1/master.cfg b/phase1/master.cfg index a85382a..be3a894 100644 --- a/phase1/master.cfg +++ b/phase1/master.cfg @@ -154,7 +154,21 @@ def getNewestCompleteTime(bldr): if not completed: return - return completed[0]['complete_at'] + complete_at = completed[0]['complete_at'] + + last_build = yield bldr.master.data.get( + ('builds', ), + [ + resultspec.Filter('builderid', 'eq', [bldrid]), + ], + order=['-started_at'], limit=1) + + if last_build and last_build[0]: + last_complete_at = last_build[0]['complete_at'] + if last_complete_at and (last_complete_at > complete_at): + return last_complete_at + + return complete_at @defer.inlineCallbacks def prioritizeBuilders(master, builders): diff --git a/phase2/master.cfg b/phase2/master.cfg index b9342cf..7742ad6 100644 --- a/phase2/master.cfg +++ b/phase2/master.cfg @@ -351,7 +351,21 @@ def getNewestCompleteTime(bldr): if not completed: return - return completed[0]['complete_at'] + complete_at = completed[0]['complete_at'] + + last_build = yield bldr.master.data.get( + ('builds', ), + [ + resultspec.Filter('builderid', 'eq', [bldrid]), + ], + order=['-started_at'], limit=1) + + if last_build and last_build[0]: + last_complete_at = last_build[0]['complete_at'] + if last_complete_at and (last_complete_at > complete_at): + return last_complete_at + + return complete_at @defer.inlineCallbacks def prioritizeBuilders(master, builders):