TCO22 Stage 2 / 3 boundary

112 posts Mon, Dec 20, 2021 at 09:21 AM in General Marathon Discussions

In the announcement email of the new Capacitated Vehicle Routing match, the schedule mentioned this match (and also the upcoming Real Time Cognitive State one) will both be part of Stage 3. In the past matches starting in December were still considered part of Stage 2. I can somewhat understand putting the two new ones into Stage 3 as they were both announced quite late (I never heard of the capacitated vehicle one until I got the email that it started), but it would be nice to have an official answer here.


View: Threaded  |  Flat


  • 99 posts Mon, Dec 27, 2021 at 02:07 PM

    It would be nice to have an answer, Especially the Vehicle Routing match is already over.
    In addition, we are still waiting on the final result for MM130, stage 2 is about to end, or I think it is, it would be nice to know quickly either if some addition challenge are counting towards stage 2, or what are the final results to know who qualified.


  • 165 posts Mon, Dec 27, 2021 at 03:25 PM

    @vdave @sullyper checking on this - will get back to you

  • 99 posts Fri, Dec 31, 2021 at 01:53 AM

    any updates?
    also with stage 3 starting, MM132 starting soon, it would be nice to know who qualified from stage 2, this requires both MM130 and MM131 to have final official results, and to have a decision on which sponsored challenges were/are part of state 2.

  • 52 posts Fri, Dec 31, 2021 at 06:54 AM

    And one more question on this topic: Does the "Cognitive State Determination Marathon Match" count towards the TCO'22 Marathon track? The name suggests it does, but in the "Submission Review" system it shows up as a "Cd" ("Code") competition, not as a "MM" one.

  • 30 posts Fri, Dec 31, 2021 at 09:42 AM

    I guess Capacitated Vehicle Routing match belongs to stage 3 (as it was menionted in emails) since Stage 2 was already announced (with MM130 and MM131) when it was added.

    Of course it is really bad decision to add a match exactly when another match is starting and even worse to make it part of TCO, but even Topcoder admins wouldn't add it to the stage that was already finishing.

  • 112 posts Sun, Jan 2, 2022 at 08:15 PM

    The only source of truth I could find was the announcement email for Capacitated Vehicle Routing Part 3, which contained the following section: email part. Based on that both sponsored matches belongs to Stage 3 (which is in contrast to how stage picking worked in previous years, e.g. for TCO20, the Mortgage Pooling marathon which started in December 2019 and ended in January 2020 counted towards the Oct - Dec 2019 stage). With the Cognitive State one still running and MM132 starting in 10 days, it would be nice to have official answers here.

  • 52 posts Mon, Jan 3, 2022 at 08:00 PM

    The rules are pretty clear regarding how matches are allocated to stages: it's based on the date when the submissions open: "Only Tournament-eligible Marathon Competitions in which submission has opened during the four (4) Stages shall be included in determining points for that Stage. Note: that for Stage 4, matches in which submission opens during Stage 4 and ends after Stage 4 won’t be included in determining points for Stage 4."

    Based on the published rules, the Capacitated Vehicle Routing match and the Cognitive State Determination match should count for Stage 2. It's a bit sad to just break the published rules - especially the Capacitated Vehicle Routing match makes no sense to me to be part of Stage 3, given that submissions both opened and closed during Stage 2. It doesn't matter when it was announced - in fact, there are easy arguments to be made that it shouldn't be part of any stage (it was announced too late for Stage 2 and too early for Stage 3 - presumably competitors planning to participate in Stage 3 weren't expecting a Stage 3 match taking place during Stage 2). So maybe the right thing to do is to not award any TCO points to the Capacitated Vehicle Routing match (that would also not be fair to competitors who participated hoping to get such TCO points).

  • 112 posts Mon, Jan 3, 2022 at 08:42 PM

    I'm afraid at this point there cannot be any fair decision.

    Making CVR part of Stage 2 is unfair for people who did not participate assuming it's in Stage 3 (from multiple Topcoder emails) and thought they already have enough points for TCO22.
    Making it part of Stage 3 is indeed violating the rules, though given it was announced for Stage 3 multiple times (in the announcement emails of both MM131 and CVP), I feel this is the least bad decision.
    Not giving TCO points for it is unfair for everyone who spent lots of time participating in it, especially that it was scheduled through the entire Christmas weekend.

  • 379 posts Tue, Jan 4, 2022 at 03:42 PM

    My apology for the response delay. Over the holidays it's always a bit wonky.

    Per the TCO22 Rules, the matches that start in Stage 2 are part of Stage 2. I missed the typo in the email so my apologies for the confusion. I'm working with a new team on the process of MMs going forward. Less mishaps and more consistency/improvements. I really appreciate your patience. You guys are so important to us. I'm looking forward to "housekeeping" this month and getting it right going forward.

    Thanks again for your patience!

  • 112 posts Tue, Jan 4, 2022 at 03:49 PM

    Thanks for the official answer, Jessie. Just to be crystal clear, this means that both the Capacitated Vehicle Routing 3 (which ended in December) and the currently running NASA Cognitive State Predictions (started in December, but still ongoing for ~2 weeks) will count towards Stage 2, is that correct?

  • 379 posts Tue, Jan 4, 2022 at 03:54 PM


  • 99 posts Wed, Jan 5, 2022 at 10:30 AM

    Thanks @jmpld40, could we also have the final result of MM130 and MM131 as well as the leaderboard for stage 2 updated with the results of the first 3 challenges.

  • 379 posts Wed, Jan 5, 2022 at 11:10 AM

    Yes, the team is actively working on it. Hope to have an update from them soon.

  • 99 posts Sun, Jan 9, 2022 at 10:20 AM

    @jmpld40 Sorry to bug you again with that, but any update? MM132 is going to start soon, so it would be nice to have a clearer picture of what happened in stage 2 (eventhough one DS MM is still running for stage 2, which is also quite unfortunate, the official results of the other 3 + the current leaderboard of the last DS MM running can provide some overview).
    In general having the final result of the previous MM before the next one starts should be nice, and it would be nice to not have 2 MMs running in the same time.

  • 99 posts Mon, Jan 17, 2022 at 09:15 AM

    ping =)

  • 52 posts Tue, Jan 25, 2022 at 11:22 AM

    @jmpld40: Any update on the TCO'22 Marathon Stage 2 Leaderboard? It's still empty, although 3 out of 4 competitions have long been completed (with final results available).

    By the way, I computed the TCO points for Stage 2 myself here, if that helps in any way (although I've done this manually, so I can't guarantee I haven't made any mistakes). If anything, it shows that it's mathematically impossible for @sullyper to not qualify from Stage 2 (congrats!). The other spots are still up for grabs, though.

  • 379 posts Tue, Jan 25, 2022 at 11:27 AM

    Should be up today, Tuesday. Apologies for the delay. Seems a step was missed and it was unclear. But now we are all sorted out!

  • 112 posts Wed, Jan 26, 2022 at 03:04 PM

    @jmpld40 The Marathon stage 2 leaderboard is still empty.

  • 99 posts Thu, Jan 27, 2022 at 08:37 AM

    Tuesday? Like next week? =)

  • 379 posts Thu, Jan 27, 2022 at 09:26 PM

    Tuesday as in Thursday :)

    Sorry guys, truly am. Trust me we are now on top of it ;)

  • 52 posts Thu, Jan 27, 2022 at 09:52 PM

    Hi @jmpld40,

    I now see that the Stage 2 Marathon leaderboard is not empty, which is great. But the points are wrong. For one, I see @RafbillFr with 240 points out of 2 challenges - this should be mathematically impossible, since the maximum number of points per challenge is 100.

    After taking a closer look at the points, I think I reverse-engineered the issue: the TCO points for MM 130 are multiplied by 2. So in the published leaderboard:

    • @RafbillFr has 200 points from MM 130 instead of 100
    • I have 150 points from MM 130 instead of 75
    • @nika has 120 points from MM 130 instead of 60
    • ....

    The fix should be pretty simple.

  • 112 posts Thu, Jan 27, 2022 at 10:02 PM

    I think there's something else as well, there are 16 people with the score of 1, which is quite impossible from 3 matches.

  • 112 posts Fri, Jan 28, 2022 at 08:22 AM

    Actually this too many people with score 1 issue is also present on the Stage 1 leaderboard, I have the impression that a score of 1 is given to everyone with a valid solution, while based on the rules only people in the top 30 should receive TCO points.

  • 134 posts Fri, Jan 28, 2022 at 12:26 PM edited Jan 28, 2022 at 12:29 PM

    New MM problem: reverse engineer how the MM leaderboard was incorrectly calculated :)

    It would be kind of cool if some MM gave you double points. Maybe that would encourage more participation :)

    On a serious note, this leaderboard is clearly wrong and we will hopefully fix it soon.

  • 99 posts Fri, Jan 28, 2022 at 01:43 PM

    If you want to double the points:
    - Announce it at the beginning of the stage, that's it's fair for everyone
    - Don't make it the first one, make it the last one, more fun like that =)

  • 379 posts Fri, Jan 28, 2022 at 10:55 AM

    The team that does this is currently out of office so I envision it will be after the weekend. Again, my apologies.

  • 57 posts Fri, Jan 28, 2022 at 04:25 PM

    MM leaderboard for stage 2 is updated. Sincere apologies for all the delay.

  • 112 posts Fri, Jan 28, 2022 at 04:33 PM

    @ufo_believer I think the duplicate points issue is indeed fixed, but there are still two problem with the leaderboard.
    1. @iehn and @bowwowforeach have points, even though they both qualified for the TCO22 finals in Stage 1.
    2. There are still lots of people with one point, with 3 matches, only up to 3 people could have ended up with 1 point. This issue is also there on the Stage 1 leaderboard (though not affecting the top 3 in either case).

  • 57 posts Fri, Jan 28, 2022 at 04:59 PM

    Thanks for highlighting this. As Jessie mentioned, we're working with the new team and these issues will be fixed once we update the scores for Cognitive State Determination Marathon Match for STAGE 2.

  • 112 posts Wed, Feb 2, 2022 at 08:19 PM

    I believe the Cognitive State marathon has final results now, it would be nice to fix the leaderboard (hopefully also for Stage 1 where it still contains some people with score 1 who never finished in the top 30).

    My own calculations are here (but as I have personal interest, don't assume mine is correct):

  • 52 posts Tue, Feb 8, 2022 at 10:03 PM

    I independently computed the same scores as yours. Congrats on snatching the 3rd spot from Stage 2 and qualifying to another MM Final, @vdave! (should be official as soon as the Cognitive State Determination MM points are also added to the official Stage 2 leaderboard). And sorry to @RafbillFr, @nika and @ap31, who were all very close! I wish them good luck in the remaining stages.

  • 112 posts Thu, Feb 10, 2022 at 07:46 AM

    Thanks, @mugurelionut, given the lots of delays and uncertainty around Marathons lately, I rather wait the official announcement before celebrating.

  • 112 posts Mon, Feb 14, 2022 at 10:27 PM

    @ufo_believer A friendly ping about final Stage 2 results, the Cognitive State marathon has final results for some time. I've seen that the Stage 3 leaderboard had also been populated now, but without a final Stage 2 ranking that may also be incorrect.

  • 57 posts Tue, Feb 15, 2022 at 07:56 AM edited Feb 15, 2022 at 11:59 AM

    Hey @vdave, We will update the leaderboard by the end of today, sorry for the delay !

  • 57 posts Tue, Feb 15, 2022 at 01:57 PM

    Hey guys, we have updated the leaderboard now. regarding the issue with many members having 1 points, as we checked it doesn't have any effect on the leaderboard results tself. So we decided not to make any changes to the current leaderboard and follow these steps from upcoming Marathon matches, thank you, and again we apologise for the delay, we will try to keep up now .. thanks..

  • 112 posts Tue, Feb 15, 2022 at 02:02 PM

    Thanks, @ufo_believer, one last thing. With the changes, now I would advance to the finals from Stage 2, which means I should not be getting any points for Stage 3 (where I'm still listed with 40 points from MM132).

  • 57 posts Tue, Feb 15, 2022 at 02:22 PM

    yes we will redo the scores for stage 3..

  • 14 posts Fri, May 13, 2022 at 01:54 PM edited May 14, 2022 at 01:56 PM

    I see that the stage 3 leaderboard was updated. However it seems incomplete to me.

    • vdave is still listed with 40 points despite already being qualified from stage 2 (which affects the points of everyone ranked below vdave in MM132)
    • not all players are considered for MM134 and Nasa Comet Detection. e.g. yowa and myself got points for MM134 while simanman and nika didn't. ~(this has been solved now) Same for the Comet Detection where ceres1 got points but selim_sef doesn't show up in stage 3.
  • 14 posts Tue, May 17, 2022 at 02:58 PM

    I made my own computations now:
    The hardest part for me is to tell which contest gives TCO points and which doesn't, so you can turn them on and off. Points are then recomputed via Javascript.

  • 112 posts Tue, May 17, 2022 at 03:17 PM edited May 17, 2022 at 03:18 PM

    I think your computation still makes one of the mistakes from the official leaderboard, namely that based on the rules only position #30 gets 1 point and not everyone who participated. If a stage had N matches, there can be at most N people with exactly 1 point.

  • 99 posts Tue, May 17, 2022 at 03:32 PM

    This is not 100% accurate either as tie could lead to more than one person getting 1 point.
    But otherwise yes, only the top 30 should get points, Top Coder made this mistake in stage 1 I believe (while it had/has no impact on the final top 3, it was/is indeed a mistake)

  • 99 posts Tue, May 17, 2022 at 03:35 PM

    MM130 belongs to stage 2 and not stage 1

  • 14 posts Tue, May 17, 2022 at 04:38 PM

    @vdave the official rules say "All other positive scores: 1 Point" below the score distribution.
    @sullyper interesting, MM130 shows as stage 2 for me. I think it's a problem with the time zone. Topcoder didn't mention which one is to be considered for the stage boundaries. In my local time it's Oct 1st but it can definitely be September for you. I'll correct this.
    @ufo_believer thank you for the background info, that's a sad story indeed.

  • 99 posts Tue, May 17, 2022 at 04:44 PM

    Funny enough, it shows as 2021-10-01.
    I would assume they are using EST or ET timezone, but as it starts at 1:00 am ET, it should have been the same day on the pacific.

  • 112 posts Tue, May 17, 2022 at 06:43 PM

    I see, this everyone gets a point is then new in the rules in 2022, in previous years only the top 30 got points. On the other hand, I believe MM130 was definitely Stage 2, it started on Oct 13 ( I believe the match entry was created Oct 1 (maybe even Sep 30 in some timezones), but submission started on Oct 13.

  • 57 posts Tue, May 17, 2022 at 04:13 PM

    Hey @eulerschezahl,
    selim_sef is not on the leaderboard as he is from belarus, and unfortunately we had to exclude him.

Sign In or Register to comment.