I no longer want to make my game :(
I actually sat down long and hard and thought about the protocol a bit, and realized I overcomplicated the protocol.
The protocol is literally just: Given two “independent” games,
if you can show that all cross-games effects are bounded by time
T, you can implement a t+T time lockstep and be
provably correct. As in, if you haven’t received a global ledger
of all operations (from all clients and the server) for
t-T (at time t), you lock.
I overcomplicated this by trying to impose some a priori known
barrier time, B, and doing a lot of deferred
scheduling around said barriers, altering the game semantics
itself (so sending bloons could be logically delayed too).
There is still a nice result from this: the t+T
model synchronizes more than the B model in a worst
case network. But the t+T model is subject to basic
intuition about queuing theory - logical packets can “back up”,
but you can make ground by consuming all the packets really fast.
This actually has no problem on any of the servers except
the predictive views (which might look weird if there wasn’t some
kind of smoothing). The clients games are source nodes that tick
at a fixed rate, and the server wants to replicate and validate as
fast as possible.
But these results are not really that interesting. The basic idea of “bound the logical time that an interaction actually affects gameplay” is a pretty trivial distributed systems hackaround.
Actually creating a fun game is harder - Bloons TD Battles 1 was completely client authoritative, was rampant with hackers, and is still a really fun game. Creating a (slightly) fancy network architecture doesn’t solve the real hard part about design.
Well, the protocol has been reduced down far enough to where literally anybody could use dumb version of it. Maybe I can open source an SDK like GGPO did. I think I want to keep on thinking about what exactly the key invariants of the game I want to preserve was - I think local input micro is non-negotiable, but it restricts architectures to this one (thankfully, we have the constraint that game simulations are “mostly” independent).