Latency issue: Force sounds into Loop Pedal, back into Force

I've been using the Force in various ways since 2020.

Currently, I use it most often for the software instruments & drum banks as playable sounds while my Headrush Looperboard is the final output :
Akai Force goes into my Looperboard along with my microphone and Chapman Stick.
ALL GOOD.

But then recently, I thought it would be cool to be able to send the Looperboard loops back into the Akai Force in order to use the cool DJ-style FX included in the Force (since the Looperboard fx are very limited) and I get a lot of FX-envy seeing what people do with the Boss 505 looper.

Once I send Loop audio into the Force, the sound quality is groovy and I can have fun with Filter sweeps etc.
But as soon as I start to record new loops along with the existing loop, the Latency begins to cascade and the new loop won't be in sync.

My next attempt will be to add an Audio Interface with the Force.
I don't expect this to change the latency issue, since the Akai still does all the sound processing.
I'll add an update if anything changes.

I'm not very optimistic that there could be a solution, but into which machine do you record when it gets out of sync, the Headrush or the Force?

If there's a solution, it probably consists of components like (1) measuring various point-to-point latencies in the signal path, (2) inserting compensating delay buffers in one or more points along the signal path, (3) automatically shifting timing offsets of recorded audio based on configured latencies, and (4) changing the routings and creating separate paths for live listening/monitoring and recording.

It may be that solution components 2 and 3 are not possible in either the Force or the Headrush device.

The only thing I know is that the Force has an adjustable sample delay plugin you can insert to channels, submixes and outputs, and that can provide some compensating delay. BUT that may not be enough to solve the whole problem. Or maybe the overall realtime latency gets too bad, after you have inserted the compensating delays.

YMMV, but I recommend drawing the whole signal routing path on paper. Without a clear map of what's happening, it will be difficult to even find the right spots to adjust, or figure out what your adjustments are doing.