How to reset increasing audio offset?

 

Hi there.

When running a longer show (like 8+ hours) we experience an increasing audio offset on the talent.
We start perfectly lipsync (using an audio delay of 120ms) but over the course of the day we end up with the audio being 1-2 seconds offset from the talent's action.

Restarting Axy fixes this issue, but unfortunately a restart in the middle of the show is not really an option.

Is there any way to trigger a re-sync of the audio similar to what a restart does?
(A short interruption of the audio in such a case wouldn't be an issue)

Thanks for any hint & all the best.

Eric.


   EricWest

 
Profile Image
EricWest
  -  

To zero in further on the culprit...

We're just running another long term test.
For about 5-6 hours audio was absolutely in sync.

Then (just around the same time it happened yesterday) out of a sudden we have a 25 frame offset.
Not slowly increasing, just happening.

As if something in our setup (particle system maybe?) needs to re-initalize for some reason, causing the sudden audio delay.

Will keep investigating and updating this thread... :-)

 
Profile Image
EricWest
  -  

Approved after filming a metronome for 7 hours...

After 6:12:24 the audio suddenly gets an extra second (25 frames).
Audio is pushed forwards (so sound comes too late), video runs flawlessly without any stutter.
Simultaneously recorded SystemStats don't show anything suspicious either.

Will keep investigating....

 
Profile Image
EricWest
  -  

Verified the Offset issue with a very simple Axy setup.
(No 3D at all, just a camera and audio input, some 2D stats, recorder, that's it)

Again after 6+ hours of UpTime the same 'out of the blue' Audio Offset of 25 Frames (1 second).
(In yesterday's test at FrameID 558649, UpTime 06:13:04
in todays's test at FrameID 558474, UpTime 06:12:59)
That's just 5 seconds apart, so for the course of 6 hours I'd say this is close enough....

Video comes in via HDMI, Audio via Dante Virtual Soundcard.

Will keep recording over night to see if we 'win' another extra second after 6 hours... :-)

 
Profile Image
TwentyStudios
  -  

@EricWest: We’ve had lots of issues with Dante in particular in Aximmetry. Eventually we just gave up on audio in Aximmetry and route everything directly to the mixer instead. I know Aximmetry had support for ASIO earlier, with a special startup argument. Since DVS can be set to ASIO mode, maybe that’s something to explore?

 
Profile Image
EricWest
  -  

@TwentyStudios: That's interesting information! Thanks a lot!
Unfortunately we will have to do everything directly in Axy, since there is no external mixing console... :-/

Here we go with some more very interesting details:

- When closing the current Flow and opening another one, the audio-offset remains the same.
So there's definitely something deep inside Axy holding on to that offset.
Only a full restart of Axy removes the offset

- We've been recording that Metronome for over 18 hours now.
Yesterday evening (after passing that magic 6 hour mark) we had the well-known offset of 25 Frames.
We expected that offset-event to happen every 6 hours and thought we'd had at least 3 seconds offset this morning.
Surprise, surprise! The offset was still exactly 25 Frames!!!
So after hickupping reproducibly after 6 hours and gaining an extra second of audio, that offset remains the same for at least 12+ hours.

Slightly unlogical, innit?.... :-)

 
Profile Image
ericmarodon
  -  

Whatever the issue is, thanks for taking the time to investigate.

Is this also happening when using the embedded sound in the HDMI or SDI cameras for instance? Or only when using Dante?

 
Profile Image
EricWest
  -  

@ericmarodon

Thanks! That would be the next test we will be doing.
Our entire audio workflow is currently based on Dante though.
So that test would just give us a clue where one could look closer to find the culprit... :-)

I will try to get back on that topic next week.

 
Profile Image
EricWest
  -  

Righto....another overnight test... :-)

We completely disabled DANTE, feeding audio into Axy via a simple USB Microphone.

Now guess what?
After 06:12:56 we have the exact same one second audio offset!

The FrameID at failing point was 558681.
(In previous tests FrameIDs were 558649 and 558474, so there seems to be a pattern....)

For now I'd say that we can rule out DANTE as the culprit.
There seems to be something fishy within Axy which might need some attention.....

If I can help with any more information or testing just lemme know.

Cheers.

Eric.


 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

Thank you very much for documenting and investigating this. As always, we appreciate your feedback and bug reports.

We started to investigate this issue. Due to its nature, it will take some time to get to the bottom of it.

Warmest regards,


 
Profile Image
EricWest
  -  

And here we go with another test...:-)

Ran the same thingy on a different machine, in a different room, with a different environment and a different Axy version (Studio DE in this case).


Again, after 6hours 13minutes we had a sudden audio offset.
Most interestingly in this case it was +18 frames (whereas our other test reproducibly yielded -25 frames....)

Also, the 06:13 magical leap seems to be related to the system UpTime, rather than to the session time (the time the compound is running for).
To prove the latter we will run another test today.

If you wanna reproduce the test, we're using this little toy here, set to 113bpm and 5 measures, running fullscreen on an iPad (plus an occasional manual clap)
https://codepen.io/olthomat/full/gOzojMj


 
Profile Image
Eifert@Aximmetry
  -  

Hi Eric,

The Restart module has been added to the latest version of Aximmetry (2023.1.0):
Added Restart Application module to allow restarting Aximmetry via any trigger with running the same or any other compound.
https://aximmetry.com/learn/software-version-history/#%E2%96%BA-latest-2023-1-0 

Note, that we didn't forget about the audio delay issue. And there is a new system under work that will fix it. Until that you can use the Restart Application module as a workaround.

Warmest regards,

 
Profile Image
EricWest
  -  

Hi Eifert.

Great news! Thanks a bunch!
I'm looking forward to the new system and I'm glad that you apparently identified the delay-culprit :-)

Cheerio.

 
Profile Image
Eifert@Aximmetry
  -  

Hi,

We're excited to share that the latest version of Aximmetry, version 2023.2.0, has been released today!
The new version revamped the input and output timing system, providing stronger protection against frame drops and audio issues. It should also fix the increasing audio offset problem.

To learn more about all the changes and improvements included in this version of Aximmetry, please visit the following link: https://aximmetry.com/learn/software-version-history/

Warmest regards,