I’ve just posted some quick performance tests here:
I don’t know if they really cover the issue that you are seeing, since you say it only shows up after a few minutes. But might still give you a general feel for how it “should” perform. I didn’t notice any such pauses when doing my tests.