My partner goes quiet and I fill the silence. We are both making it worse

My partner goes quiet and I fill the silence. We are both making it worse

When something is wrong he needs about twenty minutes alone. I read the silence as punishment and follow him around the flat asking what is wrong, which turns twenty minutes into two hours. We named it recently: he says "twenty minutes" out loud, and I do something with my hands until then. Not fixed, but no longer a spiral. If you two have opposite processing speeds, how do you handle the gap?