Agreed, and you can use the size/number of panels to emphasize the timing of certain moments:
Many panels = slower pace
Fewer panels = faster pace
Large panel = big moment
Small panel = small moment
(YMMV, feel free to break any of those “rules”—they’re shorthand, not standards)
I always found that dichotomy interesting. You'll want to use a big panel for a BIG moment, but by its nature a big single panel goes by quickly unless it's loaded with so much information that it might as well be many panels. You can undercut your big moment by letting the reader pass it too fast.
I was talking with Chaykin about this once (forgive the comics name drop) and at some point the clouds parted in my mind and I said “so in comics… space EQUALS time.” I think everyone making comics has at least a vague sense of this, but it’s helped me to be conscious of it.