Magento 2 – How to keep already built cache

Greetings,

I found out that on first visit on category page, Magento 2 cache takes it’s time on the first run, which is expected. After the first run it works great from different locations and devices. As is expected.

Then if we just change 1 small thing on one product in that category page, the entire page goes slow and cache again needs to be rebuilt.

How can I say to Magento that it should leave all .html pages alone (after first rebuild) until we decide it is time to clear and flush cache?