What shitty ISP operates like that? I frequently see IPv6 have *lower* latency because of fewer middle boxes and fewer hops in general. Your routers in the default free zone shouldn't be close to their TCAM limits with a single IPv4 routing table.