I think that's why it's so handy to start with epsilon delta limits, or even better sequence limits. Once you can screen off "infinitely" to "arbitrarily" and see when and where the substitution works, you're golden. Of course there are other things in math which might rightfully be called "infinity" but they don't come up in high school calculus