I live in a major city, have an iPhone 6S with good LTE coverage according to benchmarks, etc. and still routinely have AMP take 15+ seconds to render after the HTML has been received. I don't know if that's Mobile Safari applying strict cache limits or an issue in Google's side but the sales pitch isn't delivering.
Note that I did not say it always happens — when everything is cached, it performs as well as any other mobile-optimized site — or that it's specific to my device/carrier – it also happens on WiFi, Android, etc.
The problem is simply a brittle design which depends on a ton of render-blocking resources. The assumption is that those will be cached but my experience is simply that fairly regularly I'll click on a link, see the page title load (indicating the HTML response has started), and then have to wait a long time for the content to display. Many news sites also load a ton of stuff but since fewer of them block content display waiting for JavaScript, the experience under real-world wireless conditions is better in the worst case and no worse in average conditions.