That's a bit of a reductionist way of looking at any sort of learning; I'm not sure how it's helpful to use memorize/interpret in terms of distinguishing what language models do compared to other types of learning.

I might be missing the point though?