Trusted computing isn't so much about control as it is about trust/attestation. You can run privileged malware on your computer without trusted computing. On the other hand, you can also sandbox trusted computing.
If you don't want to run somebody else's code on your computer – don't! (Chances are you're not a cloud provider and nobody is asking you to, anyway.) If you oppose DRM – don't consume content protected by it, whether the DRM is implemented in SGX, TrustZone or just in obfuscated software.
Don't get me wrong, I am not the biggest fan of DRM either (I personally see it as an evil, maybe a necessary one though). But shunning the entire field of trusted computing is a bit like opposing theoretical physics because it ultimately has brought us nuclear weapons.
I'm not entirely against the concept of trusted computing, but it's hard to believe that "trusted computing" as we know it today is so innocuous when it's included on consumer CPUs. Why not keep it as a premium feature for server-class CPUs instead?
Remote attestation can be incredibly useful and mutually beneficial on client devices as well if you're willing to consider applications beyond DRM.
For example, every smartcard is essentially a device in your physical possession, running somebody else's code. I'd argue that for example in the case of EMV payment cards, the benefit is mutual (less fraud).
I'm comfortable with it on SmartCards or EMV chips, since those are tailored devices for a specific purpose and the trust model is understood by the participants. I'm not particularly upset that I can't root my credit card.
Mobile phones, it's disappointing, but total consumer control was kind of a lost cause from day 1.
What truly bothers me is the introduction of trusted computing into general-purpose consumer CPUs, where previously we had complete freedom. There's an old but still very good CCC talk that encapsulates my feelings about this [0].
Just to better understand your line of reasoning: Do you generally equate trusted computing with elevated privileges for the attestable code being executed?
I think this is where a lot of trust (on a meta-level) in the technology has been lost, and as far as I understand it, modern implementations are, on the contrary, tightly sandboxed.
In that sense, modern trusted computing can actually be more freedom preserving: If you are generally willing to tolerate DRM on your system, as long as it's not able to access data on your computer that are none of its business, you're more likely to see that happen on a modern, hardware-assisted DRM platform than on the rootkit based software shenanigans of the early 2000s.
Elevated privileges for a hardware blob is an even worse situation that unfortunately exists (Intel ME/AMD PSP).
For me the issue is the blob itself. "Trusted" for the manufacturer is "trust us" for the end-user. My expectation for a general-purpose CPU is that I can inspect the code that's running on it. Building a TPM into consumer CPUs defies that freedom.
So, it's about hackability (in the Hacker News sense). If you can't see what's happening inside of your computer, is it really yours?
If you don't want to run somebody else's code on your computer – don't! (Chances are you're not a cloud provider and nobody is asking you to, anyway.) If you oppose DRM – don't consume content protected by it, whether the DRM is implemented in SGX, TrustZone or just in obfuscated software.
Don't get me wrong, I am not the biggest fan of DRM either (I personally see it as an evil, maybe a necessary one though). But shunning the entire field of trusted computing is a bit like opposing theoretical physics because it ultimately has brought us nuclear weapons.