It is a drug much like... say, broccoli is a drug. Milk before bedtime is a drug. Running is a drug. It is a drug in the same sense that anything can cause addiction and withdrawal or be unhealthy.

Caffeine is, by itself, a drug in a different sense. It causes a slight addiction and can cause slight withdrawal. It isn't all that unhealthy in itself, however, especially not in the amounts we find naturally in foods. It becomes more of an issue if one is taking caffeine pills and other such things. Eating unhealthily might be more unhealthy, honestly - even though overall, having an unhealthy diet isn't a drug.

"It is a drug in the same sense that anything can cause addiction and withdrawal or be unhealthy.

Caffeine is, by itself, a drug in a different sense. It causes a slight addiction and can cause slight withdrawal."

This seems like a drug in the same sense, can you clarify?

Rather than making stuff up, here's dictionary definition of drug: a medicine or other substance which has a physiological effect when ingested or otherwise introduced into the body.

As a user of both sugar and broccoli I can anecdotally attest that they do not have the same physiological effect on my body.

Nah, that can't be true. Suppose you eat either broccoli or a sugary snack every day at 4 PM. Then I bet that habit is definitely harder to stop if it's a sugary snack.