I'm not so sure. Mammals have a huge energy budget mostly because we're endotherms. Humans are also pretty large as far as mammals go, and use more energy.
The caloric enrichment of food by cooking probably can explain this as well. We got smart enough to cook, which gave us more energy from less food to power the brain. At that point, the evolutionary advantage of metabolic efficiency matters less. It allowed early humans to continue developing on the same path, probably increasing brain complexity somewhat over time while not focusing as much on energy.
Clearly the energetic cost of having large brains was worth it for humans from an evolutionary perspective, or else we wouldn't have them. But 20% of your energy budget is still a very large cost, so if there's a way to accomplish the same brain stuff for, say, half the cost, you would expect that to be very strongly selected for (caloric scarcity being the norm throughout human evolutionary history).
I should say: this would be for pure efficiency optimizations absent any other tradeoffs. What I think you should see in birds based on their more extreme body mass constraints vs ground dwellers is different tradeoffs being worthwhile. Like, if there were an adaptation that required slightly more energy, but resulted in a brain that was much lighter; or doing various mental processes 20% less effectively for 50% less brain mass.
The caloric enrichment of food by cooking probably can explain this as well. We got smart enough to cook, which gave us more energy from less food to power the brain. At that point, the evolutionary advantage of metabolic efficiency matters less. It allowed early humans to continue developing on the same path, probably increasing brain complexity somewhat over time while not focusing as much on energy.
But I'm no biologist. I'm just guessing.