As long as the inverter keeps working, you are right; you still get some (reduced) benefit for just letting them run.
At some point the inverter (a separate box, usually inside the house) will fail and need to be replaced. At that point you'd have to balance the price of a new inverter vs. the expected return of energy from the panels.
Its not like they are an old car or something where the decrease in efficiency uses more energy / pumps out more emmisions.
The only resource they are using is roof space.