Both my wife and I have handicap placards. Neither of us has visible disabilities. I am 210% disabled, considered by the VA to be "100% Permanent and Total." My wife has doctor certifications due to hip and knee problems. When I am driving, I will drop my wife off at the front door, then I'll find a place to park, usually on the backside of the parking lot. I seldom, if ever,use the spots reserved for those with disabilities. As long as I can physically do so, I will continue to park elsewhere. The most apparent misuse here is when a perfectly healthy individual uses grandma's permit to park in a reserved spot.
As to the elderly who no longer drive, that has no bearing on using the reserved spots. If one has an elderly disabled relative, you may need to park in a reserved spot for them.
GM