The action of squeezing the pop rivet just pushes the wood paneling out of the way. It needs a backing washer to work. Try to find a stud, or go to the edges of the panel. Be wary of hidden wiring or plumbing. Also be careful of sticky back products. I've seen towel hooks etc. tear the top laminate (usually a thin photo paper or plastic) and leave a really ugly scar.
Norm