Do mattress toppers make that much of a difference.

I bought a memory foam topper that felt great when I first laid down but by the end of the night I was tossing and turning. It would get soft where I was laying but stay firm right next to me so it felt lumpy if you moved at all, I then bought a Natural Latex 2" topper and that has been absolutely great. It is about 3 times heavier than the memory foam and it does not move. I put it under a thin quilted mattress cover.