Use a razor blade to get the bulk of it off. Invest in some magic erasers (generic is just as good as the name brand MrClean). That will likely get it all off, but if there are tiny bits left, you can soak it in a bit of rubbing alcohol, giving it time to dissolve the glue, and then you can try scrubbing.
That's always worked for me.