SetScale causing frames to "jump"

[Kasheen]

Has anyone else encountered a little annoyance whereby when you use SetScale on a frame which is anchored to UIParent the frame "jumps" up or down the screen? Setting the scale higher will cause it to jump down and sometimes to the right as well, and setting it lower will cause it to jump up and sometimes to the left. Frames which are anchored to other frames and then scaled stay anchored at the correct point.

So does anyone know of a way to use SetScale without the frame "jumping" about the screen?