You have to register the button for clicks before it can receive click events, so with your code, your OnClick handler never gets called. Also, you need to actually set the OnClick handler to your OnClick function.
Code:
button:RegisterForClicks("LeftButtonUp", "RightButtonUp")
button:SetScript("OnClick", function(self, mouseButton)
if mouseButton == "RightButton" then
DestroyTotem(1)
DestroyTotem(2)
DestroyTotem(3)
DestroyTotem(4)
end
end)
However, you might want to selectively destroy only the totem whose button was actually clicked on, though without seeing your actual code I couldn't tell you how to do that, since I don't know how your buttons are set up.