How do you get fondant pieces to retain their shape on a cake when they are 3d or are leaning on it...or is there another medioum I should be using to make 3d decorations for cakes? I was thinking things like curliques, or twists that stick out of the cake. Things like that.

you need to use gumpaste for those kind of decorations

Mix a bit of gum tragacanth powder into your fondant (about 1 tsp per 500g fondant)