When I started decorating I bought a cricut, molds, tons of impression mats, hundreds of cutters, that huge Wilton decorating set, a turntable, isomalt, glitters, dusts, I could go on and on. Don't do that! I still don't use half of it and I've only just recently made room to properly store it all.
My fiance got me a subscription to MyCakeDecoratingSchool.com, it's like $30/year. They have free tutorials to teach you the basics of baking to carving a topsy turvy cake, they have a forum, recipes, a q&a, etc. I really found that helpful, as well as Cake Central, Cakes Decor, and I have some favorite blogs. There are tutorials all over the Internet that I use still. Personally, I preferred the online school to the Wilton curriculum. But some people rave about those classes, some people learn better with a teacher in the room, just a personal preference.
My problem was I wanted to do everything all at once, there are so many different mediums, and styles-it can be very overwhelming!
Definitely work with icing first, get some disposable piping bags (I don't have a dishwasher yet
and I hate washing those large bags out), some basic piping tips (which can double as cutters), and everything batter up suggested. If its something you want to take further check out the clay tools as well as the fondant tools, the ones I can get, locally anyway are typically cheaper. I use both. And depending on what you want to do you'll know what to purchase by then.