May be a silly question but as I've never stacked a cake, I'm unsure if I still need to use dowels if I'm stacking a two tier cake covered in ganache.

If I do, do I put the dowels in before I ganache the first tier or put them in after the ganache has set?

Sorry if this sounds so stupid :-)  

Yes you should always use support when doing a tiered cake-

Look up the SPS system the plates and columns are inexpensive  and your cake will look awesome not to mention it will be well supported.

If you don't have time to order the SPS?  I use jumbo straws for support and always after I ganache the cakes.

 Hope this helps.