Firming up the batter is not going to make it taste any better. But you can try adding 1/4 cup flour to your recipe and see if that helps. Keep the cupcakes in their liners, of course.
You need to use a different shape of holder. Use wood skewers and bakers cardboard cut into circles to fit the bottoms. Slide a cardboard along each skewer until the cupcake can sit on that and have some skewer inside it to hold it. Use plenty of icing glue under the cupcake.
The other stickum is to add chocolate chips or white chips to the cake batter. Stick the skewers in while the cupcakes are at room temperature, then chill. The chips will solidify enough to be edible internal "glue" for the skewers.