So, caramel itself is essentially a cooked sugar that "caramelizes" so the longer you cook it the harder it will get. If you make your own caramel, you would want to heat it to the hard ball or soft-crack stage. This would give you a caramel that will harden on your cake ball. It may still be a bit sticky since that is the nature of candy.
Your caramel recipe will matter. One that has much cream in it will not harden since that is what the cream is preventing. I would definitely have to play around with the ratios though if I tried one with cream. I might try heating to a soft crack and adding a small amount of cream for that drizzle creaminess you seem to be looking for. You can use just plain caramel (cooked white sugar only) too but that may be a harder drizzle than you want.
Another option you could try is flavoring vanilla candy melts with an oil based caramel flavor. This might be the easiest if the flavor turns out like you want it to.
If you are not looking specifically for a caramel flavor, you could also use melted butterscotch chips as a drizzle, which has a somewhat similar type of flavor.
Does this help at all?