I keep mousse fillings refrigerated at all times when storing. It will be okay for 6-8 hrs. at room temp as long as it isn't super hot, then it could have a melting problem. It also depends on the consistency of your mousse, the firmer the better.
According to ServeSafe guide lines, any product that must be refrigerated, can be kept out of the refrigerator for up to a single 4 hour block of time, after which any left overs must be discarded.
