Maybe - you could try talking to your bank/lender to see their thoughts on this. I can't really comment on the bakery side of things but My husband up until recently owned & ran a restaurant together my husband is a chef with a lot of years experience and I have a business degree (but at the time had no hospitality experience but did have customer service experience) in our dealings with lenders the first thing they asked is have you had a business before, the second is what is your experience in the field the 3rd was why do you want this business & 4th is what deposit/collateral do you have.

I don't mean to discourage you but loving to cook is very different from running a restaurant, the hours are long, the work is hard, there is a lot of pressure to get the meals out fast, dealing with customers, staff issues, paperwork, govt rules & regulations etc so you will need be prepared for that. My biggest piece of advice is to do some work in a bistro/restaurant and see how you like it, whether you think that it is something you could do & make a good living from. We did well but a lot of other nearby restaurants struggled financially, you need to know your market and manage your expenses.

Like anything there are both rewards & challenges. Its great when people tell you how much they love the food, when sales are growing and we liked the control your own business gives you. Good luck hope all goes well whatever you decide.