I live in a small town. There are basically four main bakers in this area. I am the only one with an actual shop that people can come and pick up their cakes. The other people are making cakes in their homes. And in my state that is illegal. Ever since I opened my shop I guess I have been taking a lot of their business. So now they have all started charging extremely low prices that I cannot compete with. And customers message me if and tell me that they are getting a cake much cheaper off of these other individuals. My question is should I just respond to them by saying nothing and have a nice day? Or should I let them know that my costs are higher because I have a shop that I have to pay licenses, permits, and health inspections. I don't want to sound petty but it is frustrating because what they are doing is illegal and I'm just trying to do everything the way it's supposed to be done.

I would politely reply that you understand completely and thank them for considering you for their special occasion; and then report those unlicensed  bakers to the health department or whichever body govern's that in your area. Its not a nice or pleasant thing to do but its gotten out of hand the number of unlicensed cake "businesses" recently..at least in my area. You cannot compete with someone with literally no overhead. 

That is exactly how I feel. I feel like they are trying to undercut me so much that I will just close up and they keep going about their business.

The sad thing is that people have gone out of business trying to compete with someone who isn't "playing by the rules so to speak". In my area it is a complete no-no to do cakes from your home kitchen; but it doesn't stop anyone, they get shut down one week and are back up and running the next week under a different name. Fines don't seem to be a deterrent ; nor does the fact that they are loosing money by selling a two tier cake with edible images for $50. So if you need to go ahead and tip off the health department I don't see an issue with it.