Magic in fiction is generally postulated to break the laws of physics. That’s fine. You could imagine the laws would be different. There might be some issues with even that that a physicist could point to, but I see no obvious problem with postulating such changes. However, with many forms of magic, there’s another problem that can’t be solved even by changing the laws of nature.