How to loose hard soil without backbreaking work?

I’ve got a patch of dirt in my backyard that has been covered with weedmat for a few years that I want to use to grow some flowers in, but my first problem is that it is rock hard. I don’t think it is clay, just it’s painful work getting a fork or pickaxe in there to break it all up. Is there anything else I can do to treat the soil so I’m able to get the fork in more than half a centimeter?