Whether you believe in organized religion or not, your characters need some kind of inner compass, some code they live (and die) by, some set of principles they adhere to, or even a few superstitions they follow.
This has nothing to do with preaching or forcing a particular dogma onto readers. It has everything to do with making your characters multi-dimensional and layered. I mean, come on! Give the poor, beleaguered protagonist something to hope for, something to cling to, some kind of faith for comfort. And that faith should fit the story world and the character, regardless of your personal beliefs.
So mull over what belief system your fantasy characters will have. If they have organized religion, how is it set up? Is there one faith, or several? Do these organizations clash or coexist peacefully? Is there a priesthood or an order acting as a filter between the characters and their god(s)? Do these folks worship a single god or a whole pantheon of deities? Do they attend temples, churches, chapels, open-air altars, or other public rituals?
Or perhaps your characters are respectful toward a mysterious, more mystical force that manifests through nature — i.e. a grove of sacred trees, or an awe-inspiring waterfall, or a magical flame spewing from the top of a mountain. You may choose to create elemental spirits that are either benign toward your characters or quite dangerous and malevolent.
The choices are plentiful and can be as simple or elaborate as your storyline requires. And how will your characters’ beliefs be challenged as the story progresses?
It’s been said by someone somewhere that as soon as you stand on a principle, you’re going to be asked to die for it, either literally or symbolically. The hard choices start coming rapidly when someone takes a stance: slapping an embargo on a country means living without the fine brandy or cigars that nation produces; believing in free speech means listening to opinions you disagree with; supporting your king in an insurrection means maybe losing your property or your head if he’s overthrown.
The religious aspect of a story world may be intrinsic and central to the plot or merely part of the backdrop, but don’t overlook it if you want to add a more interesting completeness to your setting.