Temples in Tainan are a dime a dozen. They are literally everywhere. If there's not a temple, there's at least some sort of alter or statue of a god. Take for example, the route I used to take to work. On my 5 minute drive, I began by passing one major temple Then, at the train tracks, there's a small, caged alter. At the intersection with the 7-11 (which are more common than temples), there was another caged god-- a traffic God.
As a side note, if you've never been to Taiwan then you don't know the magic of 7-11. In Taiwan 7-11 is a one stop shop. You can buy (mediumly) delicious food there, pay all of your bills, receive and send packages, buy more cell phone minutes, get souvenirs, call a taxi, buy a fresh coffee or latte, get any essential hygiene item, order all sorts of tickets (HSR, baseball, etc...) on the Ibon machine, buy booze, cigarettes, condoms... all of the essentials. And, you can sit outside and drink your beers at the table. Also, they are EVERYwhere. As in, when you stand at one 7-11, you can see another 7-11 about a block away. Taiwanese people love their convenience.
Back to the temples. In Taiwan, it doesn't matter what you believe or what your religion, you can go into the temple and have a look around. Most temples don't close and you can wander in at any time of night. You can do temple things, like get your fortune told or light an incense and bai bai (pray), or just look around and take some pictures. Most temples allow them, but if photography isn't allowed there's usually a sign posted.
In Tainan, one of the things I used to like to do was drive around randomly with a friend, taking the small winding streets that make up this city (impossible to navigate) and check out whatever temples we happened to pass.
During ghost month (or any month really), you'll hear fireworks at random hours during the day and night and if you live in the center of town, like I did, your commute will often be delayed by large parades. If you luck out, you'll see a stripper pole-dancing on a van or truck with blaring, English pop-music playing in the background; it's a strange way to worship, but hey, whatever works. If you're not so lucky, you might get a string of fireworks going off right next to your scooter as your drive innocently down the road (yes, I know from experience).
You can also choose to visit some of the more famous temples in/near Tainan. Here are some of my recommendations.
The Confucius Temple
The Confucius Temple is probably the most famous temple in Tainan. It's in the heart of the city, so there are lots of traditional foods nearby. It's also close to the Great South Gate (the city has four random gates). There's a tradition that each year they sacrifice an animal (usually a cow) to Confucius. It's very early in the morning and is pretty hard to get to.
The TuCheng Mazu Temple 正統鹿耳門聖母廟
Most temples are pretty old. This one is not. It's a big temple to the Sea Mother and it's out in no man's land. It's set in a beautiful backdrop and is really a pretty neat place.
Another Tainan landmark, Chikan tower is located in the center of town as well. Next to the temple is a traditional eating area where you can try coffin toast, oily rice, oyster omelets and intestine soup. All sound weird, but are actually delicious. Across the street, there is also the most famous Winter Melon tea place in Tainan, and another temple that's pretty fun to explore.
Chikan tower is a temple for students and supposedly if you touch the statue of the god's brush, you will be successful in your studies. It's also a building dedicated to the man who kicked the Dutch out of Tainan.
the Kaiyuan Temple/ Monastary
This temple is close to my heart. It was about a 5 minute bike ride from my grandmother's house and I used to visit it quite often. It's not flashy or huge like a lot of other temples, but it is old.
You enter through a courtyard with a few sleepy Banyan trees and can wander freely around the grounds. A lot of monks live there, but they won't likely say much to you.
When I was a kid, one of the monks gave me a starfruit just for being so darn cute. I love this temple for the peace it offers in a busy city. And because they have a happy Buddha statue.
Madou Temple is a temple that doesn't show up on any itinerary when you visit Taiwan, but.... it's amazing. Why? Because Madou has an animatronic Hell. That's right, like small world, Disney style Hell with robot people ripping each other up and cooking each other. For a small donation, you can walk through the very (un)realistic Disney Hell. And, after you're done, you can walk through heaven. Hell takes about 20 or 30 minutes to check out and heaven is 5.
Every Sunday around noon, they have a procession of masochists that beat themselves bloody. These guys are supposed to be possessed and flex their godliness by hitting themselves with axes and swords. You can see more about it on my personal blog.
When the lake has water, it's awesome. A few years ago, they emptied if for some reason (mosquitoes maybe?) and it looked pretty depressing. But, when I was a little younger I went and saw the lake in all its glory. I recommend this place for the sheer novelty of walking through the dragon building on the lake. They say if you walk through the dragon, it's lucky. In other words, Dragon poop is lucky and by pretending you're a dragon bowel movement, you can be lucky, too. Nearby is a huge Confucian temple called the Zuoying Confucius Temple.
Fo Guang Shan
Fo Guang Shan is a huge place. To get there, you can go to the Zuoying station and take the bus towards E-da world (it used to be big and purple). You travel past E-da and the bus will drop you off at Fo Guang Shan. It takes about 40 minutes by bus.
If you are staying in Taiwan a bit longer, or have the timing right, I would recommend staying a weekend and being a monk. They do it the first week of each month. You live a monastic life for a couple of days and learn all about monk culture. It's really a unique experience, and is especially awesome if you don't know much about the religion.
The compound itself is huuuuuge. There are loads of different temples to explore and if you get the timing right, you can see one of the Buddha's teeth.
You can also take part in a traditional tea ceremony or get your fortune told.
There are a lot of other temples I'm sure that I've missed, but these are some of the most famous or my favorites to visit. If you have one that you like in or near Tainan, add it in the comments! :)