Traditional masonry fireplaces are terribly inefficient. When they're not being used, they leak warm air through the chimney. When they are being used, they're still inefficient, because they take the air they've just heated and send most of it up the chimney.
So, what can you do to make your fireplace more energy-efficient, both when it's being used and when it's not? First, keep the damper shut when the fireplace isn't in use. If you haven't had your chimney inspected in a while, you should, not only for safety reasons but because your damper could have become warped over the years, thus losing its seal. If you never use your fireplace, you can even stop up the chimney with an old pillow or blanket stuffed above the firebox (just leave a string dangling, so you don't forget it's there!). Since a burning fire, draws the heated air around it, you lose energy even when you think the fireplace is heating the home. To combat warm air loss, open the nearest window about an inch, close the doors to the room, and lower the thermostat to 50-55 degrees.
Also, get a good set of glass doors and install them in front of the firebox. These stop heat loss when the fire is burning down but the damper is too hot to close. The best glass doors have insulation behind the frame and close tightly.