Gardenias can be very finicky, especially in pots. They require consistent moisture, but not soggy. Sounds like you may be watering enough, so it could be a nutrient deficiency. Try fertilizing it with an acidic fertilizer like those designed for azaleas. Iron deficiency can also cause yellow leaves in acid loving plants. You might try a product called 'Ironite', a liquid iron suppliment for plants. I've used it on my azaleas before.