can you tell me more?what size pot?what type soil.whats the area its setting in?is it on concrete?
Theree's a chance could be spider mites.you'd see trails on the leaves and the leaves may curl& might see ants(they heard about the bugs& came to eat)
This would be early sign of mites http://edis.ifas.ufl.edu/EDISImageP...r=HS122&tag=FIGURE%2022&credits=
also you mention lack of fertilizeing.The oldest leaves would show the 1st signs of nutrient defiencys.Longy is good for this topic.I believe he told me 1st yr fertlize 4 times& the 2nd yr 5 times. A fertilized plant
thats healthy will fight off stuff better than a poorly fertilized one.they make citris spikes also,but i'm good about fertilizeing and my kumquat proably gets more than it needs being its doubled in size at 2 years,but grown inground versis your container.I have a Meiwa. they have a tastier fruit. they do like alot of water.especially on hot days& windy days