What is wrong with my mangosteen?

It has scale in the past but I put spray on and picked them all off however these yellow spots and leaf drop still seem to continue even though I can’t spot any more scale? Any advice on how I can save it?