Mandarin tree help

Hi all Located in Adelaide. This mandarin was always a good producer, but dropped off in last few years. We did place this gravel and plant agapanthus so I don’t know if the cause is just these or something else. Water does pool above the gravel when watered. Unsure if something else or whether we need to remove the gravel/agapanthus. Thanks for the help