In order to explore the influence of the modulation circuit fault of the fiber optical current transformer (FOCT) on the output signal of the detector,and facilitate fault prediction and fault diagnosis,the mathematical model of sine wave modulation FOCT output signal which includes the phase delay between the optical circuit and the electrical circuit is firstly established in this paper. The mathematical expressions of modulation depth,driving voltage, second and fourth harmonic are given,and the basic methods of closed-loop modulation and demodulation of the target value modulator are analyzed. On this basis,a mathematical model is established to analyze the influence of modulation depth on the peak of the detector output light intensity,average light intensity,second and fourth harmonic. Therefore,it is proposed that the modulation depth decrease is the key factor leading to the abnormal output signal of the detector when the modulation loop fault occurs,and the increase of the average light intensity of the detector output,the decrease of the fourth harmonic and the increase of the second/fourth harmonic ratio should be the typical characteristics of the modulation loop fault. Finally,the practical FOCT simulation test is carried out to prove the validity of the theory.