Using the self-consistent linear field theory, Cherenkov radiation excitated by the beam-wave interaction of a thin annular relativistic electron beam in a plasma-filled, dielectric coaxial cylindrical waveguide was analyzed. The dispersion equation of the interaction, the synchronized condition and the wave growth rate were derived. The energy exchange between the wave and the electron beam in the presence of background plasma was discussed, and the effects of plasma density on the dispersion characteristics, the wave growth rate and the beam-wave energy exchange were calculated and discussed. It was clear that the Cherenkov radiation is result of the coupling between the slow TM mode propagated along the waveguide and the negative-energy space-charge mode propagated along the beam, and t