In dynamics and control of space flexible manipulators, high computing speed is always required for solving inverse dynamics. However, when the flexible multibody system suffers from high-frequency excitation such as contact impact, high order modes have to be counted to ensure adequate precisions while the computing speed is lowered down. To increase the computational efficiency of such problems, an adaptive mode method was proposed with an application in inverse dynamic modeling of a rotating flexible manipulator. In the process, the axial displacement caused by transverse deformation of the beam-like model was considered, and high order mode coordinates were reduced gradually as they damped off in the proportion of total energy due to structural damping. The control laws of driving torque for the rotary manipulator articulation were computed to account for its validity. Additionally, the influence of gravity was investigated.