We explore the in-the-wild face video quality assessment problem for the first time. Concretely, we present FVQ-20K, the first large-scale in-the-wild face video quality assessment dataset, which contains 20,000 face videos with (a) diverse source video content, (b) various distortions in both spatial and temporal domains, (c) a variety of facial attributes, and (d) high-quality MOS annotation for each video. Along with the FVQ-20K dataset, we propose FVQ-Rater, the first LMM-based method elaborately designed for the face video quality assessment task as illustrated in (e).