This work studies a challenging and practical problem, termed multi-class unsupervised anomaly detection (MUAD), which only requires normal images for training while simultaneously testing both normal and anomaly images for multiple classes. Existing reconstruction-based methods typically adopt pyramidal networks as encoders and decoders to obtain multi-resolution features, accompanied by elaborating sub-modules with heavier handcraft engineering designs. In contrast, a plain Vision Transformer (ViT) showcasing a more straightforward architecture has proven effective in multiple domains, including detection and segmentation tasks. It is simpler, more effective, and elegant. Following this spirit, we explore plain ViT features for MUAD. We first abstract a Meta-AD concept by inducing current reconstruction-based methods. Then, we instantiate a novel ViT-based ViTAD structure, effectively designed step by step from global and local perspectives. In addition, this paper reveals several interesting findings for further exploration. Finally, we benchmark various approaches comprehensively and fairly on eight metrics. Based on a naive training recipe with only an MSE loss, ViTAD achieves state-of-the-art results and efficiency on MVTec-AD, VisA, and Uni-Medical datasets, obtaining 85.4 mAD that surpasses UniAD by +3.0, while only requiring 1.1 hours and 2.3G GPU memory to complete model training by one V100 on the MVTec-AD dataset.