Evolution by natural selection requires heritable variation for fitness-associated traits and increases in the frequency of advantageous, trait-associated genetic variants over time. Case studies that illustrate both properties remain rare in natural populations, especially for polygenic traits. Here, we combined three decades of field observations with genome resequencing data for 3,124 meerkats to show that adult body mass is heritable and strongly predicts lifetime reproductive success. While rare, large effect alleles can be revealed by inbreeding, body mass is highly polygenic. By conditioning on the multi-generation population pedigree, we show that the frequencies of high mass-associated alleles tend to increase over time, consistent with positive selection for larger size. However, realized adult mass has decreased over the same period, likely due to co-occurring environmental deterioration. Our results demonstrate how genomic analyses in long-term field studies can capture polygenic responses to selection in nature, even when they are incompletely mirrored by changes in the traits themselves.