Thousands of horse racing fans are filling the stands at Colonial Downs for the much-anticipated Virginia Derby on Saturday.