.NET Core Jwt Token always expired when trying to access Authorized area

This is my Startup.cs

services.AddAuthentication(x =>{
    x.DefaultAuthenticateScheme="bearer";
    x.DefaultChallengeScheme="bearer";}).AddJwtBearer("bearer",x =>{
    x.RequireHttpsMetadata=false;
    x.SaveToken=true;//x.TokenValidationParameters = tokenValidationParameters;
    x.TokenValidationParameters=newTokenValidationParameters{ValidateIssuerSigningKey=true,IssuerSigningKey=newSymmetricSecurityKey(Encoding.UTF8.GetBytes("SecretKey")),ValidateIssuer=true,ValidateAudience=true,ValidateLifetime=true,ValidIssuer=Environment.GetEnvironmentVariable(MS_Jwt_Issuer),ValidAudience=Environment.GetEnvironmentVariable(MS_Jwt_Issuer),ClockSkew=TimeSpan.Zero,};
    x.Events=newJwtBearerEvents{OnAuthenticationFailed= context =>{if(context.Exception.GetType()==typeof(SecurityTokenExpiredException)){
                context.Response.Headers.Add("Token-Expired","true");}returnTask.CompletedTask;}};});


services.AddResponseCaching();
services.AddCors(c =>{
    c.AddPolicy("AllowOrigin", options => options.AllowAnyOrigin());});

This is how I set the token:

var key =newSymmetricSecurityKey(Encoding.UTF8.GetBytes("SecretKey"));var credentials =newSigningCredentials(key,SecurityAlgorithms.HmacSha256);var tokenDescriptor =newSecurityTokenDescriptor{Subject=newClaimsIdentity(claims),Expires=DateTime.UtcNow.AddDays(10),SigningCredentials= credentials,IssuedAt=DateTime.UtcNow,Issuer=Environment.GetEnvironmentVariable(MS_Jwt_Issuer),Audience=Environment.GetEnvironmentVariable(MS_Jwt_Issuer),};var token = tokenHandler.CreateToken(tokenDescriptor);var refreshToken = tokens.GenerateRefreshToken();var processedToken = tokenHandler.WriteToken(token);

Whenever I authenticate, I am able to get a token out properly. However, when I try to access a class protected by [Authorize], I get this exception:

Exception thrown: 'Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException' in Microsoft.IdentityModel.Tokens.dll

Here is a sample of my token. It verifies successfully with my secret key and the expiry date shown is proper and not expired:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJQaG9uZU51bWJlciI6IjEyMzIxMjMxMjMiLCJuYmYiOjE1ODcwNTAxOTksImV4cCI6MTU4NzkxNDE5OSwiaWF0IjoxNTg3MDUwMTk5LCJpc3MiOiJTUiIsImF1ZCI6IlNSIn0.WbEJq_PAOLvra1ZUwtQEKH9FRBDdb2byw26miUm-k-E

EDIT:

When I try to manually validate the token, it verifies successfully and it is not Expired. But for some reason, the [Authorize] labels it as expired