How to publish environment specific appsettings in .Net core app?

I have 3 environment specific appsettings files in my .Net core application 

in project.json I have setup publishOptions like this. ( based on suggestion here)

"publishOptions": {"include": ["wwwroot",      "appsettings.development.json","appsettings.staging.json","appsettings.production.json","web.config"
    ]
  },

I have 3 corresponding startup classes that uses appropriate appsetingsbased on environment

  var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: false, reloadOnChange: true);

However when I publish the application then all 3 appsettings files endup in all the environments. How do I publish environment specific appsetting file?