sorry for all the bullets/structure - i was thinking as I typed ;)
TLDR: Switching threads in the middleware pipeline can cause premature teardown of RequestServices
issue bootstrap
- ASP.Net 5/Web Application project created from template in VS2015 community (not update 1, yet)
- compiling against odata alpha rc1 source code (ancillary).... dependency: "Microsoft.AspNet.OData": "6.0.0-alpha1-rc1-final" - last source download approx 12-26
- compiling against my own libraries
Startup.Configure Excerpt:
app.UseNexusAPM();
app.UseIISPlatformHandler(options => options.AuthenticationDescriptions.Clear());
app.UseStaticFiles();
app.UseMvc(routes => { ..... });
app.UseOData<IBillingData>("api");app.UseNexusAPM() starts a new thread that the rest of the pipeline will run under. I've reordered and placed that line in all places, except not below UseMVC or UseOData, due to its use cases - same behavior:
- Kestrel runs fine delivering the default home page (My IISExpress is being weird tonight so no details there, but it wasn't so smooth with and without my UseNexusAPM, but was fine with only OData in the past) The default home page is the scaffolded HomeController.Index, which indicates to me that MVC is sane with its threading & cooperation with ASP.Net. But now we introduce 2 competing 3rd parties. OData(or any other i believe) & me.
- When OData tries to service a simple request (/api/Customers)
- *** it works without my UseNexusAPM - which starts a new thread, but has no other asp/mvc interaction, other than a quick IServiceProvider query ***
- *** with my call, it throws an exception in OData "Extensions/HttpContextExtensions.UrlHelper" (line 30) due to httpContext.RequestServices being null ***
- Stepping through, RequestServices is not null, early in the pipeline, even after my UseNexusAPM, but is in fact null by the time it gets there
- The call stack is started by a Task.Run (dodging the current TaskScheduler - yuck) within OData, but is not async/awaited, it just returns a Task, so this is an abandoned thread initiated by OData expected to run to completion OOB, to write the response. Changed their source to async/await to be sure a completion was involved - no difference.
This is a more generic issue than OData, and stubbing UseNexusAPM with a ThreadPool.QueueUserWorkItem(next) should be functionally equivalent.
Resolutions (from my consumer PoV):
- Tell us not to mess with the threads (I have a plan B but would prefer this 'automatic' method)
- Don't be quite so aggressive in tearing down RequestServices so that they live long enough to be used by this case of people spawning off threads that will outlive the middleware pipeline.
- The weird part is that it all works until we escape the original request threads... Are you using some form of TLS? Or more likely is some sort of race condition. Or a combination.
- On that note, my thread has a SynchronizationContext, but that may not be sufficient for items later in the pipe expecting the original thread
- I find this hard to believe because the next() function delegates are all async so this must all be anticipated so my vote is bug.
***EDIT*** Never mind i found it... in my efforts to reproduce this error using the thread pool, i discovered a scenario that allowed a Task to slip through my library without being awaited upon. Tightened down that bolt and it works! So ultimately, this was in fact caused by tasks still running on other threads but control had left the middleware space and the framework thought we were done and started tearing down the request while the response was still being written. A certain amount of forgiveness during teardown may be prudent, or leave it alone forcing the dev to use tasks correctly.