System.Exception: Shadow Vortice.Direct2D1.ID2D1Transform is dead

[Odepax]

I'm trying to use a pixel shader effect with Vortice's Direct2D. The only thing I got so far is an exception I have troubles to understand.

What I do

Draw some text in an offscreen ID2D1BitmapRenderTarget:

// OffscreenBuffer.Resource is ID2D1BitmapRenderTarget
// MenuTextFormat           is IDWriteTextFormat3
// BlueColor                is ID2D1SolidColorBrush
OffscreenBuffer.Resource.BeginDraw();
OffscreenBuffer.Resource.Clear(Colors.Transparent);
OffscreenBuffer.Resource.DrawText("Resume", MenuTextFormat, new RectangleF(8, 8, 512, 15),  BlueColor);
OffscreenBuffer.Resource.EndDraw();

Apply the shader and draw the result on screen:

// g                             is ID2D1DeviceContext6
// TextHightlightShader.Resource is TextHightlightShader, which extends ID2D1Effect
g.DrawImage(
   image: TextHightlightShader.Resource.Output,
   compositeMode: CompositeMode.SourceOver,
   interpolationMode: InterpolationMode.NearestNeighbor,
   targetOffset: new Vector2(-8, -8)
);

The shader effect is initialized as such:

// TextHightlightShader.Resource is TextHightlightShader, which extends ID2D1Effect
// g                             is ID2D1DeviceContext6
// OffscreenBuffer.Resource      is ID2D1BitmapRenderTarget
TextHightlightShader.Resource = new TextHightlightShader(g);
TextHightlightShader.Resource.SetInput(0, OffscreenBuffer.Resource.Bitmap, invalidate: true);

What I get

The application runs fine for a few moments (top ~ 1 min), then the call to g.EndDraw() — which happens after g.DrawImage() — throws. It throws without returning an error code, i.e. it doesn't even get to the point of calling .CheckError() on the Result that was returned by g.EndDraw(). Nothing indicates that the device was reset or lost.

System.Exception: Shadow Vortice.Direct2D1.ID2D1Transform is dead
   at SharpGen.Runtime.CppObjectShadow.ToCallback[T](IntPtr thisPtr)
   at Vortice.Direct2D1.ID2D1TransformVtbl.MapInputRectsToOutputRectImpl_(IntPtr thisObject, Void* _inputRects, Void* _inputOpaqueSubRects, Int32 _inputRectCount, Void* _outputRect, Void* _outputOpaqueSubRect)
   at Vortice.Direct2D1.ID2D1RenderTarget.EndDraw()
   at Uy.UyWindow.InvokeDirectXRender(Int2 windowPixelSize) in .../Uy/Windows/UyWindow.cs:line 520
   at Uy.UyWindow.RunRenderPass(Single secondsSinceFirstTick, Single secondsSinceLastTick) in .../Uy/Windows/UyWindow.cs:line 184
   at Uy.UyApplication.RunGameLoop(CancellationToken stoppingToken) in .../Uy/Application/UyApplication.cs:line 219
   at Uy.UyApplication.<>c__DisplayClass22_0.<ExecuteAsync>b__0() in .../Uy/Application/UyApplication.cs:line 100
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)

Using the shader happens inside an if branch. The application runs fine forever if the conditional doesn't take the "shader" path, then might crash immediately upon entering the path.

It seems TextHightlightShader.Implementation.DisposeCore(false) is called by the Garbage Collector. To prevent GC from happening, I should keep a reference to the Implementation object, but I don't instantiate it, so I don't see any way for me to do so...

The shader effect code

class TextHightlightShader : ID2D1Effect {
   public static void Register(ID2D1Factory7 d2Factory) => d2Factory.RegisterEffect<Implementation>();

   public TextHightlightShader(ID2D1DeviceContext context) : base(context.CreateEffect(typeof(Implementation).GUID)) {}
   public TextHightlightShader(ID2D1EffectContext context) : base(context.CreateEffect(typeof(Implementation).GUID)) {}

   [CustomEffect(1)]
   class Implementation : CustomEffectBase, ID2D1EffectImpl {
      ShaderTransform? ShaderT;

      protected override void DisposeCore(bool disposing) {
         if (disposing) {
            ShaderT?.Dispose();
            ShaderT = null;
         }

         base.DisposeCore(disposing);
      }

      public override void Initialize(ID2D1EffectContext effectContext, ID2D1TransformGraph transformGraph) {
         // https://learn.microsoft.com/en-us/windows/win32/direct2d/custom-effects#initializeid2d1effectcontext-pcontextinternal-id2d1transformgraph-ptransformgraph
         //
         // > Direct2D calls the Initialize method after the ID2D1DeviceContext::CreateEffect method has been called by the app.
         // > You can use this method to perform internal initialization or any other operations needed for the effect.
         // > Additionally, you can use it to create the effect's initial transform graph.
         base.Initialize(effectContext, transformGraph);

         ShaderT = new ShaderTransform(effectContext);

         transformGraph.AddNode(ShaderT);
         transformGraph.SetSingleTransformNode(ShaderT);
      }

      public override void SetGraph(ID2D1TransformGraph transformGraph) {
         // https://learn.microsoft.com/en-us/windows/win32/direct2d/custom-effects#setgraphid2d1transformgraph-ptransformgraph
         //
         // > Direct2D calls the SetGraph method when the number of inputs to the effect is changed.
         // > While most effects have a constant number of inputs, others like the Composite effect support a variable number of inputs.
         // > This method allows these effects to update their transform graph in response to a changing input count.
         base.SetGraph(transformGraph);
      }

      public override void PrepareForRender(ChangeType changeType) {
         // https://learn.microsoft.com/en-us/windows/win32/direct2d/custom-effects#prepareforrender-d2d1_change_type-changetype
         //
         // > The PrepareForRender method provides an opportunity for effects to perform any operations in response to external changes.
         // > Direct2D calls this method just before it renders an effect if at least one of these is true:
         // >
         // > - The effect has been previously initialized but not yet drawn.
         // > - An effect property has changed since the last draw call.
         // > - The state of the calling Direct2D context(like DPI) has changed since the last draw call.
         base.PrepareForRender(changeType);
      }
   }

   class ShaderTransform : CallbackBase, ID2D1DrawTransform {
      public ShaderTransform(ID2D1EffectContext effectContext) {
         var compilation = Compiler.Compile(
            shaderSource: typeof(Program).Assembly.GetEmbeddedResourceString("EmbeddedResources/TextHightlight.hlsl", EncodingD.Utf8NoBom),
            defines: Array.Empty<ShaderMacro>(),
            include: null!,
            entryPoint: "main",
            sourceName: "TextHightlight.hlsl",
            profile: "ps_5_0",
            shaderFlags: ShaderFlags.Debug,
            effectFlags: EffectFlags.None,
            out var blob,
            out var errorBlob
         );

         using (blob)
         using (errorBlob) {
            compilation.CheckError();

            var bytes = blob.AsBytes();
            effectContext.LoadPixelShader(typeof(ShaderTransform).GUID, bytes, bytes.Length);
         }
      }

      public int GetInputCount() => 1;

      public void MapInputRectsToOutputRect(RawRect[] inputRects, RawRect[] inputOpaqueSubRects, out RawRect outputRect, out RawRect outputOpaqueSubRect) {
         outputRect = inputRects[0];
         outputOpaqueSubRect = inputOpaqueSubRects[0];
      }

      public void MapOutputRectToInputRects(RawRect outputRect, RawRect[] inputRects) {
         inputRects[0] = outputRect;
      }

      public RawRect MapInvalidRect(int inputIndex, RawRect invalidInputRect) {
         return invalidInputRect;
      }

      public void SetDrawInfo(ID2D1DrawInfo drawInfo) {
         drawInfo.SetPixelShader(typeof(ShaderTransform).GUID, PixelOptions.None);
      }
   }
}

The HLSL shader itself doesn't do much yet:

float4 main() : SV_TARGET {
   return float4(1.0f, 1.0f, 1.0f, 1.0f);
}

[amerkoleci]

There where other people having this issue, if you setup a repo example I can try to fix it.

[Odepax]

You're referring to #315, right? I tried the solution suggested there by the author, i.e. keeping a static list of instances; to all appearances, it solves the garbage collection problem, but the solution feels... Unwholesome somehow...

I managed to build an case similar to my situation by modifying the Advanced Text Rendering sample, but was unable to reproduce the bug. This leads me to think that the snapping point lies in my own framework's code. I must investigate further, I'll be back!

[amerkoleci]

Ok, once you have reprocase we can investigate

[Odepax]

The reason I couldn't reproduce the bug was because garbage collection didn't trigger in the sample. This must be due to the sample being a smaller code base than my app.

Thanks to GC.Collect(), I was able to reproduce the bug, both for Vortice 2.4.2 / .NET 6 — I'm stuck in the past! — and Vortice 3.2.0 / .NET 7.

Interestingly:

• The Vortice 2.4.2 / .NET 6 repro throws System.Exception: Shadow Vortice.Direct2D1.ID2D1EffectImpl is dead, which is a slightly different exception than the one I saw with my own framework (...ID2D1Transform is dead); by messing around, I found out it's due to how I create my device context: from DXGI, as opposed to .CreateHwndRenderTarget() in the sample.

• The Vortice 3.2.0 / .NET 7 repro throws System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

• In all 3 cases, the dirty fix from Direct 2D Custom Effects bug. #315 works.

[amerkoleci]

@andrew-boyarshin can you point me to which changes I need to make in SharpGenTools to restore the previous behaviour here?