View Revisions: Issue #1700

Summary 0001700: Crash / memory stomp in SDF generator
Revision 2020-05-27 01:53 by jsantos
Description Hi guys,

I'm seeing an assert which is ultimately leading to a memory overwrite in the SDF Generator. We are running at 4k UI and we have a text popup we plays an animation and scales larger than the screen. The text is also drawing to the offscreen buffer, I'm not sure if that makes a difference.

The full callstack is as follows:

> Noesis::SDFGenerator::Begin(int width, int height) Line 1103 C++
     Noesis::GlyphCache::RenderSDF(Noesis::GlyphCache::Glyph * glyph, Noesis::FT_FaceRec_ * face, unsigned int index, unsigned int size, long stroke) Line 397 C++
     Noesis::GlyphCache::GetGlyph(Noesis::VGLFontFace * face, unsigned int index, unsigned int size, long stroke, bool inLayer) Line 527 C++
     Noesis::GlyphCache::CacheGlyph(Noesis::VGLFontFace * face, unsigned int index, unsigned int size, long stroke) Line 147 C++
     Noesis::VGLContext::DrawRunForeground(Noesis::VGLTextLayout * layout, const Noesis::Transform2 & mtx, Noesis::Glyph * glyphs, const Noesis::GlyphRun & run, const Noesis::TextProperties & p, Noesis::VGLPaint * paint, const Noesis::BatchGroup & k, float scale, Noesis::GeoBuilderType builder, unsigned short runIndex, unsigned char flags) Line 3028 C++
     Noesis::VGLContext::DrawTextLayout(Noesis::VGLTextLayout * layout, const Noesis::Transform2 & mtx, Noesis::VGLPaint * * paints, unsigned int numPaints, unsigned int flags_) Line 1228 C++
     Noesis::RenderTreeHelper::DrawTextLayout(Noesis::VGLTextLayout * textLayout, Noesis::BrushProxy * * brushes, unsigned int numBrushes, unsigned int flags, const Noesis::Rect & bounds, const Noesis::Transform2 & transform) Line 470 C++
     Noesis::RenderTreeHelper::DrawCommands(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, Noesis::DrawingCommandsProxy * drawCommands, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, unsigned int bitmapScalingMode) Line 274 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderDraw(Noesis::RenderNode *) Line 743 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderChildren(Noesis::RenderNode * isProjectionIdentity, const Noesis::Recti &) Line 760 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderChildren(Noesis::RenderNode * isProjectionIdentity, const Noesis::Recti &) Line 760 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     Noesis::RenderTreeHelper::RenderOffscreenNodes() Line 592 C++
     Noesis::RenderTreeHelper::RenderOffscreen(Noesis::RenderNode * node, const Noesis::Matrix4 & projection) Line 97 C++
     Noesis::RenderTree::RenderOffscreen(const Noesis::Matrix4 & projection) Line 280 C++
     Noesis::Renderer::RenderOffscreen(const Noesis::Matrix4 & projection) Line 164 C++
     Noesis::Renderer::RenderOffscreen() Line 148 C++

void SDFGenerator::Begin(int width, int height)
{
    NS_ASSERT(width * height <= (int)sizeof(mSDF)); // <--- this hits
    NS_ASSERT(width * height * (int)sizeof(DFData) <= (int)sizeof(mData));


In my case, width is 79 and height is 66. If I go back up the stack, the cbox.xMax is 3840 (the width of my render target) and the cbox.xmin is -704. So width ends up being (3840 + 704) >> 6 which is 79, and the height ends up as (2816 + 896) >> 6 which is 66.

I tried to reproduce this locally in XamlPlayer but was unable to. I'm attaching something very similar to what we have in game, but it doesn't assert or crash in XamlPlayer.
Revision 2020-05-27 00:45 by steveh
Description Hi guys,

I'm seeing an assert which is ultimately leading to a memory overwrite in the SDF Generator. We are running at 4k UI and we have a text popup we plays an animation and scales larger than the screen. The text is also drawing to the offscreen buffer, I'm not sure if that makes a difference.

The full callstack is as follows:

> Noesis::SDFGenerator::Begin(int width, int height) Line 1103 C++
     Noesis::GlyphCache::RenderSDF(Noesis::GlyphCache::Glyph * glyph, Noesis::FT_FaceRec_ * face, unsigned int index, unsigned int size, long stroke) Line 397 C++
     Noesis::GlyphCache::GetGlyph(Noesis::VGLFontFace * face, unsigned int index, unsigned int size, long stroke, bool inLayer) Line 527 C++
     Noesis::GlyphCache::CacheGlyph(Noesis::VGLFontFace * face, unsigned int index, unsigned int size, long stroke) Line 147 C++
     Noesis::VGLContext::DrawRunForeground(Noesis::VGLTextLayout * layout, const Noesis::Transform2 & mtx, Noesis::Glyph * glyphs, const Noesis::GlyphRun & run, const Noesis::TextProperties & p, Noesis::VGLPaint * paint, const Noesis::BatchGroup & k, float scale, Noesis::GeoBuilderType builder, unsigned short runIndex, unsigned char flags) Line 3028 C++
     Noesis::VGLContext::DrawTextLayout(Noesis::VGLTextLayout * layout, const Noesis::Transform2 & mtx, Noesis::VGLPaint * * paints, unsigned int numPaints, unsigned int flags_) Line 1228 C++
     Noesis::RenderTreeHelper::DrawTextLayout(Noesis::VGLTextLayout * textLayout, Noesis::BrushProxy * * brushes, unsigned int numBrushes, unsigned int flags, const Noesis::Rect & bounds, const Noesis::Transform2 & transform) Line 470 C++
     Noesis::RenderTreeHelper::DrawCommands(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, Noesis::DrawingCommandsProxy * drawCommands, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, unsigned int bitmapScalingMode) Line 274 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderDraw(Noesis::RenderNode *) Line 743 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderChildren(Noesis::RenderNode * isProjectionIdentity, const Noesis::Recti &) Line 760 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     [Inline Frame] Noesis::RenderTreeHelper::RenderChildren(Noesis::RenderNode * isProjectionIdentity, const Noesis::Recti &) Line 760 C++
     Noesis::RenderTreeHelper::RenderNodeRecursive(Noesis::RenderNode * node, const Noesis::Recti & surfaceRect, const Noesis::Transform2 & transform, const Noesis::Matrix4 & projection, bool isProjectionIdentity, float opacity, unsigned int depth) Line 703 C++
     Noesis::RenderTreeHelper::RenderOffscreenNodes() Line 592 C++
     Noesis::RenderTreeHelper::RenderOffscreen(Noesis::RenderNode * node, const Noesis::Matrix4 & projection) Line 97 C++
     Noesis::RenderTree::RenderOffscreen(const Noesis::Matrix4 & projection) Line 280 C++
     Noesis::Renderer::RenderOffscreen(const Noesis::Matrix4 & projection) Line 164 C++
     Noesis::Renderer::RenderOffscreen() Line 148 C++

void SDFGenerator::Begin(int width, int height)
{
    NS_ASSERT(width * height <= (int)sizeof(mSDF)); // <--- this hits
    NS_ASSERT(width * height * (int)sizeof(DFData) <= (int)sizeof(mData));


In my case, width is 79 and height is 66. If I go back up the stack, the cbox.xMax is 3840 (the width of my render target) and the cbox.xmin is -704. So width ends up being (3840 + 704) >> 6 which is 79, and the height ends up as (2816 + 896) >> 6 which is 66.

I tried to reproduce this locally in XamlPlayer but was unable to. I'm attaching something very similar to what we have in game, but it doesn't assert or crash in XamlPlayer.