SharpDX integration

hoahong · 10 Dec 2020, 09:45

Hello,

I want to integrate Noesis GUI with SharpDX in my project ( it is not a game project, it is a media project that can play video and sound). I run a sample project you built with SharpDX but it is tool simple to understand. It displays only one button that covers whole screen.
Can you create other project with more feature so that i can understand that It is possible to integrate to SharpDX?

Thanks
hoahong

10 Dec 2020, 10:24

Our SharpDX integration sample shows how to render a Noesis UI over a render target created using SharpDX.
If you want to render your own 3D scene it should be done after render target is set and cleared, and just before calling view.Renderer.Render().
Please, could you tell us more about what you need in your integration?

10 Dec 2020, 10:52

Although it is for C++, our Integration Tutorial explains everything you need to know for a proper integration.

hoahong · 10 Dec 2020, 11:42

Our SharpDX integration sample shows how to render a Noesis UI over a render target created using SharpDX.
If you want to render your own 3D scene it should be done after render target is set and cleared, and just before calling view.Renderer.Render().
Please, could you tell us more about what you need in your integration?

Thanks!

I want to play media file (mp4...) or (display image from camera) and display GUI on top. I use this class to play with media

public class MediaPlayer : Singleton<MediaPlayer>
    {
       
        private MediaPlayer(/*No params when using Singleton*/)
        {
            BackgroundColor = Color.Transparent;
            isVideoStopped = true;
        }
//         ~MediaPlayer()
//         {
//             Shutdown();
//         }

        //Add comments
        /// <summary>
        /// Gets whether this media player is playing a video or audio.
        /// </summary>
        public bool IsPlaying { get; private set; }

        /// <summary>
        /// Gets or sets the background color used to display the video.
        /// </summary>
        public Color BackgroundColor { get; set; }

        /// <summary>
        /// Gets or sets the url used to play the stream.
        /// </summary>
        public string Url { get; set; }

        /// <summary>
        /// Output Video texture (must be <see cref="SharpDX.DXGI.Format.B8G8R8A8_UNorm"/>)
        /// </summary>
        public SharpDX.Direct3D11.Texture2D OutputVideoTexture;


        public ByteStream MediaStream;
        
        //https://csharp.hotexamples.com/examples/SharpDX.MediaFoundation/MediaEngineEx/-/php-mediaengineex-class-examples.html
        public virtual void Init(SharpDX.Direct3D11.Device5 _device)
        {
           //SwapChain4 _swapChain = new SwapChain4(new IntPtr());
           // var texture = Texture2D.FromSwapChain<Texture2D>(_swapChain, 0);
           // var surface = texture.QueryInterface<SharpDX.DXGI.Surface>();

            lock (lockObject)
            {
                // Startup MediaManager
                MediaManager.Startup();

                // Create a DXGI Device Manager
                dxgiDeviceManager = new DXGIDeviceManager();
                dxgiDeviceManager.ResetDevice(_device);

                // Setup Media Engine attributes
                var attributes = new MediaEngineAttributes
                {
                    DxgiManager = dxgiDeviceManager,
                    VideoOutputFormat = (int)SharpDX.DXGI.Format.B8G8R8A8_UNorm
                };

                using (var factory = new MediaEngineClassFactory())
                    mediaEngine = new MediaEngine(factory, attributes, MediaEngineCreateFlags.None, OnMediaEngineEvent);
                mediaEngineEx = mediaEngine.QueryInterface<MediaEngineEx>();
            }
        }

        public virtual void OnRender(SharpDX.Direct3D11.Texture2D _backBuffer)
        {


            if (_backBuffer == null)
                return;
                //throw new ArgumentNullException(nameof(_backBuffer));


            lock (lockObject)
            {
                if (isVideoStopped)
                    return;

                if (mediaEngineEx == null)
                    return;
                if (!mediaEngineEx.OnVideoStreamTick(out _))
                    return;

                /*var backBuffer = OutputVideoTexture ?? _backBuffer;*/

                
                var desc = _backBuffer.Description;
                var region = new Rectangle(0, 0, desc.Width, desc.Height);

                mediaEngineEx?.TransferVideoFrame(_backBuffer, null, region, (ColorBGRA)BackgroundColor);
            }
        }

        public void Shutdown()
        {
            lock (lockObject)
            {
                StopVideo();

                if (mediaEngineEx != null)
                    mediaEngineEx.Shutdown();
            }
        }

        public void SetBytestream(string filename)
        {
           
            mediaEngineEx.SetSourceFromByteStream(MediaStream, Url);
            
        }

        /// <summary>
        /// Plays the audio/video.
        /// </summary>
        public void Play()
        {
            if (mediaEngineEx != null)
            {
                if (mediaEngineEx.HasVideo() && isVideoStopped)
                    isVideoStopped = false;

                if (isEndOfStream)
                {
                    PlaybackPosition = 0;
                    IsPlaying = true;
                }
                else
                {
                    mediaEngineEx.Play();
                }

                isEndOfStream = false;
            }
        }

        /// <summary>
        /// Pauses the audio/video.
        /// </summary>
        public void Pause()
        {
            if (mediaEngineEx != null)
                mediaEngineEx.Pause();
        }

        /// <summary>
        /// Gets or sets the volume.
        /// </summary>
        public double Volume
        {
            get
            {
                if (mediaEngineEx != null)
                    return mediaEngineEx.Volume;
                return 0.0;
            }
            set
            {
                if (mediaEngineEx != null)
                    mediaEngineEx.Volume = value;
            }
        }

        /// <summary>
        /// Gets or sets the balance.
        /// </summary>
        public double Balance
        {
            get
            {
                if (mediaEngineEx != null)
                    return mediaEngineEx.Balance;
                return 0.0;
            }
            set
            {
                if (mediaEngineEx != null)
                    mediaEngineEx.Balance = value;
            }
        }

        /// <summary>
        /// Gets or sets muted mode.
        /// </summary>
        public bool Mute
        {
            get
            {
                if (mediaEngineEx != null)
                    return mediaEngineEx.Muted;
                return false;
            }
            set
            {
                if (mediaEngineEx != null)
                    mediaEngineEx.Muted = value;
            }
        }

        /// <summary>
        /// Steps forward or backward one frame.
        /// </summary>
        public void FrameStep(bool forward)
        {
            if (mediaEngineEx != null)
                mediaEngineEx.FrameStep(forward);
        }

        /// <summary>
        /// Gets the duration of the audio/video.
        /// </summary>
        public double Duration
        {
            get
            {
                double duration = 0.0;
                if (mediaEngineEx != null)
                {
                    duration = mediaEngineEx.Duration;
                    if (double.IsNaN(duration))
                        duration = 0.0;
                }
                return duration;
            }
        }

        /// <summary>
        /// Gets a boolean indicating whether the audio/video is seekable.
        /// </summary>
        public bool CanSeek
        {
            get
            {
                if (mediaEngineEx != null)
                    return (mediaEngineEx.ResourceCharacteristics & ResourceCharacteristics.CanSeek) != 0 && Duration != 0.0;
                return false;
            }
        }

        /// <summary>
        /// Gets or sets the playback position.
        /// </summary>
        public double PlaybackPosition
        {
            get
            {
                if (mediaEngineEx != null)
                    return mediaEngineEx.CurrentTime;
                return 0.0;
            }
            set
            {
                if (mediaEngineEx != null)
                    mediaEngineEx.CurrentTime = value;
            }
        }

        /// <summary>
        /// Gets a boolean indicating whether the audio/video is seeking.
        /// </summary>
        public bool IsSeeking
        {
            get
            {
                if (mediaEngineEx != null)
                    return mediaEngineEx.IsSeeking;
                return false;
            }
        }

        /// <summary>
        /// Enables video effect.
        /// </summary>
        /// <param name="enable"></param>
        public void EnableVideoEffect(bool enable)
        {
            if (mediaEngineEx != null)
            {
                mediaEngineEx.RemoveAllEffects();
                if (enable)
                {
                    //mediaEngineEx.InsertVideoEffect(new Activate(Windows.Media.VideoEffects.VideoStabilization), false);
                }
            }
        }

        private void StopVideo()
        {
            isVideoStopped = true;
            IsPlaying = false;


            Func<int, int> f = (x) => x + 2;
        }

        protected virtual void OnMediaEngineEvent(MediaEngineEvent mediaEvent, long param1, int param2)
        {
            switch (mediaEvent)
            {
                case MediaEngineEvent.NotifyStableState:
                    SetEvent(new IntPtr(param1));
                    break;
                case MediaEngineEvent.LoadedMetadata:
                    isEndOfStream = false;
                    break;
                case MediaEngineEvent.CanPlay:
                    // Start the Playback
                    Play();
                    break;
                case MediaEngineEvent.Play:
                    IsPlaying = true;
                    break;
                case MediaEngineEvent.Pause:
                    IsPlaying = false;
                    break;
                case MediaEngineEvent.Ended:
                    if (mediaEngineEx.HasVideo())
                    {
                        Play();
                    }
                    isEndOfStream = true;
                    break;
                case MediaEngineEvent.TimeUpdate:
                    break;
                case MediaEngineEvent.Error:
                    break;
            }
        }

        [DllImport("kernel32.dll", EntryPoint = "SetEvent")]
        private static extern bool SetEvent(IntPtr hEvent);
    }

It is ok to render Video file but i cannot display GUI with only a Button like in your sample:

Initialize GUI:

//Global
Noesis.View view;
 RenderTargetView renderView;
 //In init function
Noesis.GUI.Init("hoahong", "8/nbTD+x20w9HcqUj0gD5kEXVQSI8vKVm8FGXIJ/Bz1tMAW6");
Noesis.Grid xaml = (Noesis.Grid)Noesis.GUI.ParseXaml(@"
                <Grid xmlns=""http://schemas.microsoft.com/winfx/2006/xaml/presentation"">
                    <Grid.Background>
                        <LinearGradientBrush StartPoint=""0,0"" EndPoint=""0,1"">
                            <GradientStop Offset=""0"" Color=""#FF123F61""/>
                            <GradientStop Offset=""0.6"" Color=""#FF0E4B79""/>
                            <GradientStop Offset=""0.7"" Color=""#FF106097""/>
                        </LinearGradientBrush>
                    </Grid.Background>
                    <Viewbox>
                        <StackPanel Margin=""50"">
                            <Button Content=""Hello World!"" Margin=""0,30,0,0""/>
                            <Rectangle Height=""5"" Margin=""-10,20,-10,0"">
                                <Rectangle.Fill>
                                    <RadialGradientBrush>
                                        <GradientStop Offset=""0"" Color=""#40000000""/>
                                        <GradientStop Offset=""1"" Color=""#00000000""/>
                                    </RadialGradientBrush>
                                </Rectangle.Fill>
                            </Rectangle>
                        </StackPanel>
                    </Viewbox>
                </Grid>");
// View creation to render and interact with the user interface
/ We transfer the ownership to a global pointer instead of a Ptr<> because there is no way
// in GLUT to do shutdown and we don't want the Ptr<> to be released at global time
view = Noesis.GUI.CreateView(xaml);
view.SetFlags(Noesis.RenderFlags.PPAA | Noesis.RenderFlags.LCD);
view.SetSize(mScreen.Width, mScreen.Height);
view.Renderer.Init(new Noesis.RenderDeviceD3D11(mGraphics.D3DDeviceContext.NativePointer));
renderView = new RenderTargetView(mGraphics.D3DDevice, mScreen.BackBuffer2D);

//In update function
MediaPlayer.Instance?.OnRender(mScreen.BackBuffer2D);
view.Update((DateTime.Now - start).TotalSeconds);

 // Offscreen rendering phase populates textures needed by the on-screen rendering
view.Renderer.UpdateRenderTree();
view.Renderer.RenderOffscreen();
mGraphics.D3DDeviceContext.Rasterizer.SetViewport(new Viewport(0, 0, mScreen.Width, mScreen.Height, 0.0f, 1.0f));
mGraphics.D3DDeviceContext.OutputMerger.SetTargets(renderView);
mGraphics.D3DDeviceContext.ClearRenderTargetView(renderView, SharpDX.Color.Black);

//Render other my resources

///Update UI
view.Renderer.Render();

//
mScreen.Present(1);

10 Dec 2020, 20:57

So the problem you are having is how to create more complex xamls and how to load them into your application?
This is related to resource providers: https://www.noesisengine.com/docs/Gui.C ... -providers
You can use our base classes to allow loading resources embedded in the assembly (EmbeddedXamlProvider), from file system (LocalXamlProvider), or create your own providers.

hoahong · 11 Dec 2020, 11:32

So the problem you are having is how to create more complex xamls and how to load them into your application?
This is related to resource providers: https://www.noesisengine.com/docs/Gui.C ... -providers
You can use our base classes to allow loading resources embedded in the assembly (EmbeddedXamlProvider), from file system (LocalXamlProvider), or create your own providers.

Can you please create a sample for it? (SharpDX integration).
Thanks

11 Dec 2020, 12:49

Please add the following MainPage.xaml to your project and set 'Build Action' to 'Embedded Resource' in its properties window.

<Grid
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  Background="#43464C">
  <DockPanel LastChildFill="True">
    <Menu DockPanel.Dock="Top">
      <MenuItem Header="File"/>
      <MenuItem Header="About"/>
    </Menu>
    <StatusBar DockPanel.Dock="Bottom">
      <StatusBarItem Content="Ready"/>
    </StatusBar>
    <StackPanel HorizontalAlignment="Center" VerticalAlignment="Center">
      <TextBlock Text="Hello World" Foreground="White"/>
      <Button Content="Button" Margin="0,10"/>
      <ComboBox SelectedIndex="0">
        <ComboBoxItem Content="First"/>
        <ComboBoxItem Content="Second"/>
        <ComboBoxItem Content="Third"/>
      </ComboBox>
      <ListBox Margin="0,10" Background="#53565C">
        <ListBoxItem Content="List item 0"/>
        <ListBoxItem Content="List item 1"/>
        <ListBoxItem Content="List item 2"/>
        <ListBoxItem Content="List item 3"/>
      </ListBox>
    </StackPanel>
  </DockPanel>
</Grid>

Then in the integration code, before setting the providers create a XamlProvider that can load embedded resources from the application:

// Create xaml provider for embedded resources
Type appType = typeof(Program);
Noesis.XamlProvider xamlProvider = new NoesisApp.EmbeddedXamlProvider(appType.Assembly, appType.Namespace);

// Setup theme
NoesisApp.Application.SetThemeProviders(xamlProvider);
Noesis.GUI.LoadApplicationResources("Theme/NoesisTheme.DarkBlue.xaml");

So you can load the embedded xaml:

// Load main xaml
Noesis.Grid xaml = (Noesis.Grid)Noesis.GUI.LoadXaml("MainPage.xaml");

// View creation to render and interact with the user interface
Noesis.View view = Noesis.GUI.CreateView(xaml);

The rest of integration code will be the same.

I hope this helps.

hoahong · 19 Dec 2020, 05:16

Please add the following MainPage.xaml to your project and set 'Build Action' to 'Embedded Resource' in its properties window.

<Grid
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  Background="#43464C">
  <DockPanel LastChildFill="True">
    <Menu DockPanel.Dock="Top">
      <MenuItem Header="File"/>
      <MenuItem Header="About"/>
    </Menu>
    <StatusBar DockPanel.Dock="Bottom">
      <StatusBarItem Content="Ready"/>
    </StatusBar>
    <StackPanel HorizontalAlignment="Center" VerticalAlignment="Center">
      <TextBlock Text="Hello World" Foreground="White"/>
      <Button Content="Button" Margin="0,10"/>
      <ComboBox SelectedIndex="0">
        <ComboBoxItem Content="First"/>
        <ComboBoxItem Content="Second"/>
        <ComboBoxItem Content="Third"/>
      </ComboBox>
      <ListBox Margin="0,10" Background="#53565C">
        <ListBoxItem Content="List item 0"/>
        <ListBoxItem Content="List item 1"/>
        <ListBoxItem Content="List item 2"/>
        <ListBoxItem Content="List item 3"/>
      </ListBox>
    </StackPanel>
  </DockPanel>
</Grid>

Then in the integration code, before setting the providers create a XamlProvider that can load embedded resources from the application:

// Create xaml provider for embedded resources
Type appType = typeof(Program);
Noesis.XamlProvider xamlProvider = new NoesisApp.EmbeddedXamlProvider(appType.Assembly, appType.Namespace);

// Setup theme
NoesisApp.Application.SetThemeProviders(xamlProvider);
Noesis.GUI.LoadApplicationResources("Theme/NoesisTheme.DarkBlue.xaml");

So you can load the embedded xaml:

// Load main xaml
Noesis.Grid xaml = (Noesis.Grid)Noesis.GUI.LoadXaml("MainPage.xaml");

// View creation to render and interact with the user interface
Noesis.View view = Noesis.GUI.CreateView(xaml);

The rest of integration code will be the same.

I hope this helps.

Thans Sfernandez,

Yes, it displays GUI but the GUI covers whole screen on top, i can not see the content of video. that is because of "view.SetSize(mScreen.Width, mScreen.Height);" ?
How can i make background transparent of GUI and how can i set actual size for GUI? (Gui size is smaller screen but actually it is called to cover whole screen)

21 Dec 2020, 10:16

The root Grid has Background property set to a color, so it is filling the entire window with that color. If you just remove that property you will be able to see through the UI.
The call to view.SetSize() is to specify the size of the GUI root element (this should be set to the size of the window client region), but you can place UI controls wherever you want depending on the containers you choose. I recommend you start reading our tutorials to get general concepts on how Noesis and XAML work before building your app.

SharpDX integration

SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Re: SharpDX integration

Who is online