Add two media sources in topology and display mix video - Windows Media Foundation - ms-media-foundation

Question

I am exploring windows media foundation.
I want to mix and display two video stream in one window.
I am following few of the sample provided by MS.
I am trying to Add multiple media sources to topology, I want to add two media files to topology.
As per the below link, I am following code to add media source in topology:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms701605(v=vs.85).aspx
Below is the code to add source node to the topology:
HRESULT hr = pPD->GetStreamDescriptorByIndex(iStream, &fSelected, &pSD);
if (FAILED(hr))
{
goto done;
}
if (fSelected)
{
// Create the media sink activation object.
hr = CreateMediaSinkActivate(pSD, hVideoWnd, &pSinkActivate);
if (FAILED(hr))
{
goto done;
}
// Add a source node for this stream.
hr = AddSourceNode(pTopology, pSource, pPD, pSD, &pSourceNode);
if (FAILED(hr))
{
goto done;
}
// Create the output node for the renderer.
hr = AddOutputNode(pTopology, pSinkActivate, 0, &pOutputNode);
if (FAILED(hr))
{
goto done;
}
// Connect the source node to the output node.
hr = pSourceNode->ConnectOutput(0, pOutputNode, 0);
}
But I am not able to add multiple media source in my topology.
My single file playback is working properly but I am not able to mix and display two files.

Answer 1

I recommend split your task on five step:
1. write code for playing ONE video file. On MSDN there is example code: How to Play Media Files with Media Foundation.
2. research code of WORKABLE player for point, where created MediaSource from video file path(URL).
3. create TWO MediaSource from two video file paths(URL).
4. by function MFCreateAggregateSource create from TWO MediaSource - ONE MediaSource and retuurn MediaSource from method player HRESULT CreateMediaSource(PCWSTR sURL, IMFMediaSource **ppSource)
5. call 'hr = AddOutputNode(pTopology, pSinkActivate, 0, &pOutputNode);' twice: 'hr = AddOutputNode(pTopology, pSinkActivate, 0, &pOutputNode);' for first video stream and hr = AddOutputNode(pTopology, pSinkActivate, 1, &pOutputNode); for second video stream.
Regards.
P.S. If you will use two video with audio streams, then You will have FOUR streams in Aggregate MediaSource - it could need FIND the stream ID for video stream.
P.S.S It is not easy recommend from viewing only demo code, but in CreateMediaSinkActivate you will find code hr = MFCreateVideoRendererActivate(hVideoWindow, &pActivate);. In your code, you must create this Activate before:
// For each stream, create the topology nodes and add them to the topology.
for (DWORD i = 0; i < cSourceStreams; i++)
{
hr = AddBranchToPartialTopology(pTopology, pSource, pPD, i, hVideoWnd);
if (FAILED(hr))
{
goto done;
}
}
then set this crated Activate as argument for AddBranchToPartialTopology
for example:
hr = MFCreateVideoRendererActivate(hVideoWindow, &pVideoRendererActivate);
// For each stream, create the topology nodes and add them to the topology.
for (DWORD i = 0; i < cSourceStreams; i++)
{
hr = AddBranchToPartialTopology(pTopology, pSource, pPD, i, pVideoRendererActivate);
if (FAILED(hr))
{
goto done;
}
}
In AddBranchToPartialTopologyyou must write something like this:
HRESULT AddBranchToPartialTopology(
IMFTopology *pTopology, // Topology.
IMFMediaSource *pSource, // Media source.
IMFPresentationDescriptor *pPD, // Presentation descriptor.
DWORD iStream, // Stream index.
IMFActivate* aVideoRendererActivate) // VideoRenderer for video playback.
{
IMFStreamDescriptor *pSD = NULL;
IMFActivate *pSinkActivate = NULL;
IMFTopologyNode *pSourceNode = NULL;
IMFTopologyNode *pOutputNode = NULL;
BOOL fSelected = FALSE;
HRESULT hr = pPD->GetStreamDescriptorByIndex(iStream, &fSelected, &pSD);
if (FAILED(hr))
{
goto done;
}
DWORD iStreamID = 0;
if (fSelected)
{
// Create the media sink activation object.
hr = CreateMediaSinkActivate(pSD, iStreamID, aVideoRendererActivate, &pSinkActivate);
In 'CreateMediaSinkActivate' you must write something like this:
DWORD globalVideoIndex = 0;
HRESULT CreateMediaSinkActivate(
IMFStreamDescriptor *pSourceSD, // Pointer to the stream descriptor.
DWORD& iStreamID, // ctream index
IMFActivate *pVideoRendererActivate, // Handle to the video renderer activate.
IMFActivate **ppActivate
)
{
IMFMediaTypeHandler *pHandler = NULL;
IMFActivate *pActivate = NULL;
// Get the media type handler for the stream.
HRESULT hr = pSourceSD->GetMediaTypeHandler(&pHandler);
if (FAILED(hr))
{
goto done;
}
// Get the major media type.
GUID guidMajorType;
hr = pHandler->GetMajorType(&guidMajorType);
if (FAILED(hr))
{
goto done;
}
// Create an IMFActivate object for the renderer, based on the media type.
if (MFMediaType_Audio == guidMajorType)
{
// Create the audio renderer.
hr = MFCreateAudioRendererActivate(&pActivate);
}
else if (MFMediaType_Video == guidMajorType)
{
// Share the video renderer.
hr = pVideoRendererActivate->QueryInterface(IID_PPV_ARG(pActivate))
iStreamID = globalVideoIndex++;
}
else
{
// Unknown stream type.
hr = E_FAIL;
// Optionally, you could deselect this stream instead of failing.
}
if (FAILED(hr))
{
goto done;
}
// Return IMFActivate pointer to caller.
*ppActivate = pActivate;
(*ppActivate)->AddRef();
done:
SafeRelease(&pHandler);
SafeRelease(&pActivate);
return hr;
}
In AddBranchToPartialTopology you must write:
// Create the output node for the renderer.
hr = AddOutputNode(pTopology, pSinkActivate, iStreamID, &pOutputNode);
if (FAILED(hr))
{
goto done;
}
for Audio streams iStreamID will zero, but for video stream it will increment from global variable globalVideoIndex.
Idea is that code create Activate for video renderer BEFORE create Topology - it is OK. Then this ONE video renderer activate share by ref pointer between ALL video streams in MediaSource by checking condition if (MFMediaType_Video == guidMajorType). Each VIDEO stream has got unique id from 0 by incrementing global variable globalVideoIndex++ - this id set in method hr = AddOutputNode(pTopology, pSinkActivate, iStreamID, &pOutputNode);. As a result, all video streams will be drawn by ONE video renderer, and video stream with iStreamID is 0 will reference background, while other video streams will additional.