Configuring new document in MFC

Here's what I did* based on Brian's links
First create a dialog resource with the properties:

• border FALSE
• 3D look FALSE
• client edge FALSE
• Popup style
• static edge FALSE
• Transparent TRUE
• Title bar FALSE

and you should end up with a dialog window with no frame or anything, just a grey box. override the Create function to look like this:

BOOL LightBoxDlg::Create(UINT nIDTemplate, CWnd* pParentWnd)
{

    if(!CDialog::Create(nIDTemplate, pParentWnd))
        return false;
    RECT rect;
    RECT size;

    GetParent()->GetWindowRect(&rect);
    size.top = 0;
    size.left = 0;
    size.right = rect.right - rect.left;
    size.bottom = rect.bottom - rect.top;
    SetWindowPos(m_pParentWnd,rect.left,rect.top,size.right,size.bottom,NULL);

    HWND hWnd=m_hWnd;  
    SetWindowLong (hWnd , GWL_EXSTYLE ,GetWindowLong (hWnd , GWL_EXSTYLE ) | WS_EX_LAYERED ) ;
    typedef DWORD (WINAPI *PSLWA)(HWND, DWORD, BYTE, DWORD);
    PSLWA pSetLayeredWindowAttributes;
    HMODULE hDLL = LoadLibrary (_T("user32"));
    pSetLayeredWindowAttributes = 
        (PSLWA) GetProcAddress(hDLL,"SetLayeredWindowAttributes");
    if (pSetLayeredWindowAttributes != NULL) 
    {
        /*
        * Second parameter RGB(255,255,255) sets the colorkey 
        * to white LWA_COLORKEY flag indicates that color key 
        * is valid LWA_ALPHA indicates that ALphablend parameter 
        * is valid - here 100 is used
        */
        pSetLayeredWindowAttributes (hWnd, 
            RGB(255,255,255), 100, LWA_COLORKEY|LWA_ALPHA);
    }


    return true;
}

then create a small black bitmap in an image editor (say 48x48) and import it as a bitmap resource (in this example IDB_BITMAP1)
override the WM_ERASEBKGND message with:

BOOL LightBoxDlg::OnEraseBkgnd(CDC* pDC)
{

    BOOL bRet = CDialog::OnEraseBkgnd(pDC);

    RECT rect;
    RECT size;
    m_pParentWnd->GetWindowRect(&rect);
    size.top = 0;
    size.left = 0;
    size.right = rect.right - rect.left;
    size.bottom = rect.bottom - rect.top;

    CBitmap cbmp;
    cbmp.LoadBitmapW(IDB_BITMAP1);
    BITMAP bmp;
    cbmp.GetBitmap(&bmp);
    CDC memDc;
    memDc.CreateCompatibleDC(pDC);
    memDc.SelectObject(&cbmp);
    pDC->StretchBlt(0,0,size.right,size.bottom,&memDc,0,0,bmp.bmWidth,bmp.bmHeight,SRCCOPY);

    return bRet;
}

Instantiate it in the DoModal of the desired dialog, Create it like a Modal Dialog i.e. on the stack(or heap if desired), call it's Create manually, show it then create your actual modal dialog over the top of it:

INT_PTR CAboutDlg::DoModal()
{
    LightBoxDlg Dlg(m_pParentWnd);//make sure to pass in the parent of the new dialog
    Dlg.Create(LightBoxDlg::IDD);
    Dlg.ShowWindow(SW_SHOW);

    BOOL ret = CDialog::DoModal();

    Dlg.ShowWindow(SW_HIDE);
    return ret;
}

and this results in something exactly like my mock up above

*there are still places for improvment, like doing it without making a dialog box to begin with and some other general tidyups.

When opening a CDialog, the trick is to use a WindowsInteropHelper to get the parent WPF dialog's HWND. Then, you can use CWnd::Attach to wrap that HWND in a CWnd class to pass to the CDialog's constructor.

The problem I had was that I already had the CDialog constructed., but not yet displayed. The various versions of SetParent can only be used if your target child window already has a valid handle. I had to write a new function in my CDialog class to set m_wndParent, which is what it uses as the parent when it finally creates the dialog. Then everything works great!

Somehow creating WPF dialogs from MFC dialogs "just works". It's magic.