C++ – Open & write to console from a C++ DLL

c++consoledllvisual-c++

As a quick note before starting my question, I am quite new to C++ and especially DLLs.

I am currently coding a C++ DLL in VS2012, for use with a program. Note that I do not have source access to the specific program.
I'd like to write some output to a seperate console window from the DLL when it gets asked by the program in question to do something.

My question is, how would I go about doing something like this? To clarify exactly what I want, I'd like a seperate console window to open as soon as the DLL gets attached to the program, and have it then write output to the console. When the DLL gets detached from the program it should also close the console window.

My guesses so far have been to make a seperate Win32 console application project in my solution, and then possibly reference it in my DLL, and have it execute certain methods from it, with the output contained in the arguments of the call. I have no idea how to go about this though, especially since I would want this seperate console project to be included in the same DLL.

Thanks for any input.

Best Solution

The following class will give you an idea of what you will need to do. On construction, a CConsole attempts to use AllocConsole to create a new console for the process.

If AllocConsole fails, nothing is changed -- a console already exists and the class assumes that C-Runtime handles have already been set up somewhere else.

If AllocConsole succeeds, the objects currently associated with stdout and stdin are saved, then these are replaced with ones created for the new console. Now C-Runtime output functions (like printf) will output to the new console. cout and cin will also use the new console.

The RemoveMenu call will prevent a user from closing the console window, terminating the process. It is not necessary beyond this function.

When a CConsole is destroyed (and AllocConsole was successful), stdout and stdin are restored, then the console is closed with a call to FreeConsole.

I find this class to be inconvenient when you want your console to persist beyond the function which created it -- you need to get a new CConsole and keep track of its pointer until you close it with a delete. But its implementation lays out the steps you will need to take in your own project. I have never tried this with a dll, but I don't see any reason for this to pose a problem.

Console.h:

#pragma once
#include <stdio.h>

class CConsole {
    FILE m_OldStdin, m_OldStdout;
    bool m_OwnConsole;
public:
    CConsole();
    ~CConsole();
};

Console.cpp:

#include <windows.h>
#include <conio.h>
#include <FCNTL.H>
#include <io.h>
#include "Console.h"

static BOOL WINAPI MyConsoleCtrlHandler(DWORD dwCtrlEvent) { return dwCtrlEvent == CTRL_C_EVENT;}

CConsole::CConsole() : m_OwnConsole(false) {
    if(!AllocConsole()) return;

    SetConsoleCtrlHandler(MyConsoleCtrlHandler, TRUE);
    RemoveMenu(GetSystemMenu(GetConsoleWindow(), FALSE), SC_CLOSE, MF_BYCOMMAND);
    const int in  = _open_osfhandle(INT_PTR(GetStdHandle(STD_INPUT_HANDLE)),  _O_TEXT);
    const int out = _open_osfhandle(INT_PTR(GetStdHandle(STD_OUTPUT_HANDLE)), _O_TEXT);
    m_OldStdin  = *stdin;
    m_OldStdout = *stdout;

    *stdin  = *_fdopen(in,  "r");
    *stdout = *_fdopen(out, "w");

    m_OwnConsole = true;
}

CConsole::~CConsole() {
    if(m_OwnConsole) {
        fclose(stdout);
        fclose(stdin);
        *stdout = m_OldStdout;
        *stdin  = m_OldStdin;
        SetConsoleCtrlHandler(MyConsoleCtrlHandler, FALSE);
        FreeConsole();
    }
}

Enjoy!