Python 3.8.0a3 failed to get the Python codec of the filesystem encoding - cpython

Question

When I call the embedded python from my c++ app, the application crashes. Something in _PyCodec_Lookup is not working. Debugging seems to indicate the error comes from this line in file codecs.c:
PyInterpreterState *interp = _PyInterpreterState_GET_UNSAFE();
if (interp->codec_search_path == NULL && _PyCodecRegistry_Init())
goto onError;
interp->codec_search_path is NULL and _PyCodecRegistry_Init() returns -1.
This problems is already known; Solutions like removing python27, or setting environment variables PYTHONPATH or PYTHONHOME to either "" or the src path do not work.
I link to cpython version 3.8.0a3 which I've compiled with the included scripts inside the PCBUILD folder, using the same compiler as my applications (MSVC 15 2017).
The OS is windows 8.1 64 bit.
My application links to the produced binary python38.dll / python38.lib successfully but then crashes at runtime, and I have no idea what might cause the trouble - people suggest the environment is polluted, but it must not depend on the system's environment. How would I confidently ship the app to other people's computers? Did I maybe miss some compile flags or defines?

Answer 1

I'm sure you are, but you are doing this after Py_Initialize ?
I include the python3.8 dll but when starting up I take the executable full
path and then add that to the python path (the code below is not tested but
it's roughly what I do)
void py_add_to_path (const char *path)
{
PyObject *py_cur_path, *py_item;
char *new_path;
int wc_len, i;
wchar_t *wc_new_path;
char *item;
new_path = strdup(path);
py_cur_path = PySys_GetObject("path");
for (i = 0; i < PyList_Size(py_cur_path); i++) {
char *tmp = strappend(new_path, PATHSEP);
myfree(new_path);
new_path = tmp;
py_item = PyList_GetItem(py_cur_path, i);
if (!PyUnicode_Check(py_item)) {
continue;
}
item = py_obj_to_str(py_item);
if (!item) {
continue;
}
tmp = strappend(new_path, item);
free(new_path);
new_path = tmp;
free(item);
}
/* Convert to wide chars. */
wc_len = sizeof(wchar_t) * (strlen(new_path) + 1);
wc_new_path = (wchar_t *) malloc(wc_len, "wchar str");
mbstowcs(wc_new_path, new_path, wc_len);
PySys_SetPath(wc_new_path);
}

Answer 2

Seems a bit old now but looks like there is need to add the Lib folder including all the python system files to the search paths of the application.
Take a look at this Python port for UWP
It has an embedded python34app.zip that includes the codec files.