The config directory location is taken from the GetUserDataDir function provided by wxWidgets.
If you think it should return a different value according to your environment variables, please report your feature request to the wxWidgets feature tracker.

But it seems supports it (xdg-dirs are mentioned the in source code)

Maybe it should be GetDocumentsDir
http://trac.wxwidgets.org/browser/wxWidgets/trunk/src/unix/stdpaths.cpp (Ctrl-F XDG_CONFIG_HOME)

wxWidgets supports XDG since version 3.1.1 (merged 15.03.2017, released 19.02.2018).
Slic3r seems to use wxPerl which had it's last release 0.9932 18.04.2017 so the important parts may be missing. However I wasn't able to find any documentation on how to access the Wx::StandardPaths even through it's already used in the code.

If it's possible to access the underlying functions this should be quite easy to implement:

1. Set wxStandardPaths::SetFileLayout(FileLayout_XDG)

That's it but while we're at it we could also enhance the used folder.
The following steps are optional:

2. Change GetUserDataDir to GetUserConfigDir since only config files are stored anyway.
3. Because of the previous change it's now needed to append the wxAppConsole::GetAppName() to the folder path.

Either if it's possible or not this feature request should be reopened as Slic3r needs some changes to adapt to the standard.

wxPerl only supports wxWidgets 3.0.4, and it's not very feasible to access the underlying functions last I checked (not w/o modifications to wxPerl). I'd accept work on this for the C++ GUI (which is under work).

(I know because I spent a little bit of time trying to get wxPerl built with 3.1 unsuccessfully).

@Lucki I'd prefer to put a double check in there. First check UserDataDir ~/.Slic3r, then move to UserConfigDir.