C++ – Linker library path in makefile confusion


I've been programming a while but I still don't fully understand how the linker behaves.

For example, today I downloaded and installed a library that I want to use in my application in Linux. (It was Xerces – for parsing XML files).

I created a makefile and gave it the path to the .so and .a files in my command : -L/usr/local/lib and also told it the name of the library to include : -lxerces-c-3.1.

My application compiled fine but failed during runtime with "cannot open shared object file libxerces-c-3.1.so". Why would this be the case, when I properly give it the path and name in the makefile?

I then added the library path to the LD_LIBRARY_PATH variable in my .bashrc file and then it worked. That's fine , but if I now remove the path to the library in my makefile and don't even include the name of the library , it still works.

I'm confused as to what is going on here. How can it still find the correct library just by assigning the path to the LD_LIBRARY_PATH variable and will only work if I do so? I have read elsewhere to not even use LD_LIBRARY_PATH.

I appreciate any answer for this. The question is a bit long and hopefully not off-topic but I hope others can learn from it too. Thanks

Best Solution

compilation and running are different things. :)

1) A make file contains rules on how to build your application. So when you write a rule like:

    -L/usr/local/lib -lxerces-c-3.1

You are passing options to the linker. The -L option tells the linker to add additional libraries (in this case '/usr/local/lib') to the linker's search path. The -l option names the library that should be linked in.

2) When you go to run an executable, the loader needs to find all the required libraries. For example, on a Linux system you can use the ldd command to see what shared libraries are used. For example on my system:

ldd FEParser
    linux-vdso.so.1 =>  (0x00007ffcdc7c9000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f835b143000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f835ae3d000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f835ac27000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f835a862000)
/lib64/ld-linux-x86-64.so.2 (0x00007f835b447000)

From this, you can see the name of the library that is linked to the left of the '=>' token, and the path to the library to the right. If a library is not found it will show up as missing.

Now in your case, you were able to successfully compile and link your program due to giving the path and library name to use. You were not able to run the program due to the loader not being able to find the library at run time.

You have several options here:

1) you can move the library in a directory that is in the loaders search path.

2) you can modify LD_LIBRARY_PATH which adds additional directories to the loaders search path. Additionally, the directories specified in LD_LIBRARY_PATH will be passed to the linker (it will be appended after all -L flags). You do need to be careful with this, as if you set it globally (say in .bashrc), then it will effect all compilations that you do. You may or may not want this behavior.

3) As others have specified you can use -Wl,rpath=...., but it is deprecated, and I rarely use it.

4) If you need the library installed in an unusual location, you can add a create a file under /etc/ld.so.conf.d that contains, for example (file is yaml.conf):

# yaml default configuration

At system boot, the loader reads all the files in this directory and appends the paths to the loader path. If you make a modification to this directory you can use the ldconfig command to cause the loader to reprocess /etc/ld.so.con.d. N.B. I know this works on centOS and Ubuntu flavors of GNU/Linux - can't speak authoritatively on other flavors.