Determining which EDoc++ data files need to be used

When using EDoc++ on an application it is necessary to include EDoc++ data for all libraries and objects that are linked into the application.

The short of the story is that EVERYTHING (library or object) that is linked into an application "should" have an EDoc++ data file representing that entity that is merged together in order to check the exception safety of an application. However in most real situations at least while EDoc++ is relatively unknown, very few libraries will provide EDoc++ data files and thus either the user will be required to recompile the library using EDoc++ to generate appropiate data files or the user will have to accept that some information required for a complete view of exception propagation can not be accomplished. Some libraries (especially C libraries) that the application uses may not offer much if any useful data to EDoc++.

A number of C libraries will not require EDoc++ data to have been generated for them depending on their usage. In particular it is generally accepted that exceptions will not propagate from a C library (There may be a few exceptions to this rule : pun un-intended). If this is a suitable assumption then it is usually fine to omit the EDoc++ data for those libraries. Note however that it may be necessary to provide some suppressions in orer to remove WUNIMPL warnings or other that may result from omitting the necessary data.

There is however one other point that needs to be made for these C libraries. If the library make use of function pointers provided by the user of the library, and some of those function pointers can point to C++ functions, then it is possible for exceptions to propagate from the C++ code into the C code. This is generally undesirable as very little C code is written to work well in the presence of exceptions. As a result if you are providing such function pointer implementations, it is generally a good idea to enforce that no exceptions propagate into the C code using a function throws() specifier.

Resource types

When giving data to the edoc application to process you can flag that data as one of two types.

  1. Primary data

    This is data that you want edoc to give you information about. If you want to generate doxygen documentation for a library called libmyshared, then you will add libmyshared as a primary input source.

  2. Additional data

    This is data that may be required for generating accurate information, however the additional data is NOT primarily the data you want information about. Following on from the previous example if libmyshared made use of libstdc++ then you will want to generate documentation for libmyshared but NOT for all the functions defined in libstdc++. This would mean that libstdc++ is additional information required to produce an accurate result and should be thus flagged as aditional input data.

By default the edoc application will attempt its best to suppress any errors/warnings related to the additional data, since you are generally not debugging the additional data but the primary data. It will do this by automatically importing any .eds file that may be associated with the additional data. It will also mark all functions defined in the additional data as NOT visible, so no documentation will be produced for those functions.

As an example we will use libmyshared.

Lets run edoc over the data for libstdc++ AND libmyshared both as primary data sources.

EDOC -> $ edoc --disable-auto-import --format simple libmyshared.so $HOME/build/install/edoc_patched/lib/libstdc++.so
OR...
EDOC -> $ edoc --disable-auto-import --format simple --primary libmyshared.so $HOME/build/install/edoc_patched/lib/libstdc++.so
OR...
EDOC -> $ edoc --disable-auto-import --format simple --primary libmyshared.so --primary $HOME/build/install/edoc_patched/lib/libstdc++.so

The examples above are identical. By default all data added to edoc is added as primary data so in the above example the --primary flag is unnecessary.

Now lets run edoc over the same data, but this time we will add libstdc++.so as additional data.

EDOC -> $ edoc --disable-auto-import --format simple libmyshared.so --additional $HOME/build/install/edoc_patched/lib/libstdc++.so
OR...
EDOC -> $ edoc --disable-auto-import --format simple --primary libmyshared.so --additional $HOME/build/install/edoc_patched/lib/libstdc++.so
OR...
EDOC -> $ edoc --disable-auto-import --format simple --additional $HOME/build/install/edoc_patched/lib/libstdc++.so --primary libmyshared.so

The examples above are also identical. However note that in the last example the --primary must not be omitted, as it needs to set edoc back into the primary state for the libmyshared to be added as primary data. The edoc application only starts off in primary state.

Notice the data generated when adding libstdc++.so as additional data does not include all the documentation for the functions implemented in libstdc++. This is a very helpful feature.

In order to suppress all the warnings/errors that occur when importing libstdc++, edoc automatically imports the libstdc++.so's suppression file: libstdc++.so.eds You can see this file is located in the directory right next to the libstdc++.so file. You can also tell edoc to NOT import this suppressions file automatically by adding: --disable-auto-suppressions to the command line. Again this works like the primary and additional flags in that you can re-enable auto import for later libraries using --enable-auto-suppressions

Static Libraries

If a library is linked statically with an application and both the library and the application where created using EDoc++ with embedded data, then there is no need to manually supply the EDoc++ data file for that library. The linker will automatically store the EDoc++ data files into the resulting binary for both the library and the objects that make up the application. You may however want to include the libraries suppression files though as for static libraries this is currently not done automatically for you.

As an example we will expect that the EDoc++ data from libmystatic will already be embedded in the library_user application. To check this lets run the edoc application on the library_user application and see:

edoc -d1 --disable-auto-import library_user
Loading edc file: library_user:../../../gcc-4.0.4/gcc/crtstuff.c.edc...
Loading edc file: library_user:/home/bcosta/build/edoc/dist/src/example-0.2.1/src/apps/library_user/main.cpp.edc...
Loading edc file: library_user:/home/bcosta/build/edoc/dist/src/example-0.2.1/src/libs/mystatic/mystatic.cpp.edc...
Loading edc file: library_user:/home/bcosta/build/edoc/dist/src/example-0.2.1/libltdl/ltdl.c.edc...
Logs going to file: /tmp/edoc_i8yeV7
Loading edc file: library_user:../../../gcc-4.0.4/gcc/crtstuff.c.edc...

You can see from the above listing that the library_user application contains a file that was extracted called: mystatic.cpp.edc This file belongs as part of the static library: libmystatic and has automatically been included into this application at link time.

Dynamic Libraries

All dynamic libraries linked with an application also need to have EDoc++ data files added when processing EDoc++ data for the application. In order to see which dynamically linked libraries an application makes use of so you can include their EDoc++ data files, you can use the ldd command. For example running this on the library_user application gives the following:

ldd library_user
	linux-gate.so.1 =>  (0xb7ef5000)
	libmyshared.so.0 => /home/bcosta/build/edoc/dist/install_example/lib/libmyshared.so.0 (0xb7ef1000)
	libstdc++.so.6 => /home/bcosta/build/edoc/install/edoc_patched/lib/libstdc++.so.6 (0xb7e0f000)
	libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7dff000)
	libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7dda000)
	libgcc_s.so.1 => /home/bcosta/build/edoc/install/edoc_patched/lib/libgcc_s.so.1 (0xb7dcf000)
	libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7c80000)
	/lib/ld-linux.so.2 (0xb7ef6000)

First thing to notice is that there are additional libraries we did not explicitly link with that this application is dependent upon. Some of these came from using libtools libltdl library, however you will notice that some of these libraries exist in almost every C++ application. This includes:

  • linux-gate.so.1

  • libstdc++.so.6

  • libgcc_s.so.1

  • libc.so.6

Of the above libraries the following are distributed as part of the modified GCC that is installed with EDoc++. As a result they will already have EDoc++ data generated for them.

  • libstdc++

  • libgcc_s

For all dynamic libraries it is necessary to look at what functions from those libraries the application in question uses and then make a decision as to whether to include the EDoc++ data for that library.

Note: It is not always easy to obtain the EDoc++ data files for a given library. I hope that in the future that more libraries might be distributed with EDoc++ data embedded or at least installed alongside the library, however for now it is expected that a user will generally have to compile all dependant libraries themselves in order to obtain this data.

In this section we will attempt to import EDoc++ data for the libraries this application makes use of in order to try and satisfy the dependencies. In later sections we will discuss suppressions and will attempt to reduce the number of requirements we add into the final calculations and remove some of these libraries. In a real project we would just choose one method or the other, but this is done for demonstration purposes.

Firstly i will note that we will NOT include EDoc++ data for the following libraries as the EDoc++ data is just too difficult to obtain for demonstration purposes. You can figure out how to compile these using the EDoc++ modified GCC compiler at a later date for yourselves.

  • linux-gate.so.1

  • libdl.so.2

  • libm.so.6

  • libc.so.6

  • ld-linux.so.2

So that leaves us with the following files to include in the EDoc++ merge operation.

  • libmyshared.so.0

  • libstdc++.so.6

  • libgcc_s.so.1

Auto Import

Before going much further it will be helpful to describe the EDoc++ auto-import feature. As of version 0.1.0 EDoc++ has the ability to automatically run ldd on any binary application specified on the command line to obtain the list of dynamic libraries that the application makes use of. For each of these libraries EDoc++ will then attempt to locate the EDoc++ data for those libraries and add it as additional data. This can greatly simplify EDoc++ usage as the average usage will be just to run:

edoc appname --format=doxygen --output=appname.dox

By default auto-import is enabled. In many of the previous examples we have been explicitly disabling it. This has been to simplify the output for demonstration purposes. The flag to disable auto import is simply: --disable-auto-import This will affect any binary objects placed on the EDoc++ command line following the flag. In order to re-enable the automatic importing of shared libraries simply use the flag: --enable-auto-import which will re-enable automatic importing of shared library EDoc++ data for all binary objects placed on the command line following the statement.

We will complete the library_user application example with auto-import disabled.

libmyshared

This library is compiled as part of the example project. As a result it has already been compiled with EDoc++ data embedded into it. So there is no more we need to do for this library but include it on the edoc application command line:

edoc -d1 --disable-auto-import library_user ../lib/libmyshared.so

Executing that command you will notice that the data for libmyshared is included into the merge.

libstdc++ & libgcc

These two libraries are a part of the GCC distribution. By default when compiling the EDoc++ project it will automatically generate EDoc++ data for its GCC libraries and embed it into the installed binaries.

So we include these libraries in the command line to import the embedded data. We will do a complete merge of all these libraries into a EDoc++ data file for the library_user application called: library_user.edc

So in order to run edoc for the application we can run the following command:

edoc -d1 --disable-auto-import --merged-output library_user.edc \
   ../lib/libmyshared.so \
   $HOME/build/install/edoc_patched/lib/libstdc++.so \
   $HOME/build/install/edoc_patched/lib/libgcc_s.so

The more libraries you add to be merged, the longer it will take to process the data. There is quite a lot of processing occurring and as a result it is sometimes just simpler to omit libraries that you know will have little (If any) effect on the result.

The above will have generated a file: library_user.edc that can then be used for later calculations of exception information. We will refer to this file in a later section.

Plugins

You will notice that after taking into account the static and shared libraries, there is still one element of our program missing and that is the plugin: myplugin. It is not always possible to include plugins since a plugin may be created by a user at a later date which is then used by the program, however it may be desirable to include it in the calculations if it is available.

As a side note, it may be helpful to use function throw specifiers for all functions/methods that span the boundary of application/plugin. This way you are guaranteed that no additional exceptions will come through from a plugin interface. That will then place the onus on the plugin writer to then ensure that their plugin does not generate any exceptions that are not expected.

Adding plugins to the mixture can be more difficult as there is currently no mechanism to determine what symbol names are being loaded from a plugin. This means that a user really needs to add explicit Function::address_taken list items using the suppressions functionality or explicitly replace calls to the symbol function pointers with calls to the appropriate functions.

@@@Brendon WRITEME The rest of this section is a TODO item that will be completed later. If you need to know this information now then send an email to the mailing list.