DK tools and libraries 3.0.9

Get DK tools and libraries at SourceForge.net. Fast, secure and Free Open Source software downloads.
 

FAQ

Basics

What is dktools?

The dktools package contains a set of applications and some libraries needed to build them.

Which license terms are applied to the package?

  • dktools
    Copyright (C) 2000-2013 Dipl.-Ing. Dirk Krause
  • Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
    • Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
    • Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
    • Neither the name of the Dirk Krause nor the names of contributors may be used to endorse or promote products derived from this software without specific prior written permission.
  • This software is provided by the copyright holders and contributors ``as is'' and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed.
    In no event shall the copyright owner or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.

Where can I get the latest version?

This project is hosted on SourceForge, check http://dktools.sourceforge.net/ for information.
The download area is below the project page http://sourcforge.net/projects/dktools/

Version 3 FAQ

What's the reason for the rewrite?

  • Pure C/C++
    Version 2.x.x contained software written in C, Perl, and Java.
    Skills in these three languages were required to build binary installation packages, users needed to install Perl and a Java runtime environment.
    Version 3.0.0 is written completely in C/C++, so it should be easier to create binary installation packages.
  • Better internationalization
    The computer systems I use have different text encodings today:
    • On Linux we find UTF-8 encoding on most systems today. You will recognize the LANG environment variable ending in ".UTF-8" or ".utf8".
    • On some Linux/UNIX systems there are people running legacy applications not capable of UTF-8. So we use the ASCII character set for english with some extensions in the range 0x80...0xFF for non-english characters.
    • On Windows the UTF-16 encoding is widely used. If we want programs to print non-english texts in the command prompt, we should use /D_UNICODE=1 when building the software.
    The library contains text routines for dealing with plain 8-bit character strings, UTF-8 encoded 8-bit character strings, UTF-16 encoded 16-bit character strings and partially with unencoded 32-bit character strings. Conversion routines are available for conversion between UTF-8, UTF-16, and 32 bit characters strings. Conversion from these encodings to plain 8-bit characters strings will loose all characters above 0xFF.
  • The const keyword
    I want to be able to compile the code using C++ compilers too, so I have to take care of the "const" keyword. The const qualifier is applied to all functions where it is applicable.
  • Code cleanup and simplification
    • Some code for DOS (i.e. DOS-style search for the executable name) was removed.
    • Some configurable settings were removed, fixed settings are used instead. The ${datadir} directory for example is now detected automatically, not configured via preferences. This results in smaller code and in less administration settings.
    • String tables (or message catalog files) are UTF-8 encoded text files now, not longer binary files. So all one needs to localize an application is a text editor capable of saving files in UTF-8 encoding. No string table compiler is needed.
  • Diagnostics built into library functions
    Most of the library function have a prototype "return-type fct_xyz_app(...arguments, dk3_app_t *app);" where the app argument is allowed to be NULL.
    If app is not NULL the function uses it to write diagnostics if a problem arises within the function.
    Application code can use 
    dkChar *mycopy;
mycopy = dk3str_dup_app(original, app);
if(mycopy) {
  /* Do something with the copy */
}
    instead of 
    dkChar *mycopy;
mycopy = dk3str_dup(original);
if(mycopy) {
  /* Do something with the copy */
} else {
  /* Print error message, not enough memory */
}
    So the application code will be smaller, error messages will be more consistent (same message for the same problem).
  • Documentation cleanup and simplification
    Now the main documentation is in HTML. For the "--help" option the programs show a minimalistic documentation on standard output.
    In previous versions I had documentation in several formats: HTML, PDF, man pages and the --help texts. This large amount of documentation was difficult to maintain.

Installation FAQ

Directory structure

Which directory structure is used?

When installing a software package you have binaries, configuration files, resource files and documentation.
While the software is running it might create temporary temporary files, log files...
Typically you will have a root directory - sometimes referred to as ${prefix} and subdirectories for the different file types.

Name Directory
${prefix} Installation root directory
${sysconfdir} System configuration directory (contains configuration files)
${bindir} Directory for executable files
${libdir} Directory for libraries and shared libraries
${sbindir} Directory for executable files used only by the super user
${datadir} Directory for resource files
${localstatedir} Directory for log files and temporary files
${mandir} Directory for man pages (documentation)

Here are some example configurations:

Typical installation from source on Unix/Linux

Name Directory
${prefix} /usr/local
${sysconfdir} /usr/local/etc
${bindir} /usr/local/bin
${libdir} /usr/local/lib
${sbindir} /usr/local/sbin
${datadir} /usr/local/share
${localstatedir} /usr/local/var
${mandir} /usr/local/share/man

Installation from source using "./configure --prefix=/opt/mysoftware"

Name Directory
${prefix} /opt/mysoftware
${sysconfdir} /opt/mysoftware/etc
${bindir} /opt/mysoftware/bin
${libdir} /opt/mysoftware/lib
${sbindir} /opt/mysoftware/sbin
${datadir} /opt/mysoftware/share
${localstatedir} /opt/mysoftware/var
${mandir} /opt/mysoftware/share/man

Typical installation using a binary package

Name Directory
${prefix} /usr
${sysconfdir} /etc
${bindir} /usr/bin
${libdir} /usr/lib
${sbindir} /usr/sbin
${datadir} /usr/share
${localstatedir} /var
${mandir} /usr/share/man

Typical installation on a Windows system

Name Directory
${prefix} C:\Program Files\Krause
${sysconfdir} C:\Program Files\Krause\etc
${bindir} C:\Program Files\Krause\bin
${libdir} C:\Program Files\Krause\lib
${sbindir} C:\Program Files\Krause\sbin
${datadir} C:\Program Files\Krause\share
${localstatedir} C:\Program Files\Krause\var
${mandir} C:\Program Files\Krause\share\man

How does a program find the directory structure?

At the beginning of the program (when dk3app_open() is used to create the application object) the program retrieves the file name of the executable file.
The final part is removed from this file name, so we get the binary directory ${bindir} where the executable file resides. If the ${bindir} ends on "/bin" (Unix, Linux) or "\bin" (Windows) we continue as follows:

  • If ${bindir} matches the value specified at compile time we use the settings specified at compile time for ${sysconfdir}, ${datadir} and ${localstatedir} too.
  • If ${bindir} differs from the value specified at compile time the final "/bin" is replaced by "/etc", "/share" and "/var" to construct ${sysconfdir}, ${datadir}, and ${localstatedir}.

If the ${bindir} does not end on "/bin" or "\bin" all the directories ${sysconfdir}, ${datadir}, ${localstatedir} use the same value as ${bindir}.

Relocation

What is software relocation?

Relocating a software during installation means to install a software to a directory different from the installation directory specified in the build process.
When building from source you will most likely run

./configure
make
make install

This builds the software with ${prefix}, ${sysconfdir}, ${bindir} and ${localstatedir} pointing to "/usr/local", "/usr/local/etc", "/usr/local/bin" and "/usr/local/var" and installs the software into these directories.

If you decide to use

make DESTDIR=/opt/mylargedisk install

instead of

make install

the software goes to "/opt/mylargedisk/usr/local", "/opt/mylargedisk/usr/local/etc", "/opt/mylargedisk/usr/local/bin" and "/opt/mylargedisk/usr/local/var" instead.
This might cause problems as the directories differ from the defaults compiled into the software during build.

When installing a binary package with relocation (i.e. "rpm --prefix=/opt/mylargedisk -i xxx.rpm") we have the same situation: The package was configured and built using

./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make

and contains built-in default values "/usr", "/etc", "/usr/bin" and "/var" but the software goes to "/opt/mylargedisk/usr", "/opt/mylargedisk/etc", "/opt/mylargedisk/usr/bin" and "/opt/mylargedisk/usr/share".

Can I relocate the software?

Yes, you can.
If the ${sysconfdir} and ${localstatedir} directories are subdirectories of ${prefix} there is no problem and no need for additional actions.
If the ${sysconfdir} and ${localstatedir} directories are outside the ${prefix} directory - typically the case when installing binary packages - you have to make the configuration files for the software in "/etc" also available in "/opt/mylargedisk/usr/etc". You can use symbolic links to do so, i.e.

ln -s /etc /opt/mylargedisk/usr/etc

Installation on Unix/Linux

System preparation FAQ

What does "system preparation" mean?

Before you install this software or any of the required/recommended libraries you must make sure the software can be built.
The C compiler must be ready to use both header files distributed with your system and as a part of additional software. The linker must be configured to use both system libraries and additional libraries.
Some C compilers and linkers are only pre-configured to use libraries shipped with the OS or with the compiler, but no additional libraries. In such cases you need to extend the search paths.
If additional software is installed into /usr/local (found on many Unix/Linux systems), /sw (found on MacOS) or /opt/another-directory (just an example) the C compiler must be configured to search for header files in /usr/local/include, /sw/include and /opt/another-directory/include in addition to the default directories. The linker must be configured to search for library files in /usr/local/lib, /sw/lib and /opt/another-directory/lib.

Why doesn't the configure script handle an incomplete setup?

  • If software distributions provide a C compiler and libraries and install both into /sw, it is common sense that C compiler and linker must have /sw/include and /sw/lib in the search paths. The software distributions installation process should set this automatically or remind you to do it manually.
  • The effort would be growing with m*n*p where m is the number of directories (/usr/local, /sw, /opt/another-directory, /opt/yet-another-directory...), n is the number of compilers to support (one wants CFLAGS, one wants C_INCLUDE_PATH...) and p is the number of projects to maintain.
  • Automatic guessing of directories is not the usual behaviour of configure scripts. Most users expect the configure script to use exactly the search paths they have configured.
  • To check for a number of directories and to support a number of compilers would produce more code in the configure script than the real configure script contents. But I like to keep things small and simple.
  • This large amount of work is possibly (and likely) for the trash can after software distribution comes with a useful setup.

[gcc] How can I report the include search path?

Type

gcc -v -E -

and press CTRL-D after inspecting the output.
Normally the library search path is corresponding to the include search path, the final "include" subdirectory is replaced by "lib".

How can I set the search paths?

You can set the environment variables CFLAGS and LDFLAGS, i.e.

CPPFLAGS="-I/usr/local/include -I/sw/include -I/opt/another-directory/include"
CFLAGS="-I/usr/local/include -I/sw/include -I/opt/another-directory/include"
LDFLAGS="-L/usr/local/lib -L/sw/lib -L/opt/another-directory/lib"
export CPPFLAGS CFLAGS LDFLAGS

To have the settings available immediately after each login, add a section

if [ "X$CPPFLAGS" = "X" ]
then
  CPPFLAGS="-I/usr/local/include -I/sw/include -I/opt/another-directory/include"
else
  CPPFLAGS="$CPPFLAGS -I/usr/local/include -I/sw/include -I/opt/another-directory/include"
fi
if [ "X$CFLAGS" = "X" ]
then
  CFLAGS="-I/usr/local/include -I/sw/include -I/opt/another-directory/include"
else
  CFLAGS="$CFLAGS -I/usr/local/include -I/sw/include -I/opt/another-directory/include"
fi
if [ "X$LDFLAGS" = "X" ]
then
  LDFLAGS="-L/usr/local/lib -L/sw/lib -L/opt/another-directory/lib"
else
  LDFLAGS="$LDFLAGS -L/usr/local/lib -L/sw/lib -L/opt/another-directory/lib"
fi
export CPPFLAGS CFLAGS LDFLAGS

to your .profile file.
To make the settings available to all users, add the section to /etc/profile.
Note: Different shells might use different file names and a different syntax.

[gcc] How can I set default search paths?

For gcc you can use environment variables instead of command line options to extend the search paths. Add

if [ "X$C_INCLUDE_PATH" = "X" ]
then
  C_INCLUDE_PATH="/usr/local/include:/sw/include:/opt/another-directory/include"
else
  C_INCLUDE_PATH="${C_INCLUDE_PATH}:/usr/local/include:/sw/include:/opt/another-directory/include"
fi
if [ "X$CPP_INCLUDE_PATH" = "X" ]
then
  CPP_INCLUDE_PATH="/usr/local/include:/sw/include:/opt/another-directory/include"
else
  CPP_INCLUDE_PATH="${CPP_INCLUDE_PATH}:/usr/local/include:/sw/include:/opt/another-directory/include"
fi
if [ "X$LIBRARY_PATH" = "X" ]
then
  LIBRARY_PATH="/usr/local/include:/sw/include:/opt/another-directory/include"
else
  LIBRARY_PATH="${LIBRARY_PATH}:/usr/local/include:/sw/include:/opt/another-directory/include"
fi
export C_INCLUDE_PATH CPP_INCLUDE_PATH LIBRARY_PATH

either to your .profile (for a user-specific setup) or to /etc/profile (for a system-wide setup).
Note: Different shells might use different file names and a different syntax.

Build process FAQ

Which libraries should I install before building dktools?

  • zlib (required)
    http://www.zlib.net/
    The zlib library is used to read and write compressed files. The dkbif library (which is in turn used by bmeps) uses this library, the fig2vect program too.
  • libbz2
    http://www.bzip.org/
    The libbz2 library is used for compression.
  • libpng (required)
    http://www.libpng.org/pub/png/libpng.html
    The libpng library is used by the dkbif library (which is in turn used by bmeps) and by the fig2vect program.
  • JPEG library
    http://www.ijg.org/
    The JPEG library is used by the dkbif library (which is in turn used by bmeps) and by the fig2vect program.
  • TIFF library
    http://www.remotesensing.org/libtiff
    The TIFF library is used by the dkbif library (which is in turn used by bmeps).
  • OpenSSL (required)
    http://www.openssl.org/
    OpenSSL is used to create messages digests (hashes) of files in kls and to create random numbers in the dkappr module (which is used by createp and rndbytes).
  • At least one of the following is recommended: NDBM is a database API, some commercial Unix variants (i.e. Solaris) have NDBM support in their C runtime libraries. On these systems I recommend to use the NDBM functions shipped with the system. On systems without "native" NDBM you should install Berkeley DB.

I installed all required libraries. Why do I get error messages about *.h files not found?

Make sure to install the include files and other files for development process too! For some Linux distributions the header files for library xyz are in a package having "xyz-devel" or "xyz-developer" in the name.

On some operating systems the db.h file is installed as /usr/include/db45/db.h instead of /usr/include/db.h. This header file is not found if /usr/include/db45 is not in the search path.
There are two solutions:

  • The correct one:
    Contact the vendor of the package including the db.h file to correct the installation routine.
    On my Fedora Core 14 system I had different versions of BDB installed: the current one from the db4/db4-devel packages and previous versions from compat-db45/compat-db46/compat-db47 packages. The db4/db4-devel packages contain the latest version and install the header files to /usr/include, the compat-dbxy packages install their header files into dedicated subdirectories. So the current version is ready to use and can be included using 
    #include <db.h>
    as described in the Berkeley DB manuals.
    On Scientific Linux 6.3 the situation is similar: There are db.h files in /usr/include/db4.3.29, /usr/include/db4, /usr/include/wx-2.8/wx and /usr/include/db4.2.52. The /usr/include/db.h file is a symbolic link to /usr/include/db4/db.h (most recent version).
  • The dirty hack:
    Execute the following commands (sh, bash) before running the configure script: 
    export CPPFLAGS="$CPPFLAGS -I/usr/include/db45"
export CFLAGS="$CFLAGS -I/usr/include/db45"

How do I handle incompatible versions of recommended libraries?

The configure script only checks for the presence of header files in most cases. For some libraries there are significant API changes between different versions. Run

  make clean

and edit config.h manually to remove the "HAVE_..." lines enabling the problematic header files.
Run

  make

again.

Example: Disabling use of the Berkeley DB
The dksdbdb module was written to use BDB 4.5 (or above). If you have version 4.4 you may get compilation errors.
The main header file for Berkeley DB is ``db.h''.
So run

  make clean

and edit the ``config.h'' file.
Replace

  #define HAVE_DB_H     1

by

  #define HAVE_DB_H     0

and continue compilation.

How do I build and install the software

After unpacking the archive using (version number may differ)

unzip dktools-x.y.z

change into the new "dktools" directory and run

./configure
make
make install

Why doesn't the installation use the NDBM I have available on my system?

There are different implementation of the NDBM API, I know at least of native NDBM libraries shipped with the OS (i.e. Solaris) and an NDBM implementation which is included in the GDBM distribution.

For GDBM the GNU General Public License is used, so each work using GDBM can be distruted under the GPL terms only.
As I prefer to distribute dktools under a BSD license, I do not use GDBM's NDBM implementation.
Short summary: If both NDBM and GDBM are found by the configure script, the NDBM library is not used.
If you are on a system with native NDBM and you have installed GDBM additionally and you want to use the native NDBM libraries with dktools, do the following:

  • Make sure -lndbm uses the native NDBM libraries, not the NDBM library from GDBM. The LDFLAGS environment variable contain the -Ldirectory option for the native NDBM library before the option for the directory where the GDBM implementation resides.
  • Between "./configure" and "make" edit the files config.h and dk3conf.h, change the definitions of HAVE_GDBM_H and DK3_HAVE_GDBM_H from 1 to 0.

[Mac OS + Fink] Are there special installation instructions?

Note: I'm not a Mac user, so I can not give very detailed information. But I'll attempt to describe the process here.

  • The dktools project depends on some other libraries. Mac OS versions of these libraries are available in the Fink project.
  • If you did not yet install Fink (see http://www.finkproject.org), install it now.
    If you can choose packages manually, make sure to install
    • zlib and zlib-devel
    • libbz2 and libbz2-devel
    • libpng and libpng-devel
    • libjpeg and libjpeg-devel
    • libtiff and libtiff-devel
    • openssl and openssl-devel
    The package names shown here are as on my Linux system. May be names differ on Mac OS / Fink. The important point is to not just install binaries but header files and development support too.
  • I do not know whether or not the Fink installation process sets up include paths and library paths for your compiler and linker to use the Fink header files and libraries.
    Previous versions didn't.
  • So before running configure we have to make sure to set the environment variables properly: 
    CFLAGS="-I/sw/include $CFLAGS"
CPPFLAGS="-I/sw/include $CPPFLAGS"
LDFLAGS="-L/sw/lib $LDFLAGS"
export CFLAGS
export CPPFLAGS
export LDFLAGS
./configure
make
make install
    If you have a 64-bit version of Fink, you might have an /sw64 directory instead of /sw. If you have a /sw64 directory you need to replace all occurances of ``/sw'' by ``/sw64'' in the commands shown above.
  • The default installation goes into the /usr/local directory. You must make sure the directory ``/usr/local/bin'' is listed in the PATH environment variable of your system.
  • If you want to compile and install software using the dktools project you have to set environment variables properly to contain ``/usr/local/include'' and ``/usr/local/lib'' before you call the configure script of the other packages: 
    CFLAGS="-I/sw/include -I/usr/local/include $CFLAGS"
CPPFLAGS="-I/sw/include -I/usr/local/include $CPPFLAGS"
LDFLAGS="-L/sw/lib -L/usr/local/lib $LDFLAGS"
export CFLAGS
export CPPFLAGS
export LDFLAGS
    Again, if you have an ``/sw64'' directory instead of ``/sw'' you must replace all occurances of ``/sw'' by ``/sw64'' in the commands shown above.

How can I get help using the configure script?

The usual options can be provided to the configure script, run

./configure --help

to see a list of possible options.

Building shared libraries fails. What can I do to build the software?

The configure script chooses one of three variants to build the libraries:

  • If the configures script finds the Gnu C compiler (gcc) it produces a Makefile to build shared libraries using gcc with appropriate options.
  • If the configure script does not find the Gnu C compiler but another compiler run as ``cc'' it runs ``uname -s''. If this command responds ``SunOS'' the Makefile is written for Sun Workshop compiler and the appropriate options to produce shared libraries.
  • If none of the tests above succeeded the Makefile will produce static libraries.

If the configure script comes to wrong conclusions how to produce shared libraries you can enforce static libraries by running:

./configure --with-static-libraries
Package and port builders FAQ

Which packages should I build?

I suggest to build 2 packages/ports:

  • dktools
    containing the software from dktools-x.y.z.zip.
    This package depends on zlib, bzip2, libpng, jpeg, tiff, openssl and netsnmp.
  • dktools-doc
    containing the documentation from dktools-doc-x.y.z.zip. This is a copy of the dktools web site, it does not depend on any other package, but the user should have a HTML reader and a PDF reader available.

How do I build the software to make a package?

  • In your home directory create a new directory (i.e. ``pkg'') to keep the files for packages.
  • In that new directory create three subdirectories ``dktools'' and ``dktools-doc''.
  • Unpack the dktools-x.y.z.zip archive.
    Change into the dktools-x.y.z directory and run 
    ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
make DESTDIR=$HOME/pkg/dktools install
  • Unpack the dktools-doc-x.y.z.zip archive.
    Change into the dktools-doc-x.y.z directory and run 
    ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
make DESTDIR=$HOME/pkg/dktools-doc install

Now you have all the files which must go into the root directory during the installation of your packages in the directories $HOME/pkg/dktools and $HOME/pkg/dktools-doc.
Creating the *.rpm, *.deb, *.pkg or whatever installation packages is out of the scope of this FAQ as this depends on the package management system used by your distribution and I have no experiences in these things.

RPM users FAQ

Can I relocate the RPM

Yes, you can.
But manual changes to configuration files may be necessary, i.e. to include the .../bin and .../sbin directories into the PATH, to set LD_LIBRARY_PATH and MANPATH.

Administration FAQ

File search

What is the search path for configuration files?

If the application myapp from the application group myappgroup searches for the configuration file xyz.conf, it tests the presence of the following files:

  • ${datadir}/dk3app/xyz.conf
  • ${sysconfdir}/dk3app/xyz.conf
  • ${datadir}/dk3app-site/xyz.conf
  • ${sysconfdir}/dk3app-site/xyz.conf
  • ${datadir}/myappgroup/xyz.conf
  • ${sysconfdir}/myappgroup/xyz.conf
  • ${datadir}/myappgroup-site/xyz.conf
  • ${sysconfdir}/myappgroup-site/xyz.conf
  • ${datadir}/myapp/xyz.conf
  • ${sysconfdir}/myapp/xyz.conf
  • ${datadir}/myapp-site/xyz.conf
  • ${sysconfdir}/myapp-site/xyz.conf
  • ${HOME}/.dk3app/xyz.conf
  • ${HOME}/.dk3app/myappgr/xyz.conf
  • ${HOME}/.dk3app/myapp/xyz.conf

The configuration file names at the start of the list are processed first, they have lowest priority. The configuration file names at the end of the list are processed last, they have highest priority.

Why does configuration file search return a collection of files instead of simply returning one file name?

Using multiple configuration files gives more flexibility. Typically the contents of files processed later modifies/overwrites contents of the files processed earlier.

Why do you search for configuration files in ${datadir} and in ${sysconfdir}?

This feature is for those installing applications on a file server. Typically the ${datadir} is on the same share or file system as the ${bindir}.
The ${sysconfdir} typically is a per-machine directory.
So when installing software on a fileserver placing configuration files in ${datadir} makes them immediately available to all the clients without visiting each client.

What's the purpose of .../dk3app and .../dk3app-site?

The directories ${datadir}/dk3app, ${sysconfdir}/dk3app, ${datadir}/application-group, ${sysconfdir}/application-group, ${datadir}/application, and ${sysconfdir}/application are maintained by the installation routines of the software. Installation routines are allowed to replace files in these directories without asking for confirmation. You should not modify or customize configuration files in these directories.

The directories ${datadir}/dk3app-site, ${sysconfdir}/dk3app-site, ${datadir}/application-group-site, ${sysconfdir}/application-group-site, ${datadir}/application-site, and ${sysconfdir}/application-site are the places for customized configuration files. These files are not overwritten during updates.

Preferences

What's the purpose of preferences?

Preferences are key/value pairs used for configuration, both key and value are text strings. Applications can retrieve values by name.
We have system preferences, user preferences, preferences specified on the command line and preferences set by the program itself.

Where can I set preferences?

System preferences can be set by an administrator in one of the following files:

  • ${datadir}/dk3app/dk3pref.conf
  • ${sysconfdir}/dk3app/dk3pref.conf
  • ${datadir}/dk3app-site/dk3pref.conf
  • ${sysconfdir}/dk3app-site/dk3pref.conf
  • ${datadir}/application-group/dk3pref.conf
  • ${sysconfdir}/application-group/dk3pref.conf
  • ${datadir}/application-group-site/dk3pref.conf
  • ${sysconfdir}/application-group-site/dk3pref.conf
  • ${datadir}/application/dk3pref.conf
  • ${sysconfdir}/application/dk3pref.conf
  • ${datadir}/application-site/dk3pref.conf
  • ${sysconfdir}/application-site/dk3pref.conf

A user can set personal preferences in the following files:

  • ${HOME}/.dk3app/dk3pref.conf
  • ${HOME}/.dk3app/application-group/dk3pref.conf
  • ${HOME}/.dk3app/application/dk3pref.conf

Preferences set by the program are written to ${HOME}/.dk3app/application/dk3pref.conf at the end of the program.

When running a program a user might overwrite the "/log/stderr/level" preference using the command line option

--/log/stderr/level=debug

How do I write a dk3pref.conf file?

A dk3pref.conf file is a text file containing scopes (for which users/applications/application groups/hosts/languages/regions will we use the values?) and key/value pairs.
Example:

# This is a comment.
# Comments have a raute character ("#") as first character in the line.
# Empty lines are ignored.

# Next we do some setup for all users
[user=*]
/log/stderr/level=error

# Next we overwrite this for one user
[user=joe]
/log/stderr/level=debug

# Now we overwrite this again for all hosts except the host named "pc0815".
# We first open a new scope and include all hosts, in the next line
# we explicitly exclude one host.
[host=*]
[-host=pc0815]
/log/stderr/level=progress

# Now we use an and-combination of conditions
[host=pc0815,user=jim,language=en,region=us,application=jimapp]
/log/stderr/level=debug

How do I specify a scope in preferences files?

A scope can be specified by one scope line or a sequence of scope lines.
A scope line consists of scope information in square brackets.
The first scope line at the start of a file or after a non-scope line resets the in-scope flag.
The contents of the scope line and the following scope lines can be used to set and reset the flag. If all the conditions in a scope line are true the in-scope flag is set if there is no "-" after "[". The flag is reset if there is a "-" after "[".

Should I use scopes in ${HOME}/.dk3app/application/dk3pref.conf?

No, please do not!
This file is overwritten if the application exits, no scopes are written to that file. All scope information you have entered will be lost.

Is there a list of well-known preference names?

Yes, of course. See Administration => Preferences => Well known preferences.

Should I set up the language using preferences?

No, this is intended for special situations only. By default the programs use the system settings for language and region.
Imagine the following situation: You have installed all your computers with support for english and german language. Now there comes one user speaking spanish and he wants to use mainly application xyzware.
You install xyzware with support for english, german, and spanish and create a preference file entry

[user=mendoza,application=xyzware]
/ui/lang=sp

in ${sysconfdir}/xyzware-site/dk3pref.conf.

Configuration

How can a system administrator establish default settings?

Some programs - i.e. dkt - read configuration files at startup. These configuration files can set up things which are normally set up by command line arguments.
Example:
Instead of running

dkt ls --digest-type=sha-512.ra85 ...

we could specify the digest type and encoding in a configuration file dkt.conf in an ls section:

[ls]
digest type = sha-512.ra85

and run

dkt ls ...

to have SHA-512 as default message digest algorithm and reverse ASCII-85 encoding as message digest encoding.

The administrator can place the configuration files in subdirectories of ${sysconfdir} and/or ${datadir}.

You should use this feature extremely carefully.
You might change the default settings for programs from the default values mentioned in the program documentation to anything else surprising your users.
Users may react by stopping processing of system level configuration files at all or by using the "-R" option to ignore any settings made in configuration files.
So I recommend to use system level configuration files for just one purpose: Increasing security. The example was shown above: Use of a probably more secure message digest algorithm and use of an encoding using lower overhead than the default one.
Remember: The system administrator can configure suggestions, users may decide to ignore them.

How can a user override the administrators default settings?

The ${HOME}/.dk3app/dk3conf.conf file can have entries like:

ignore system config files = filename
ignore system config files = *
use system config files = filename
use system config files = *

Ignoring system config files for a specified (short) file name means not to search in the ${sysconfdir} and ${datadir} subdirectories for that file name, only in the users home directory.
Priority: First matching entry (either with file name or "*") rules, no further lines are examined!

Advanced users might use

ignore system config files = *

to ignore all the system administrators suggestions, but this requires them to configure many things theyrselves.

Note: Disabling system configuration files does not work for "dk3pref.conf".

How can a user override all settings from configuration files?

Use "-R" or "--reset" as first command line option to go back to the built-in default settings.

Random number generation

Which pseudo random number generators (PRNGs) can be used?

During the build process the configure script checks for the following PRNGs:

  • OpenSSL PRNG.
  • srandom()/random() function set.
  • srand48()/lrand48() function set.
  • srand()/rand() function set.

Support for the found mechanisms is compiled in.
The binary Windows distribution is compiled to use the OpenSSL PRNG.

How can I specify which PRNG(s) should be used?

You can set the "/rand/types" preference to contain a comma-separated list of PRNG type names.

Only those PRNGs which are both supported by your system and contained in the list will be used.

How does a PRNG work?

A PRNG is typically a state machine storing an internal state in a variable (or a complex structure) calculating the next state and an output depending on the current state when triggered (asked for output).
When using a PRNG for crypto purposes or authentication the output from the PRNG should be as unpredictable as possible.

What is seeding?

Seeding means to bring the state machine into an initial state before retrieving any output.
If you don't seed a PRNG the PRNG starts in a known state, so output would be predictable.

Should I allow the use of seed files?

It depends...
For users having their home directory on a NFS-shared file system - no.
For users like root having the home directory on a local disk not shared it might be acceptable.

Developer FAQ

Source code

For patches, should I modify *.c or *.ctr?

When sending me patches or patched files, please change the *.ctr files. The *.c files are generated from *.ctr files. Any changes in *.c files are lost when dkct is run again.

Customization and localization

What do I need to create a localized string table or help text?

You need a text editor which can save UTF-8 encoded files.
I use "gedit" with LANG set to "de_DE.utf8".
If the LANG environment variable ends on ".utf8" or ".UTF-8" most Unix/Linux text editor use UTF-8 encoding automatically when saving text.

Where do I place my localized string table?

Run

which dkt

to see where your dkt program is placed.
If you see output like "/usr/local/bin/dkt" run the command

find /usr/local/share -name 'dk3app.str'

to find the string table already installed. You should see output like "/usr/local/share/de/dk3app.str".
Now inspect your LANG environment variable. If it is - for example - "sp_SP" you have to place your localized string table as file "/usr/local/share/sp/dk3app.str".

What about the structure of a string table file?

The file format is very simple:

  • Each non-comment line contains exactly one string which will be read into the string array.
  • A comment line is started by "#" immediately at the beginning of the line. No leading spaces allowed!
  • If you need a string started by "#", use a backslash to escape the "#": "\#".
  • If you need newlines in a string, write it as one text line, use "\n" for the newlines.

How do I know which text to put at which position?

To start, copy the german string table to your new file name and edit the new file.
The german version contains comments in english showing line number(s), purpose and english text version for each of the german strings like:

#
# 129/130: Error message: Unknown encoding "..."!
#
Unbekanntes Encoding: "
"!

Leave the comment section as is, only change the german texts (last two lines) to appropriate texts in your language.
Work carefully, the number and order of texts must not change!

Using the libraries for your own programs

Characters and strings

What's the purpose of the dkChar data type?

Some systems - i.e. Unix/Linux - use the char data type for "normal" text processing. On other systems - i.e. Windows - the wchar_t data type is the preferred type for dealing with text.
Depending on the value of DK3_CHAR_SIZE which can be 1 or 2 the dkChar type is mapped to char or wchar_t.
If you don't set DK3_CHAR_SIZE explicitly in your project/compiler settings it is defined to 1 unless you are on Windows and have _UNICODE defined.

How do I specify dkChar string and character literals?

Use the "dkT" macro.
Example:

dkChar const one_string[] = { dkT("This is a string") };
dkChar c = dkT('\n');

How do I write a dkChar string or character into a file?

Use the dk3sf_fputs() and dk3sf_fputc() functions.

dkChar const one_string[] = { dkT("This is a string") };
dkChar c = dkT('\n');
...
dk3sf_fputs(one_string, stdout);
dk3sf_fputc(c, stdout);

How do I initialize standard output for dkChar use?

Before writing the first dkChar character or string, use the dk3sf_initialize_stdout() function.
This function writes a BOM (byte order marker) if necessary.

dkChar const one_string[] = { dkT("This is a string") };
dkChar c = dkT('\n');
...
dk3sf_initialize_stdout();
dk3sf_fputs(one_string, stdout);
dk3sf_fputc(c, stdout);

Which string function do I use for which data type?

There are variants and macros for the different data types. The functions working for char typically have a "_c8_" in the name, functions for Windows wchar_t have a "_c16_" in the name.
Example:

Data type Function
char dk3str_c8_cpy()
Windows wchar_t dk3str_c16_cpy()
dkChar dk3str_cpy()
Program structure

Which include files do I need?

The easiest way to include the headers is to use:

#include <dk3all.h>

How do I declare the main() function?

#include <dk3all.h>

DK3_MAIN
{
  /* We have int argc and dkChar *argv[] available. */
}

The DK3_MAIN macro is mapped to either "int main(int argc, char *argv[])" or "int wmain(int argc, wchar_t *argv[])" depending on the DK3_CHAR_SIZE value. So we have a "dkChar *argv[]" available.

Application objects

How do I create and destroy an application object?

#include <dk3all.h>

/**     Exit code returned at program end.
*/
static int exval = 0;

/**     Application group name.
*/
static dkChar const gn[] = { dkT("tests") };

/**     Main function.
        @param  argc    Number of command line arguments.
        @param  argv    Command line arguments array.
        @return 0 on success, any other value indicates an error.
*/
DK3_MAIN
{
  dk3_app_t *app = NULL;

  app = dk3app_open_command(argc, (dkChar const * const *)argv, gn);
  if(app) {
    /* ... do something ... */
    dk3app_close(app); app = NULL;
  } else {
    /* ERROR: Failed to create application! */
    exval = 1;
  }
  exit(exval); return exval;
}

How do I load localized string tables?

#include <dk3all.h>

/**     Exit code returned at program end.
*/
static int exval = 0;

/**     Application group name.
*/
static dkChar const gn[] = { dkT("tests") };

static dkChar const * const def_texts[] = {
  dkT("Hello!"),
  dkT("Goodbye"),
  NULL
};

/**     Main function.
        @param  argc    Number of command line arguments.
        @param  argv    Command line arguments array.
        @return 0 on success, any other value indicates an error.
*/
DK3_MAIN
{
  dk3_app_t *app = NULL;
  dkChar const * const *msg;    /* For the localized messages. */
  app = dk3app_open_command(argc, (dkChar const * const *)argv, gn);
  if(app) {
    msg = dk3app_messages(app, dkT("test.str"), (dkChar const **)def_texts);
    /* ... do something ... */
    dk3app_close(app); app = NULL;
  } else {
    /* ERROR: Failed to create application! */
    exval = 1;
  }
  exit(exval); return exval;
}
  • The def_texts array contains the default texts to use if the string table file is not found or if there is any problem while reading the file.
    The array must be finished by a NULL pointer.
  • The dk3app_messages() function searches for the specified string table file and reads the contents. On success it returns an array containing the localized strings. On error it returns the default texts. So the result of dk3app_messages() is always a usabel pointer to a string array.
  • The memory needed to store the localized texts from the file is managed by the application object and freed automatically in dk3app_close(). You must not attempt to free the memory in the application code.
  • After dk3app_close() you must not attempt to use the msg array because the memory was released.
Diagnostics (reporting warnings, errors...)

How should I report warnings, errors...

If you use the DK libraries in your own programs, you should use the library diagnostics whenether possible for two reasons:

  • User experience
    For the same error condition the user expects to see the same error message.
  • Efforts for localization
    If you create a new text for an error condition with diagnostics already available, your new error message must be localized to all languages again.

The dk3app_log_1(), dk3app_log_3(), and dk3app_log_5() functions can be used for messages containing fixed texts and variable texts. The fixed texts are provided by a localized string table you have to specify.

The dk3app_log_i1(), dk3app_log_i3(), and dk3app_log_i5() functions can be used for messages containing fixed texts provided by the DK libraries and variable text.

Both function sets are documented in the documentation generated by doxygen, see documentation for the dk3app.h file.

Style

Which options should I implement?

  • -h
    --help
    to show help text.
  • -v
    --version
    to show version information.
  • --license
    to show license information.
  • -R
    --reset
    to ignore settings from configuration files and to return to built-in default settings.
  • -i encoding
    --input-encoding=encoding
    to specify the default input encoding when processing text files.
  • -o encoding
    --output-encoding=encoding
    to specify the default output encoding for programs producing text output.
  • -c
    --ignore-case
    for text processing programs to indicate "case-insensitive".
  • -e
    --echo
    for text processing programs to indicate that input text is specified as command line argument, not stdin or file.
  • -p
    --plain
    to indicate that standard input is 8-bit on Windows systems.
    As the recommended character size for "internationalized" programs on Windows systems is 16 bit our programs should produce 16-bit output and expect 16-bit input by default. For backward compatibility (i.e. someone might want to sort the output from an 8-bit program) we must have some way to switch back to 8-bit.
    On 8-bit-character systems the -p option should be ignored.

How should I name long options?

If configuration files may contain an equivalent to a long option, the long option name should be the configuration file entry name normalized using "-".
Example:
The long option for a configuration file entry

a very important setting        =       value

should be:

--a-very-important-setting=value
Dirk Krause
Schmalkalden (Germany), 2013-05-13
http://dktools.sourceforge.net