Finally, after a long waiting period, KBibTeX 0.9.2 got tagged, tar-balled, and released! It is a bug fix release, so virtually no new features since 0.9 got released; dependencies have not changed.

The highlights from the ChangeLog include:

Many thanks to the people who supported this release and make the continuous development of KBibTeX possible!

So far, most of my blog postings that appeared on Planet KDE were release announcements for KBibTeX. Still, I had always planned to write more about what happens on the development side of KBibTeX. Well, here comes my first try to shed light on KBibTeX's internal working …

Active development of KBibTeX happens in its master branch. There are other branches created from time to time, mostly for bug fixing, i. e. allowing bug reporters to compile and test a bug fix before before the change is merged into master or a release branch. Speaking of release branches, those get forked from master every one to three years. At the time of writing, the most recent release branch is kbibtex/0.9. Actual releases, including alpha or beta releases, are tagged on those release branches.

KBibTeX is developed on Linux; personally I use the master branch on Gentoo Linux and Arch Linux. KBibTeX compiles and runs on Windows with the help of Craft (master better than kbibtex/0.9). It is on my mental TODO list to configure a free Windows-based continuous integration service to build binary packages and installers for Windows; suggestions and support are welcome. Craft supports macOS, too, to some extend as well, so I gave KBibTeX a shot on this operating system (I happen to have access to an old Mac from time to time). Running Craft and installing packages caused some trouble, as macOS is the least tested platform for Craft. Also, it seems to be more difficult to find documentation on how to solve compilation or linking problems on macOS than it is for Windows (let alone Linux). However, with the help of the residents in #kde-craft and related IRC channels, I was eventually able to start compiling KBibTeX on macOS (big thanks!).

The main issue that came up when crafting KBibTeX on macOS was the problem of linking against ICU (International Components for Unicode). This library is shipped on macOS as it is used in many other projects, but seemingly even if you install Xcode, you don't get any headers or other development files. Installing a different ICU version via Craft doesn't seem to work either. However, I am no macOS expert, so I may have gotten the details wrong …

Discussing in Craft's IRC channel how to get KBibTeX installed on macOS despite its dependency on ICU, I got asked why KBibTeX needs to use ICU in the first place, given that Qt ships QTextCodec which covers most text encoding needs. My particular need is to transliterate a given Unicode text like ‘äåツ’ into a 7-bit ASCII representation. This is used among others to rewrite identifiers for BibTeX entries from whatever the user wrote or an imported BibTeX file contained to an as close as possible 7-bit ASCII representation (which is usually the lowest common denominator supported on all systems) in order to reduce issues if the file is fed into an ancient bibtex or shared with people using a different encoding or keyboard layout.

Such a transliteration is also useful in other scenarios such as if filenames are supposed to be based on a person's name but still must be transcribed into ASCII to be accessible on any filesystem and for any user irrespective of keyboard layout. For example, if a filename needs to have some resemblance the Scandinavian name ‘Ångström’, the name's transliteration could be ‘Angstrom’, thus a file could be named Angstrom.txt.

So, if ICU is not available, what are the alternatives? Before I adopted ICU for the transliteration task, I had used iconv. Now, my first plan to avoid hard-depending on ICU was to test for both ICU and iconv during the configuration phase (i. e. when cmake runs) and use ICU if available and fall back to iconv if no ICU was available. Depending on the chosen alternative, paths and defines (to enable or disable specific code via #ifdefs) were set.
See commit 2726f14ee9afd525c4b4998c2497ca34d30d4d9f for the implementation.

However, using iconv has some disadvantages which motivated my original move to ICU:

1. There are different iconv implementations out there and not all support transliteration.
2. The result of a transliteration may depend on the current locale. For example, ‘ä’ may get transliterated to either ‘a’ or ‘ae’.
3. Typical iconv implementations know less Unicode symbols than ICU. Results are acceptable for European or Latin-based scripts, but for everything else you far too often get ‘?’ back.

Is there a third option? Actually, yes. Qt's Unicode code supports only the first 2¹⁶ symbols anyway, so it is technically feasible to maintain a mapping from Unicode character (essentially a number between 0 and 65535) to a short ASCII string like AE for ‘Æ’ (0x00C6). This mapping can be built offline with the help of a small program that does link against ICU, queries this library for a transliteration for every Unicode code point from 0 to 65535, and prints out a C/C++ source code fragment containing the mapping (almost like in the good old days with X PixMaps). This source code fragment can be included into KBibTeX to enable transliteration without requiring/depending on either ICU or iconv on the machines where KBibTeX is compiled or run. Disadvantages include the need to drag along this mapping as well as to updated it from time to time in order to keep up with updates in ICU's own transliteration mappings.
See commit 82e15e3e2856317bde0471836143e6971ef260a9 where the mapping got introduced as the third option.

The solution I eventually settled with is to still test for ICU during the configuration phase and make use of it in KBibTeX as I did before. However, in case no ICU is available, the offline-generated mapping will be used to offer essentially the same functionality. Switching between both alternatives is a compile-time thing, both code paths are separated by #ifdefs.

Support for iconv has been dropped as it became the least complete solution (see commit 47485312293de32595146637c96784f83f01111e).

Now, how does this generated mapping look like? In order to minimize the data structure's size I came up with the following approach: First, there is a string called const char *unidecode_text that contains any occurring plain ASCII representation once, for example only one single a that can be used for ‘a’, ‘ä’, ‘å’, etc. This string is about 28800 characters long for 65536 Unicode code points where a code point's ASCII representation may be several characters long. So, quite efficient.

Second, there is an array const unsigned int unidecode_pos[] that holds a number for every of the 65536 Unicode code points. Each number contains both a position and a length telling which substring to extract from unidecode_text to get the ASCII representation. As the observed ASCII representations' lengths never exceed 31, the array's unsigned ints contain the representations' lengths in their lower (least significant) five bits, the remaining more significant bits contain the positions. For example, to get the ASCII representation for ‘Ä’, use the following approach:

const char16_t unicode = 0x00C4; ///< 'A' with two dots above (diaeresis)
const int pos = unidecode_pos[unicode] >> 5;
const int len = unidecode_pos[unicode] & 31;
const char *ascii = strndup(unidecode_text + pos, len);

If you want to create a QString object, use this instead of the last line above:

const QString ascii = QString::fromLatin1(unidecode_text + pos, len);

If you would go through this code step-by-step with a debugger, you would see that unidecode_pos[unicode] has value 876481 (this value may change if the generated source code changes). Thus, pos becomes 27390 and len becomes 1. Indeed and not surprisingly, in unidecode_text at this position is the character A. BTW, value 876481 is not just used for ‘Ä’, but also for ‘À’ or ‘Â’, for example.

Above solution can be easily adjusted to work with plain C99 or modern C++. It is in no way specific to Qt or KDE, so it should be possible to use it as a potential solution to musl (a libc implementation) to implement a //TRANSLIT feature in their iconv implementation (I have not checked their code if that is possible at all).

I am glad to announce the availability of KBibTeX 0.9 Beta 2 (0.8.91) for download. Whereas Beta 1 had some issues and was never formally announced, Beta 2 is quite stable and ready to use for everyone able to compile and install from a tar-ball and willing to test code.

The source code in tar-ball form is available at KDE's download infrastructure:
https://download.kde.org/unstable/KBibTeX/kbibtex-0.8.91.tar.xz
Signed with GnuPG:
https://download.kde.org/unstable/KBibTeX/kbibtex-0.8.91.tar.xz.asc (key: 1808CB466328F4380685A3B1A264FD738D861F41)

Unless there are severe issues, I plan to make a final release of 0.9 in early summer.

KBibTeX towards 0.10

The current master code contains a number of changes due for 0.10, such as refactored preferences/settings system and better integration to GitLab's CI system both to run automated tests as well as to run a Coverity Scan, among others. If you want to test some bleeding edge, pull the code from KDE's Git repository.

A common problem with bug reports received for KBibTeX is that the issue may already be fixed in the latest master in Git or that I can provide a fix which gets submitted to Git but then needs to be tested by the original bug reporter to verify that the issue has been indeed fixed for good.

For many distributions, no ‘Git builds’ are available (or the bug reporter does not know if they exist or how to get them installed) or the bug reporter does not know how to fetch the source code, compile it, and run KBibTeX, despite the (somewhat too technical) documentation.

Therefore, I wrote a Bash script called run-kbibtex.sh which performs all the necessary (well, most) steps to get from zero to a running KBibTeX. The nicest thing is that all files (cloned Git repo, compiled and installed KBibTeX) are placed inside /tmp which means no root or sudo are required, nor are any permanent modifications made to the user's system.

There is a README.txt file explaining the script in greater detail.

The only requirement is that the user installs the usual KDE-related development tools and libraries. If a tool or library is missing, the script will abort, but the error message (most likely some output from cmake) can be searched for in order to learn which package to install. Once this is done, simply restart run-kbibtex.sh until all steps succeed.

I have tested the script with several Linux distributions and gave earlier versions to bug reporters for testing, so I am almost sure that it will work as promised. Please send suggestions or bug reports via email to me.

As promised, here comes the one intermediate pre-release between KBibTeX 0.7.90 (0.8-beta1) and the final release of 0.8: KBibTeX 0.7.95 aka 0.8-rc1. The most important changes between Beta1 and RC1 are the following:

• Update of the ChangeLog (alternatively here) to reflect recent releases' changes,
• Removal of ISBNdb as it is no longer a free service, and
• Various translation updates.

The release is available on KDE's mirror infrastructure:

After a beta version in September and a release candidate in October, there is finally a release of KBibTeX 0.7.
A tag has been set and tar balls have been published.

The only changes compared to the release candidate are attempts to fix online search issues with Google Scholar and IEEE Xplore.

After some delay, I am finally pushing forward towards a final release of KBibTeX for KDE 4. The first step is the tagging and releasing of tar balls for version 0.7's Beta 1.

After quite some delay, I finally assembled a second release candidate for KBibTeX 0.6.1. Version 0.6.1 will be the last release in the 0.6.x series.

The following changes were applied since the release of 0.6:

Update: In an earlier version of this posting, the title called this release ‘0.6.1-alpha1’. Of course, it should have been ‘0.6.1-beta1’ as discussed in the text.

Quick update on the next bugfix release for the KBibTeX 0.6 series: KBibTeX 0.6.1-beta1 (0.6.0.90) has just been released. Differences to the alpha version from two weeks ago are two minor bug fixes and updated translations for Galician and Italian (thanks!).

In this posting, I am going to tell about the changes and development done in KBibTeX during the last few months. Most notably, KBibTeX has been ported to KDE Frameworks 5, but also some effort has been spent into code quality.

Port to Frameworks

Since Spring 2015, I have been working on a KDE Frameworks 5 (KF5) port of KBibTeX. In August, this KF5 port has become master and the last KDE 4 version resides in branch kbibtex/0.7 which is still due to be released. Except for the major refactoring for KF5, both master and kbibtex/0.7 are still quite similar.

The KF5 version is stable enough for (my) daily work and from bug reports I know that some people are already using it. For everyone who tries to run a Frameworks/Plasma 5-exclusive environment, I can recommend this version.

For both kbibtex/0.7 and master (which would become version 0.8) I plan to make releases really soon^TM. Some known bugs need to be addressed and then building of tar balls for alpha/beta/rc releases can start. Some fixes will be ported to kbibtex/0.6 as well and there will be a minor bugfix release for 0.6 as well.
Now, if those names like kbibtex/0.6 or master have been confusing, please see my posting from June as well as the development documentation.

Code Quality

The other large focus next to the KDE Frameworks 5 port was code quality using code analysis tools. In particular, I am employing three tools:

• Coverity Scan is a commercial tool, but offers to scan open source projects for free. Very useful at identifying dead code, uninitialized variables, bad memory management. It even found a copy&paste errors where variables were renamed inconsistently. Some hard-to-trace crashes have been identified as well. This tool can be strongly recommended. Usage requires to compile your code with a special wrapper provided by Coverity that generates a considerable amount of data that needs to be packed and uploaded to Coverity for analysis (automation for CI possible).
• Krazy 2 through the English Breakfast Network. This tool I have been using the longest. From what I understand, most recommendations are more about ‘best practice’ than finding ‘real’ bugs. Still very useful and no overhead for a developer as EBN does make automated Git checkouts to run Krazy.
• Clazy is the new kid on the block. It makes use of LLVM/Clang and checks code for proper use of Qt-specific features such as efficient construction of strings without temporary instances. Maybe of less usefulness for non-Qt projects, but very helpful to Qt/KDE projects. It requires a bleeding-edge LLVM and there are only few Linux distributions (e. g. ArchLinux) that ship such a version by default at the time of writing.

I don't recall seeing any overlap in identified issues between those three tools, so for best results in your code, you should use all three.

By now virtually all issues and warnings found by those three tools have been fixed in KBibTeX, at least in the master branch. Some fixes have been backported to older branches. I recommend to check out the Git logs, as messages state which code checker a commit/change was due.

DBus

One major feature currently under development and one of the largest external code contributions in recent times is DBus support by Shunsuke Shimizu. With ‘support’ I mean that you can remotely control KBibTeX via DBus. As of now, basic operations such as opening, saving, and closing files as well as inserting elements exist. The code resides in a special branch (branch bugs/kde332380 in repository clones/kbibtex/thomasfischer/kbibtex.git) for the bug report originally requesting DBus support. An example Python 3 script demonstrates how to control KBibTeX through DBus.