Thomas Fischer

KBibTeX 0.10-alpha2 aka 0.9.81

2020-08-05T15:32:47Z

The release of KBibTeX 0.10 is approaching with the release of KBibTeX 0.10-alpha2 aka 0.9.81.

Improvements and changes since 0.9.2 are less on the user interface, but on the code behind it:

( Read more... )

Many thanks go to contributors such as bug reporters, translators, and everyone else who contributed to KBibTeX!

Source code is available as tar ball, cryptographically signed using GnuPG key 1808CB466328F4380685A3B1A264FD738D861F41.

comments

KBibTeX 0.9.2

2020-04-27T11:37:48Z

Finally, after a long waiting period, KBibTeX 0.9.2 got tagged, tar-balled, and released! It is a bug fix release, so virtually no new features since 0.9 got released; dependencies have not changed.

The highlights from the ChangeLog include:

( Read more... )

Many thanks to the people who supported this release and make the continuous development of KBibTeX possible!

comments

Rudimentary KBibTeX client using Kirigami2

2019-10-13T14:08:13Z

KBibTeX is a bibliography editor (BibTeX and somewhat BibLaTex) used in conjunction with LaTeX and friends. Based on this code base, a SailfishOS client called ‘BibSearch’ exists which allows to search for bibliographic data in various online sources (IEEE Xplore, Google Scholar, ACM Digital Library, …). BibSearch's code makes use of KBibTeX's C++ code, has its user interface implemented in SailfishOS's Silica QML, and provides just two C++ files on its own to glue together everything.

It has long been my goal to provide a QML or Kirigami-based client of KBibTeX of similar functionality, either to be used as Plasma widget or for Plasma Mobile (thus equivalent to the SailfishOS implementation). Finally I have been able to put together an initial implementation using Kirigami2. There are several rough edges and bugs, and much of the intended functionality is missing, but at least one can search for bibliographic data. If you are familiar with Kirigami, it should be a low-hanging fruit to fix the obvious issues, especially given that there is already a Silica-based client as reference. Code contributions to branch feature/kirigami are welcome!

comments

Follow-up on ‘ASCII Transliteration without ICU or iconv’

2019-10-13T13:51:24Z

In a recent blog posting I presented some code on how to transliterate common Unicode characters into ASCII-only representations using a offline-generated lookup table to avoid dependencies on ICU which would normally do this job.

By an anonymous commenter, I got pointed to that Unicode (in Qt) is slightly more complicated than I had considered when writing the code: I missed to handle planes beyond the Basic Multilingual Plane (BMP) and the ‘surrogates’ between code points 0xD800 and 0xDFFF. In a series of recently pushed Git commits I addressed problem of surrogates and fixed some more issues. Some preparatory work has been done to support more planes in the future, but as of now, only the BMP is supported. For details, please have a look at the five commits posted on 2019-10-12.

comments

ASCII Transliteration without ICU or iconv

2019-08-09T20:44:59Z

So far, most of my blog postings that appeared on Planet KDE were release announcements for KBibTeX. Still, I had always planned to write more about what happens on the development side of KBibTeX. Well, here comes my first try to shed light on KBibTeX's internal working …

Active development of KBibTeX happens in its master branch. There are other branches created from time to time, mostly for bug fixing, i. e. allowing bug reporters to compile and test a bug fix before before the change is merged into master or a release branch. Speaking of release branches, those get forked from master every one to three years. At the time of writing, the most recent release branch is kbibtex/0.9. Actual releases, including alpha or beta releases, are tagged on those release branches.

KBibTeX is developed on Linux; personally I use the master branch on Gentoo Linux and Arch Linux. KBibTeX compiles and runs on Windows with the help of Craft (master better than kbibtex/0.9). It is on my mental TODO list to configure a free Windows-based continuous integration service to build binary packages and installers for Windows; suggestions and support are welcome. Craft supports macOS, too, to some extend as well, so I gave KBibTeX a shot on this operating system (I happen to have access to an old Mac from time to time). Running Craft and installing packages caused some trouble, as macOS is the least tested platform for Craft. Also, it seems to be more difficult to find documentation on how to solve compilation or linking problems on macOS than it is for Windows (let alone Linux). However, with the help of the residents in #kde-craft and related IRC channels, I was eventually able to start compiling KBibTeX on macOS (big thanks!).

The main issue that came up when crafting KBibTeX on macOS was the problem of linking against ICU (International Components for Unicode). This library is shipped on macOS as it is used in many other projects, but seemingly even if you install Xcode, you don't get any headers or other development files. Installing a different ICU version via Craft doesn't seem to work either. However, I am no macOS expert, so I may have gotten the details wrong …

Discussing in Craft's IRC channel how to get KBibTeX installed on macOS despite its dependency on ICU, I got asked why KBibTeX needs to use ICU in the first place, given that Qt ships QTextCodec which covers most text encoding needs. My particular need is to transliterate a given Unicode text like ‘äåツ’ into a 7-bit ASCII representation. This is used among others to rewrite identifiers for BibTeX entries from whatever the user wrote or an imported BibTeX file contained to an as close as possible 7-bit ASCII representation (which is usually the lowest common denominator supported on all systems) in order to reduce issues if the file is fed into an ancient bibtex or shared with people using a different encoding or keyboard layout.

Such a transliteration is also useful in other scenarios such as if filenames are supposed to be based on a person's name but still must be transcribed into ASCII to be accessible on any filesystem and for any user irrespective of keyboard layout. For example, if a filename needs to have some resemblance the Scandinavian name ‘Ångström’, the name's transliteration could be ‘Angstrom’, thus a file could be named Angstrom.txt.

So, if ICU is not available, what are the alternatives? Before I adopted ICU for the transliteration task, I had used iconv. Now, my first plan to avoid hard-depending on ICU was to test for both ICU and iconv during the configuration phase (i. e. when cmake runs) and use ICU if available and fall back to iconv if no ICU was available. Depending on the chosen alternative, paths and defines (to enable or disable specific code via #ifdefs) were set.
See commit 2726f14ee9afd525c4b4998c2497ca34d30d4d9f for the implementation.

However, using iconv has some disadvantages which motivated my original move to ICU:

There are different iconv implementations out there and not all support transliteration.
The result of a transliteration may depend on the current locale. For example, ‘ä’ may get transliterated to either ‘a’ or ‘ae’.
Typical iconv implementations know less Unicode symbols than ICU. Results are acceptable for European or Latin-based scripts, but for everything else you far too often get ‘?’ back.

Is there a third option? Actually, yes. Qt's Unicode code supports only the first 2¹⁶ symbols anyway, so it is technically feasible to maintain a mapping from Unicode character (essentially a number between 0 and 65535) to a short ASCII string like AE for ‘Æ’ (0x00C6). This mapping can be built offline with the help of a small program that does link against ICU, queries this library for a transliteration for every Unicode code point from 0 to 65535, and prints out a C/C++ source code fragment containing the mapping (almost like in the good old days with X PixMaps). This source code fragment can be included into KBibTeX to enable transliteration without requiring/depending on either ICU or iconv on the machines where KBibTeX is compiled or run. Disadvantages include the need to drag along this mapping as well as to updated it from time to time in order to keep up with updates in ICU's own transliteration mappings.
See commit 82e15e3e2856317bde0471836143e6971ef260a9 where the mapping got introduced as the third option.

The solution I eventually settled with is to still test for ICU during the configuration phase and make use of it in KBibTeX as I did before. However, in case no ICU is available, the offline-generated mapping will be used to offer essentially the same functionality. Switching between both alternatives is a compile-time thing, both code paths are separated by #ifdefs.

Support for iconv has been dropped as it became the least complete solution (see commit 47485312293de32595146637c96784f83f01111e).

Now, how does this generated mapping look like? In order to minimize the data structure's size I came up with the following approach: First, there is a string called const char *unidecode_text that contains any occurring plain ASCII representation once, for example only one single a that can be used for ‘a’, ‘ä’, ‘å’, etc. This string is about 28800 characters long for 65536 Unicode code points where a code point's ASCII representation may be several characters long. So, quite efficient.

Second, there is an array const unsigned int unidecode_pos[] that holds a number for every of the 65536 Unicode code points. Each number contains both a position and a length telling which substring to extract from unidecode_text to get the ASCII representation. As the observed ASCII representations' lengths never exceed 31, the array's unsigned ints contain the representations' lengths in their lower (least significant) five bits, the remaining more significant bits contain the positions. For example, to get the ASCII representation for ‘Ä’, use the following approach:

const char16_t unicode = 0x00C4; ///< 'A' with two dots above (diaeresis)
const int pos = unidecode_pos[unicode] >> 5;
const int len = unidecode_pos[unicode] & 31;
const char *ascii = strndup(unidecode_text + pos, len);

If you want to create a QString object, use this instead of the last line above:

const QString ascii = QString::fromLatin1(unidecode_text + pos, len);

If you would go through this code step-by-step with a debugger, you would see that unidecode_pos[unicode] has value 876481 (this value may change if the generated source code changes). Thus, pos becomes 27390 and len becomes 1. Indeed and not surprisingly, in unidecode_text at this position is the character A. BTW, value 876481 is not just used for ‘Ä’, but also for ‘À’ or ‘Â’, for example.

Above solution can be easily adjusted to work with plain C99 or modern C++. It is in no way specific to Qt or KDE, so it should be possible to use it as a potential solution to musl (a libc implementation) to implement a //TRANSLIT feature in their iconv implementation (I have not checked their code if that is possible at all).

comments

KBibTeX 0.9 released

2019-06-16T20:02:43Z

Finally, KBibTeX 0.9 got released. Virtually nothing has changed since the release of beta 2 in May as no specific bugs have been reported. Thus ChangeLog is still the same and the details on the changes since 0.8.2 as shown on the release announcement for 0.9-beta2.

Grab the sources at KDE's download mirrors.

comments

KBibTeX 0.9-beta2

2019-05-02T20:22:00Z

I am glad to announce the availability of KBibTeX 0.9 Beta 2 (0.8.91) for download. Whereas Beta 1 had some issues and was never formally announced, Beta 2 is quite stable and ready to use for everyone able to compile and install from a tar-ball and willing to test code.

( Read more... )

The source code in tar-ball form is available at KDE's download infrastructure:
https://download.kde.org/unstable/KBibTeX/kbibtex-0.8.91.tar.xz
Signed with GnuPG:
https://download.kde.org/unstable/KBibTeX/kbibtex-0.8.91.tar.xz.asc (key: 1808CB466328F4380685A3B1A264FD738D861F41)

Unless there are severe issues, I plan to make a final release of 0.9 in early summer.

KBibTeX towards 0.10

The current master code contains a number of changes due for 0.10, such as refactored preferences/settings system and better integration to GitLab's CI system both to run automated tests as well as to run a Coverity Scan, among others. If you want to test some bleeding edge, pull the code from KDE's Git repository.

comments

KBibTeX 0.8.2

2019-01-04T21:03:20Z

KBibTeX 0.8.2 has been released as a bug fix release on 0.8.1. It fixes a number of known and reported bugs (see below) as well as some other minor issues. Online search engines that were broken should work now again.

( Read more... )

comments

Running KBibTeX from Git repository has become easier

2018-11-17T20:37:04Z

A common problem with bug reports received for KBibTeX is that the issue may already be fixed in the latest master in Git or that I can provide a fix which gets submitted to Git but then needs to be tested by the original bug reporter to verify that the issue has been indeed fixed for good.

For many distributions, no ‘Git builds’ are available (or the bug reporter does not know if they exist or how to get them installed) or the bug reporter does not know how to fetch the source code, compile it, and run KBibTeX, despite the (somewhat too technical) documentation.

Therefore, I wrote a Bash script called run-kbibtex.sh which performs all the necessary (well, most) steps to get from zero to a running KBibTeX. The nicest thing is that all files (cloned Git repo, compiled and installed KBibTeX) are placed inside /tmp which means no root or sudo are required, nor are any permanent modifications made to the user's system.

There is a README.txt file explaining the script in greater detail.

The only requirement is that the user installs the usual KDE-related development tools and libraries. If a tool or library is missing, the script will abort, but the error message (most likely some output from cmake) can be searched for in order to learn which package to install. Once this is done, simply restart run-kbibtex.sh until all steps succeed.

I have tested the script with several Linux distributions and gave earlier versions to bug reporters for testing, so I am almost sure that it will work as promised. Please send suggestions or bug reports via email to me.

comments

Finally: First stable release of KBibTeX for KDE Frameworks 5

2018-06-23T19:53:38Z

After almost exactly two years of being work-in-progress, the first stable release of KBibTeX for KDE Frameworks 5 has been published! You can grab the sources at your local KDE mirror. Some distributions like ArchLinux already ship binary packages.

( Read more... )

comments

KBibTeX 0.7.95 aka 0.8-rc1

2018-05-28T17:38:18Z

As promised, here comes the one intermediate pre-release between KBibTeX 0.7.90 (0.8-beta1) and the final release of 0.8: KBibTeX 0.7.95 aka 0.8-rc1. The most important changes between Beta1 and RC1 are the following:

Update of the ChangeLog (alternatively here) to reflect recent releases' changes,
Removal of ISBNdb as it is no longer a free service, and
Various translation updates.

The release is available on KDE's mirror infrastructure:

( Read more... )

comments

KBibTeX 0.7.90 a.k.a. 0.8-beta1: On the Road to KDE Frameworks 5

2018-05-05T18:56:40Z

Finally, the release of KBibTeX 0.8 is on its track again. I tagged (Phabricator) and tar-balled the code of the current Git branch kbibtex/0.8 (Phabricator) as KBibTeX 0.7.90 (a. k. a. 0.8-beta1) and asked the KDE sysadmins to put it on KDE's content distribution network.

Only afterwards I noticed that I totally had forgotten to update the ChangeLog which was still stuck on the ancient release of 0.6.1. Properly updating the changelog records will be my next step. In case I did't mention it before, the biggest change from 0.7 to 0.8 is the migration from KDE4 to KDE Frameworks 5. User interface and functionality has stayed surprisingly stable, though.

For you, my fellow KBibTeX users, you may fetch the tar ball kbibtex-0.7.90.tar.xz and test if everything is working for you. This request is especially relevant if you are a translator or a package maintainer or at least know a little bit of either to see if translations and/or package building works on your setup. If you find any issues, please report them at KDE's Bugtracking System (don't forget to set the version in your report to 0.8). There are known bugs in the code, some of which I fixed in the master branch (to become 0.9) already but did not integrate into 0.8 due to the feature freeze.

This release contains code contributions from, among others, Antonio Rojas, Frederik Schwarzer, Joao Carreira and Pino Toscano. Thank you very much!

My preliminary and optimistic time plan predicts a stable, final release of KBibTeX 0.8 at the end of May (this year). There'll may be some more pre-releases in between in case relevant issues were found and fixed.

Looking forward to your feedback!

comments

KBibTeX 0.7 (the final version for KDE4)

2018-01-02T20:39:21Z

After a beta version in September and a release candidate in October, there is finally a release of KBibTeX 0.7.
A tag has been set and tar balls have been published.

The only changes compared to the release candidate are attempts to fix online search issues with Google Scholar and IEEE Xplore.

( Read more... )

comments

KBibTeX 0.7-rc1 (0.6.95)

2017-10-28T19:31:19Z

As no new bug reports came in for KBibTeX 0.7-beta1 which got released in early September and only very few changes got applied since then (the ChangeLog is virtually unchanged), it is now time to release KBibTeX 0.7-rc1 (0.6.95). A tag has been set and tar balls have been published. Unless there is a showstopper bug or real-life™ interference, expect a final release in November.

( Read more... )

comments

KBibTeX 0.7-beta1

2017-09-08T20:16:46Z

After some delay, I am finally pushing forward towards a final release of KBibTeX for KDE 4. The first step is the tagging and releasing of tar balls for version 0.7's Beta 1.

( Read more... )

comments

KBibTeX 0.6.2 released

2017-03-19T21:23:25Z

Today, KBibTeX 0.6.2 has been released. It replaces the never published 0.6.1 release, where a show-stopping bug was detected after tagging the release.

tl;dr: source tar ball, ChangeLog in Git

( Read more... )

comments

KBibTeX 0.6.1-rc2 released

2017-01-16T22:04:25Z

After quite some delay, I finally assembled a second release candidate for KBibTeX 0.6.1. Version 0.6.1 will be the last release in the 0.6.x series.

The following changes were applied since the release of 0.6:

( Read more to learn which changes were applied )

comments

KBibTeX 0.6.1-rc1 (0.6.0.95)

2016-08-26T19:48:38Z

After some time of activity on KBibTeX's master branch, I finally returned to the stable branches to push forwards some releases.

( Read more... )

comments

KBibTeX 0.6.1-beta1 (0.6.0.90)

2016-05-05T19:06:17Z

Update: In an earlier version of this posting, the title called this release ‘0.6.1-alpha1’. Of course, it should have been ‘0.6.1-beta1’ as discussed in the text.

Quick update on the next bugfix release for the KBibTeX 0.6 series: KBibTeX 0.6.1-beta1 (0.6.0.90) has just been released. Differences to the alpha version from two weeks ago are two minor bug fixes and updated translations for Galician and Italian (thanks!).

( Read more... )

comments

KDiff3 for KDE Frameworks 5 becomes usable

2016-04-20T18:21:36Z

After recently starting to port KDiff3 to KDE Frameworks 5, I made a few commits today making the software actually usable.

Commit 468652ce70b1214842c passes command line arguments, most importantly filenames of files to diff and merge, to the inner classes which do the actual work. The old code uses KDE 4's KCmdLineArgs which provided static functions to retrieve command line arguments from anywhere in the code. The new code processes command line arguments using QCommandLineParser in the main function and then passes this object down into inner classes. This makes the code working although it may not be the best design (I may consider a refactoring in the future).

comments

KBibTeX 0.6.1-alpha1 (0.6.0.80)

2016-04-20T18:12:17Z

I just made a release of KBibTeX 0.6.1-alpha1 (0.6.0.80), which is the first preview release of the upcoming bugfix release in the 0.6 series. Please note that the 0.6 series is still based on KDE 4.

If you are a distribution packager, please check if you can download, build, and install the sources and that KBibTeX starts and runs without any obvious bugs or crashes. If you are a translator (GUI or documentation), please check if translations are complete, correct, and appear in the right places.

ChangeLog

KDE Bug 351455: Removing soversion from KBibTeX Part
KDE Bug 353898: Fixing build issues on ARM architecture
KDE Bug 354785: Using QTextDocument/QTextEdit instead of WebKit/WebEngine: more lightweight and supported on all platforms
Correcting choke on PubMed searches to 10 seconds
Setting foreground color of colored rows to either black or white for better readability
Disabling OCLC WorldCat (request for support denied by this organization)
Generally improved code quality as detected by code checkers such as Clazy or Coverity
Fixing handling of URLs and their protocols for local files
Fixing setting default id suggestion
Adding 'Keywords' field to .desktop file
Removing file that was licensed under CC BY-NC, but never got installed
Updating translations
Other minor backports from master. See git log v0.6..v0.6.1-alpha1 or git diff v0.6..v0.6.1-alpha1 for details.

Tar balls

Tar balls of this release are available at http://download.gna.org/kbibtex/0.6/, more specifically:

The used GnuPG key is 0xA264FD738D861F41.

Future Releases

Once release 0.6.1 got published without problems, I plan to focus on releasing 0.7 which will be the last series based on KDE 4. Most likely, there will be a few bugfix releases in the next years as long as there are users still using KDE 4.

Following a successful release of KBibTeX 0.7, version 0.8 will be the first stable series building on KDE Frameworks 5.

Both code bases for 0.7 (in Git kbibtex/0.7) and 0.8 (master as of now) are quite stable, i. e. it should be a matter of weeks instead of months to get both releases out of the door. Both 0.7 and 0.8 will be quite similar featurewise, the main differences are due to the porting from KDE 4 to KDE Frameworks 5. If you want to run more current code than the 0.6 series offers, please fetch the sources and test for yourself.

comments

KDiff3 Port to KDE Frameworks 5 Started

2016-03-25T21:43:04Z

I am a regular user of KDiff3, a two-way or three-way merging tool for KDE. The tool is very useful to quickly compare and merge source code and LaTeX files or to resolve .rpmnew/.pacsave files.

Unfortunately, there has been very little activity in this project (last commit September 2014) and it is still stuck on Qt4/KDE4. The original maintainer, Joachim Eibl, has unfortunately no time left to develop this project.
To keep this very useful tool alive, I stepped forward and ported the code base in a few evenings from Qt4/KDE4 to Qt5/Frameworks 5. Well, the port is not complete, I have to admit …

( Read more... )

comments

Updates on KBibTeX

2016-01-07T21:42:09Z

In this posting, I am going to tell about the changes and development done in KBibTeX during the last few months. Most notably, KBibTeX has been ported to KDE Frameworks 5, but also some effort has been spent into code quality.

Port to Frameworks

Since Spring 2015, I have been working on a KDE Frameworks 5 (KF5) port of KBibTeX. In August, this KF5 port has become master and the last KDE 4 version resides in branch kbibtex/0.7 which is still due to be released. Except for the major refactoring for KF5, both master and kbibtex/0.7 are still quite similar.

The KF5 version is stable enough for (my) daily work and from bug reports I know that some people are already using it. For everyone who tries to run a Frameworks/Plasma 5-exclusive environment, I can recommend this version.

For both kbibtex/0.7 and master (which would become version 0.8) I plan to make releases really soon^TM. Some known bugs need to be addressed and then building of tar balls for alpha/beta/rc releases can start. Some fixes will be ported to kbibtex/0.6 as well and there will be a minor bugfix release for 0.6 as well.
Now, if those names like kbibtex/0.6 or master have been confusing, please see my posting from June as well as the development documentation.

Code Quality

The other large focus next to the KDE Frameworks 5 port was code quality using code analysis tools. In particular, I am employing three tools:

Coverity Scan is a commercial tool, but offers to scan open source projects for free. Very useful at identifying dead code, uninitialized variables, bad memory management. It even found a copy&paste errors where variables were renamed inconsistently. Some hard-to-trace crashes have been identified as well. This tool can be strongly recommended. Usage requires to compile your code with a special wrapper provided by Coverity that generates a considerable amount of data that needs to be packed and uploaded to Coverity for analysis (automation for CI possible).
Krazy 2 through the English Breakfast Network. This tool I have been using the longest. From what I understand, most recommendations are more about ‘best practice’ than finding ‘real’ bugs. Still very useful and no overhead for a developer as EBN does make automated Git checkouts to run Krazy.
Clazy is the new kid on the block. It makes use of LLVM/Clang and checks code for proper use of Qt-specific features such as efficient construction of strings without temporary instances. Maybe of less usefulness for non-Qt projects, but very helpful to Qt/KDE projects. It requires a bleeding-edge LLVM and there are only few Linux distributions (e. g. ArchLinux) that ship such a version by default at the time of writing.

I don't recall seeing any overlap in identified issues between those three tools, so for best results in your code, you should use all three.

By now virtually all issues and warnings found by those three tools have been fixed in KBibTeX, at least in the master branch. Some fixes have been backported to older branches. I recommend to check out the Git logs, as messages state which code checker a commit/change was due.

DBus

One major feature currently under development and one of the largest external code contributions in recent times is DBus support by Shunsuke Shimizu. With ‘support’ I mean that you can remotely control KBibTeX via DBus. As of now, basic operations such as opening, saving, and closing files as well as inserting elements exist. The code resides in a special branch (branch bugs/kde332380 in repository clones/kbibtex/thomasfischer/kbibtex.git) for the bug report originally requesting DBus support. An example Python 3 script demonstrates how to control KBibTeX through DBus.

comments

KBibTeX 0.5.2 and 0.6 Released

2015-08-09T19:46:19Z

The following two relases of KBibTeX are now available:

( Read more... )

comments

KBibTeX 0.5.2-beta1 and 0.6-beta1

2015-06-19T17:43:29Z

The following two pre-relases of KBibTeX are now available:

( Read more... )

comments