• KOffice Developer Meeting (pictures).
• Akademy time! One Kexi dev was available for hugs^wdiscussion; chatted a bit with the ownCloud hackers about exposing Kexi databases though it, KDevelop guys about injecting Kexi database plugin in a form of KDevelop's "Database view" and mobile guys about possible options for Kexi Mobile.
• http://userbase.kde.org/Kexi updated. New tutorial added: Kexi Reports for Beginners
• Adam works on porting the remaining Q/K3 code in Kexi, starting with the Project Navigator. Ported now.
• The works on Kexi 2.2 Handbook started within the KDE Userbase wiki: http://userbase.kde.org/Kexi/Handbook
• Kexi now supports multipage (large up to 64 KB) memo values when importing data from MS Access. And it's the only open source app doing it!
• Some work on Predicate, database connectivity and creation library, a new iteration of KexiDB library developed within Kexi (Jaros?aw). Expect a more in August.

Kexi devlog based on identi.ca notes:

• Kexi is apparently ready for 2.2 rc1 release. Combobox and tab widgets in forms fixed!
• Fresh logo inserted on the current kexi web site http://kexi-project.org - these pages really need web admins and artists.
• The odyssey from Kexi 1.1 to 2.2 took 36 months.
• Kexi/KOffice 2.2.0 packages for Debian
• Kexi 2.2 is here! With KOffice 2.2. Thanks for your support and we're asking for more!

  

No, we do not skip March, for Kexi it was just too silent month to blog about it alone Here we go (based on identi.ca notes):

• Adam now understands the koodf library, exporting a report from Kexi into a spreadsheet can be done 'the right way'
• New address of the Kexi Fan Club on Facebook: http://www.facebook.com/kexi.project

• Follow-up on the SQLite the secure delete thing: Oh boy, the guys rock - they have implemented all my requests for 3.6.23. We're going to recommend this version at least.
• Autofield widget and layouts will be disabled in Kexi 2.2
• Fixed visibility of form widget properties in the designer; that was really obscure bug
• Kexi 2.2 beta 2 arrives in 2 days with KOffice 2.2 beta 2, please test! Yes because of quantum leap there's no Kexi 2.0 nor 2.1 in the wild.
• Adam just begun work on reports for Kexi 2.3 even before 2.2 is out the door! Supporting plugins and a cleaned up core, so hopefully less bugs (April 4).
• Adam just added image loading from the database for reports. Closing a bug with 1 line of code.
   
  
• KOffice Developers Sprint 2010 is coming
• Kexi is in bug fixing mode in preparation for 2.2. Please try out the betas and let us know what you think on bugs.kde.org or #kexi on freenode (April 19)
• We've just started development of Kexi Version 2.3 Alpha 1 (a part of KOffice 2.3 Alpha 1)! This is in parallel to perparing 2.2 series.
• Predicate rather won't be used in Kexi 2.3; doing that is too complex so we keep KexiDB until 2.4 release
• Group boxes and tab widgets fixed in Kexi forms. Last fixes for 2.2.
• Optimized display of scaled image entries in table view. Scrolling is smooth for large images too and scaling is smooth.
   
  
• And last but not least: much of the work is possible because of quality bug reporting, thanks for that and we're asking for more These weeks special thanks go to George Goldberg!

(brought to you by Adam and myself)

What's new, based on identica notes:

• Thoughts on deploying SQLite turned out to be work in progress. Valuable input from our distro friends. A special ">wiki page has been created. See the last item of this entry
• KoReport, Kexi's rpt backend has undergone some refactoring to make it more generic.
• Fix number 1, charts working again in the designer.
  
• Working through the huge list of issues krazy has found with the kexi codebase, not glamorous, but necessary!
• Refactored the report renderers, and given better class name, to make the report library suitable for adoption over all of KOffice.
• Do you want well working MSAccess to Kexi converter? It's up to you - request it!
• KOffice now has a suite-wide reporting library! (placed in koffice/libs/koreports/)
• Kexi switched to "system" SQLite again. We want to play with distros well, but will check SQLite features at compilation to maintain high quality standards.

(brought to you by Adam and myself)

"There's nothing easier" -- you say -- about packaging and deploying SQLite. "Just take the software with default settings and package as a shared lib plus SQLite shell".

It's not that simple.

The SQLite project is developed at impressive speed assuming complexity of the software. It's already part of many operating systems like OS X and Symbian. Linuxes use it somewhat at system level. Browsers use SQLite for storage via HTML 5, earlier via Google Gears.

The fact is that all of the uses we can spot are for a specific cases. For each case slightly different configuration is beneficial. SQLite has two kinds of configurable options: runtime and compile-time. The latter includes configurable limits. SQLite deployment is typical to embedded software, which is efficient but requires developers to be aware the specifics.

Google Gears expected SQLite to have certain features enabled. This is also the case with Kexi or in general any app that uses SQLite for desktop databases. As you can read in the backlog of Kexi development for January, I have switched to system SQLite. I even proposed update to FindSqlite.cmake to make sure minimum version with enough features is in place. That lasted just one day.

Then I have immediately switched back. There were a few subtle and one main reason - security. SQLite provides one nice compile-time option:

SQLITE_SECURE_DELETE

    This compile-time option causes SQLite to overwrite deleted information
    with zeros in addition to marking the space as available for reuse.
    Without this option, deleted data might be recoverable from a database
    using a binary editor. However, there is a performance penalty for using this option.

I have assumed that we want this flag to be on, so one important workflow in Kexi is more secure. When you delete tables or even just table rows (records) from a database, and send the .kexi file (which is based on SQLite database) to others, you basically expect not to have the deleted information in the file. Unless SQLITE_SECURE_DELETE is enabled, this is not the case. Databases, includeing SQLite, like to just mark the deleted records are deleted without removing the empty space or cleaning up the bytes.

Enabling the feature at the cost efficiency is the current design decision. An alternative to SQLITE_SECURE_DELETE would be to vacuum the database on closing. But what if the application was terminated uncleanly (application or system crash)? And what if user sends the database by email while Kexi is still running? These question do not exist when SQLITE_SECURE_DELETE is on, and that's why I like the flag.

SQLITE_SECURE_DELETE is not always needed however, for example when you share the data through a web server or remote connections. But these use cases are not yet supported by a stable implementation in Kexi.

All in all, the current set of options for Kexi's copy of SQLite is as follows (kexidb/drivers/sqlite/CMakeLists.txt) -- look to just get an idea of possible future features of Kexi and Predicate library (KexiDB 2):

ADD_DEFINITIONS(
    # sqlite compile-time options, http://sqlite.org/compile.html
    -DSQLITE_SECURE_DELETE
    -DSQLITE_ENABLE_COLUMN_METADATA # Some additional APIs that provide convenient access to meta-data
                                    # about tables and queries
    -DSQLITE_ENABLE_FTS3 # Version 3 of the full-text search engine
    -DSQLITE_ENABLE_FTS3_PARENTHESIS # Modifies the query pattern parser in FTS3 such that it supports
                                     # operators AND and NOT (in addition to the usual OR and NEAR)
                                     # and also allows query expressions to contain nested parenthesesis.
    -DSQLITE_ENABLE_MEMORY_MANAGEMENT # Extra logic to SQLite that allows it to release unused memory upon request
    -DSQLITE_ENABLE_RTREE # Support for the R*Tree index extension
    -DSQLITE_ENABLE_STAT2 # Additional logic to the ANALYZE command and to the query planner that can help SQLite
                          # to chose a better query plan under certain situations
    -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT # Optional ORDER BY and LIMIT clause on UPDATE and DELETE statements
    -DSQLITE_ENABLE_UNLOCK_NOTIFY # Enables the sqlite3_unlock_notify() interface and its associated functionality
                                  # (http://sqlite.org/unlock_notify.html)
    -DSQLITE_SOUNDEX # Enables the soundex() SQL function (http://sqlite.org/lang_corefunc.html#soundex)
)

As mentioned in the reviewboard comment even while SQLite is not packable for general use as a shared library, this is by design. Because there are many compile-time switches, so many not-fully compatible versions of SQLite can be found in particular distros.

By having own copy of regularly updated SQLite, whas has been started in 2004, we can also patch SQLite to add some esoteric features, e.g. provide progress information of the .dump operation, so we can have display the progress in the GUI, what is good for large files.

I am also thinking about related proposal: having a copy of SQLite moved from Kexi into some place like kdesupport, with sane build defaults. With the lib name altered to something like libksqlite to avoid clashes with distro-packaged SQLite. Then something like simple FindKSqlite, would be used within KDE.

Do you have any opinions on the matter? Please share it in the comments below.

With 2010 we've started to employ identica (then connected to Twitter and Facebook) as an channel for our live changelog at the {power}user level. Here's the dump for the past ~30 days (oh I should have used an XSLT).

• We're replacing serialized QFont attrs with ODF equivalents in Kexi Reports file format; e.g. fo:font-family; it's extension of OpenRPT
• Finally we're still embedding SQLite as many options are not set in distros, e.g. SECURE DELETE should be the default http://bit.ly/amZfJ3

• Kexi switched to using sqlite bundled with the operating system instead of using a fork. The newest stable version is always recommended.
• Sidebars behave now as expected: when collapsed, vertical buttons appear like in Kate. It's now natural to put widgets palette as a sidebar.
• An extra ko dev has taken an interest in the report backend used in kexi (koreport), working to making it usable in kplato, a win allround!
• We have line style combo box in the Property Editor. Today high precision point type also added too. Good for accurate printouts (reports).
• Just ported color selector for the property editor. Now we're defaulting to KDE's Oxygen palette instead of the old school VGA colors.
• Exporting in action! http://www.piggz.co.uk/kexi/kexi-export-1.ogv
• MS Access 2010 drops support for many formats: Paradox <=7, Lotus 1-2-3, Access 1/2. But 64-bit ver not even planned. http://bit.ly/5xZZV7
• Is zoomable table view reasonable idea for you? http://bit.ly/8AWp6c Added as todo just yesterday for Kexi 2.6 http://bit.ly/64C4jb
• MS discovered SVG? http://bit.ly/76lrNr Kexi just switched to svg ns for XML tags in Kexi Reports format-took 2 hours! http://bit.ly/6uD7dy
• All the most ugly painting glitches now fixed in Kexi Table View. So time-consuming but pays-off...
• Some rendering issues in the Table View widget fixed now. Looks like the Table View will stay Qt3-based until Kexi 2.3 - porting takes time.

Many small fixes are a building block of the Kexi porting effort - the goal is joining the KOffice 2.2. Many of the fixes and refactoring is related to forms. Much more left and we're scheduling works on crazy features even up to Kexi 2.6 already.

The next is porting task, I hope, is the table view port to QScrollArea from QScrollView, what's rather demanding task to me at least since while the table view can be called widget itself, it is the biggest one (# of SLOC) of those I have touched directly so far in KDE (at least it bundles a dedicated model/view framework developed long before the first announcement of Qt4 Model View API).

Sometimes one needs to add some procrastination, and this time I added an indicator/emblem-like icon to forms so you can easily notice that what widgets are bound to data source (and to which).

Kexi Forms: Data source emblems for data-aware widgets; click to enlarge

Other small improvement that you can find in above screenshot is frame-around-labels in the design view. Before that, we had to live with borderless labels, like in Designer. As Kexi generally lowers barriers for new users, we display the frame more like in case of a text frame in a word processor.

In related news, I am also excited that Nuno has admitted that one of the forthcoming items on his list are Oxygen icons specific to Kexi.

Speaking of procrastination, we (me and Nuno and Hugo Pereira) just yesterday had a small session of three graphics-loving guys regarding some bits of Oxygen style. Two outcomes of that are:

1. Idea for cleaning up vertical lines for the case when docks+tabwidget+splitter+another frame is used in an application. This is the case for Kexi as well everywhere in KOffice and most complex apps featuring sidebars. I bet the idea is not new and the improvement is pretty much demanded. Two proposals (less and more extreme):

(compare these improvements with the current look presented earlier.

2. Exclusive QGroupButton styling for Oxygen; my early proposals, need fixes but the 3rd is somewhat acceptable. The mockups use Kexi's view switchers as a base.

On the technical side, I think the look can be either set default or can be supported via K* overload of QGroupButton or addition to KStyle or by setting a Qt4's dynamic property (so developer can decide to alter the look only for some group buttons). Whatever we recommend, it could find its way to the HIGs first.

There's also one note that we agree on, that since the Oxygen now employs fade in-out when you switch between tabs in QTabWidget, this effect should be measured and switched off when there are too many delays. The goal is not only to support slower machines or conserve power but also cases when there are large Konqueror's (or Kexi's, which is indeed my test case) tab areas much harder to fade in-out than small ones. So there could be some logic employed depending on the size of the tab widget.

As Maemo Summit 2009 starts in a few hours. While I am not there, for me one of the most interesting parts is the Handheld Glom: Easy database applications presentation. Glom is a desktop database developed by GNOME friends using gtkmm (C++). Originally bound directly to PostgreSQL, recently (early 2009) has gained SQLite file database support (default engine in Kexi since 2004). That was a must I guess if someone wants to cover needs of mobile devices, so imagine how easier it is going to share data files between various apps one day. While Glom offers somewhat simpler feature set than than Kexi, a gnomedb library db layer has been also developed in the meantime, having partially similar goals as Predicate library in KDE.

Perhaps because the future version of maemo (Harmattan) migrates to Qt from from GTK+, there is also recent development of a simple mobile Qt GUI for glom, called called qlom (notice the "q" ). Here come my thoughts on Kexi mobile port, perhaps let's say Kexi Mobile, i.e. simplified Kexi GUI aimed at small form factor, probably lacking complex parts like Form Designer. Such project becomes easier these days with new Qt ports (maemo, S60, Windows CE) and new more powerful hardware from various vendors. As much as such development would be a great distractor during the current development (aimed at releasing stable 2.2 version after absence of 2.0 and 2.1), I of course encourage you (yes, you...) to play with the idea and contribute with the mobile GUI. Here's the recipe: join the friendly KOffice developers at #koffice or #kexi on the FreeNode IRC or write to KOffice development mailing list, ask, ask and ask, then release early, and release often . Let's see what happens...

While explaining the story behind his great ppttoxml tool, Jos also mentioned

Since about a year, Microsoft has, after significant political pressure, put documentation for their file formats on-line.

That's fine and solved some issues. But there are MS Access proprietary file formats (mdb, accdb) that remain to be secret. These are not planned to be replaced by XML formats (what would be overkill in databases). I guess there was no pressure to open the formats, what looks like an overlook in EU and the USA (correct me if there's another reason like patents). If you google for that, it is hard to find even a single mention of file format specifications in the above meaning, and even explanations from MS employees or backers show that they do not fully realize one thinf: MSA formats are not covered by the process of said "opening of the legacy formats".

MS Access formats are currently only accessible (I mean openly accessible, i.e. via 100% Free Software code, not via MS ADO dlls, used for instance by the oo.org plugin, what excludes non-Windows systems) through the mdbtools library project or its descendants like the Kexi's MDB plugin, that contains some improvements for mdbtools. The mdbtools project was a huge effort of reverse-engineering mdb formats (that are nasty direct binary dumps of some initialized and uninitialized chunks of memory owned by MSA, more strictly MS Jet db engine). The project now faces stagnation but it is possible to extend it unless MSA moves to entirely different storage format.

So unfortunately MSA file formats are not quite fair play game in the small but important branch of desktop databases. While anti-competitive behaviour is a valuable corporate weapon, this neglected area contradicts the recent buzz related to publishing various document format specifications. By still using the mdb formats in 2009, you may not only have troubles with not owning your software written with MSA (forms, reports, code...), but also with not owning your data. That is why I emphasize importancy of the issue, even if MSA is on its decline after we moved to the web era, and after MSA broke compatibility in 2003 version and also recently in 2007 and 2010 (but sure, it makes rather good consulting business, as any mess in IT).

To solve the issue once and in a definite way I'd like to hear any feedback from MS.

PS: On the Kexi side, due to our desire for releasing only stable software, we're in the way to KOffice 2.2 (i.e. 2010), so there will be no version 2.0 and 2.1. A lot of things have been ported to KDE 4, including forms, and there shall be more database and file formats handled, there are new features, e.g. reports.

A post on the KDE forum motivated me to write some info about what's new with Kexi 2.0. For sanity I groupped a set of quickies as an 2.0 Alpha 12 changelog. For uninformed, most applications within KOffice will meet the stable 2.0 release (already passed a promising RC1 stage), while Kexi and Kivio neds more time for development. The hope is to synchronize nicely at the 2.1 stage.

"The main highlights behind Alpha 12 is a completely new Report Designer, and Property Editor, both based on new Qt 4's facilities. Report Designer is a creative fork of OpenRPT libraries, and already makes up a rich reporting tool that well integrates with Kexi principles. Property Editor is a component used by any designer in Kexi, i.e. currently Table, Query, Form, Report and Script Designer. The new quality of the Property Editor will hopefully improve user experience.
As a true Free and Open Source Software project, Kexi has enjoyed more new developments such as Oracle and ODBC drivers." (more)

Forms
Forms have started to work again about February, after quite huge refactoring stage. I am now close to fulfilling the promise that there will be no unneeded abstraction layer that allows to embed the Forms Designer within 3rd-party applications (there were attempts to do so in KDE3's KDevelop and there was a standalone KFormDesigner in KDE 3, mostly a main window on top of the framework). Currently, less abstractions leads to simpler code, even if some other kind of abstraction will go when we'll want to share e.gour redo/undo command classes between forms and reports. Qt Designer's code is LGPL now, so people willing to embed its property editor have more chances than we had in Kexi had, in about 2003.

Kexi Forms (Oxygen Style, Windows) - click to enlarge

Property Editor
During development of the new property editor and the abstract property framework "KoProperty2", I have encountered a number of places where some new ideas can be applied or just the code refactored. As an example of small functional changes, a new "revert" button appeared that undoes changes made to a single property. In KoProperty1 (Kexi 1.x) and Qt Designer appears on the right hand of the property value, in KoProperty2 appears on the right hand of the property name.

Property Editor 2 with "revert" actions exposed

Qt Designer's property editor with "revert" button

This lets us avoid issues when the editor is getting narrower because of the undo button, what is the case of Qt Designer's property editor.
Moreover, the new way of displaying if the button is visually similar to how "clear" action is displayed in KLineEdit, so I hope you'll like it.

Widgets palette
Widgets palette went temporarily to a custom tabbed menu's tab (see the Form Designer's screen shot above), but it is possible that the palette will find its way within a context side bar. Whether it will be the left hand (used also by Project Navigator now) or the right hand (used by the Property Pane) -- is not a closed question. Perhaps there are more points in favor of the first option, because the Property Pane is usually used in parallel with the widgets palette -- if so, then it is better to provide a way for having both visible.

The third, easiest to implement -- and typically not too much thought out -- option is the current temporary Widgets tab. It takes a lot of space: vertical space is too small to fit even all the standard widgets we have now; and I did not even count actions for form designing, such as tab order setting, layouts, alignment... (see the screnshot again). If these went to an Edit tab, we would have (again) a problem with users forced to switch between two tabs too frequently.

Tabbed toolbar
I am BTW also somehow interested on how (and where) will the Korat UI go; looking forward to see possibilities of reusing it for Kexi. Well, "interested" is one thing, but it is also easy to get excited about one thing: how common in FOSS it is to discover that an independent development started for a feature you have put on your TODO stack months before
Currently for Kexi we are using rather simple QTabWidget, and just adding some dynamism from time to time in an evolutionary way, as there are still more urgent TODO items than the main window.

New year means some snow and cold noses here in Warsaw, and a new job to me, this time in the mobile industry, what has rather diversified my day, and that's good. Happy 2009 to you, to your friends and family.

But I am on the KDE board too, it's not going to change. What's recently time-consuming for me is refactoring of the Native Kexi Forms. The most serious and anticipated decision is dropping the (implemented in 2004..2005) idea of the forms component reusable at a rich API level for other applications. The idea has introduced too man layers after months of development, too many to have things maintainable, with just proof-of-concept KFormDesigner being the only app using the framework except Kexi.

Since 2005 things have changed, Qt Designer gained its own reusable libraries. Before someone asks - we don't use them in Kexi Forms not just because that would affect the licensing (LGPL) or because of backward compatibility required by the current Forms' XML format (which is ~95% the same as Designer's but the 5% makes the difference). My point (or read it as feelings) is that while there would be no technical overhead in reusing Qt Designer, we would have overhead counted in man-months.

The first step, mostly finished now was to remove Qt3 Support dependencies. A bit late? True, but not too late. The code using the meta objects and properties is too fragile to receive massive changes in one go. Unlike KoProperty that received rather massive rewrite to Qt4's model/view API (I am not 100% happy with the results yet ) since last summer, Forms are more complex and I am employing strategy of incremental improvements for them. But after all, did I mention that continuous advances in many other parts of KOffice (if not just entire KDE 4) work as true motivation?

From KDE4-enterprise-functionality dept.: As already noted in the Commit Digest, another database driver (i.e. data provider) has been built on top of Kexi's database abstraction layer (KexiDB 2, codenamed Predicate). It is Sybase driver, also aimed at handling MS SQL Server connections.

Even better news is that these goodies come from the very newest Kexi developer, Sharan Rao, which I have met at Akademy 2007. Before he has been working on Umbrello UML Modeller, which is related to databases too.

As MS SQL Server is practically a fork of Sybase, it is accessible by rather similar connection protocol. Therefore both servers are handled by the same KexiDB driver. Its functionality, now in experimental stage, is implemented on top of so-called 'dblib' layer of the FreeTDS library, which is a result of reverse-engineering of the servers' closed TDS protocol.

KexiDBTest application's output - creation of a new database and test tables on a Sybase server (click to enlarge)

Successful result of the tests (click to enlarge)

One of the fundamental rules for development of Kexi and underlying frameworks is avoiding discrimination against various operating systems and hardware platforms. This is the reason for not using native Windows libraries. Like in the case of import function for MS Access files (mdb), where (unlike of the current OpenOffice.org Base fro Windows, which employs Windows-only library) we are using MDB Tools library, one that is portable to most operating systems and architectures.

Database created on the Sybase server and opened with the KexiDBTest application (click to enlarge)

The Sybase/MSSQL driver has been rather developed for KexiDB/Predicate, not directly for Kexi itself, so we maintain ability of transparent access to the databases from any KDE 4 code. What's interesting, for convenience Sharan utilizes SQL Anywhere for his tests, i.e. an embedded Sybase version -- single-file-based, similar in terms of approach to already well-known SQLite database.

PS: Let me ask you - do not hesitate to contact us if you know someone with working Sybase and/or MSSQL installations that could be accessible remotely from time to time for the purpose of testing the driver.