Zionworx

If I do a keyword search on 'lord' - it finds songs with the word 'lords' in them, but if a song contains 'lord' but not 'lords' it gets missed.

So it does - an I have other examples of words that do not appear when searched for, so I thought I would do a little research. If I search for "lord" or "lords" I get 41 results on my copy of the database. Searching for "jesus" gives 863 results.

I opened the database in Turbo DB Viewer (search Google for this) and updated the full-text index (in viewer I opened MainTable, Table menu > Create full-text index). One of the questions asked during this process was what is the upper limit for keywords that occur too often. The Turbo DB Viewer set this to 1000 (note that this is greater than the 863 for "jesus"). I set this value to 9999 and build a new full-text index. I reopened Zionworx and now searched for "lord". I got 1286 results. I suspect that Zionworx has the same upper limit and this ought to be increased in the next update of the product - say maybe 2 or 3 thousand. There cannot be many words that turn up that often, but given that the database product does not index them rather than returning a "too many" error is not consistent with general users experience of search engines. It ought, but it would seem to be outside the scope of the database engine used, to say "here is 1000 results for 'lord', please redefine your query".

To finish my test, I rebuilt the index and stopped at 500 repeated words. This time a search for "jesus" found 1 result - a song with the word "jesus-" in it! So, a recommendation and a workaround.

Recommendation: Increase the search frequency count in the full-text index to a few thousand or add this to the .ini file to allow different churches to decided on own what they want (as increasing beyond current values might slow the indexing down on very large song databases with slow computers)

Workaround: Download Turbo DB Viewer and build your own index and not use the index updater in Zionworx. Zionworx comes with a file called KeywordExclude.txt which should be used during the index generation program (and of course check that this file does not contain the word "lord" as well).

A few additional notes to make on this subject - I have a two year old, fast laptop, and searching for "lord" is visibly slower than (say) "jesus". So the extra (in my case) 500 instances of the word have an impact.

And of course, when I suggest increasing the search frequency to 2000 or 3000 per word, I have a song database with 2500 songs and the results in the above post are the number of returned songs and not the number of times the word was found (which I think I might have implied).

A later test on the church computer - which is considerably slower than my laptop showed no difference (afaik) in searching for Lord once I rebuilt the index in Turbo DB Viewer.

But an interesting resutl turned up for combined searches. I have over 400 songs that contain the words "lord" and "jesus". So if I search for "lord jesus" I should get these 400 results - but I do not. With the default index setting which does not index words that are too frequent it removes "lord" from the index and so returns 15 results - this must be those 41 songs that are "lords" that also contain "jesus"!

So a cross posting in the Wishlist forum is needed as this really does need changing.

brainier wrote:Workaround: Download Turbo DB Viewer and build your own index and not use the index updater in Zionworx.

I follow this bit

brainier wrote:Zionworx comes with a file called KeywordExclude.txt which should be used during the index generation program (and of course check that this file does not contain the word "lord" as well).

Not sure how to execute this bit, how do you use this txt file during indexation?

During the index wizard steps (on the page which asks for the upper limit for frequency) you are asked for a meaningless words list. You enter (or browse to) the above file (in program files\zionworx) and then the index generation process will ignore words such as "the" which now might appear within your frequency paramaters, but need not be searched for.

The latest version of Zionworx (v2.6.52) fixes this issue. I can now search for "lord" and get 1295 results and "lords" and get a different number.

Upon installing the latest version you need to do an "Update full text index" from the Database menu before the new results count feature works.