MP 0.72 search fails if using image base path

Reported bugs that have been closed and/or resolved

Moderators: XnTriq, helmut, xnview, Dreamer

CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

MP 0.72 search fails if using image base path

Post by CameronD »

I had Settings->Catalog->Base path of your pictures assigned a non-default value of "G:\Photos".

If I perform a search for anything with "use catalog" enabled then the search returns empty if the search starting folder is under the base path.

If I search starting at G:\ then it fails to find anything.
However, if I search starting from "My Computer" then it finds lots of files listed under "@..." (which I presume is the base path indicator used in the DB).

Examples I tried:
Category: bird
IPTC:Caption: contains blue
XMP:Subject: contains bird

The IPTC and XMP searches return files if the use catalog option is disabled.
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

Workaround:
If, like me, you have many files already catalogued and you decide that the base path is not as useful as you thought, then it can be removed with a bit of SQL manipulation.
Initially, I had decided to use a base path to reduce problems when I migrate to a new PC, plus I had ideas about sharing it between different systems (but I have not started that yet, so there is no loss to me at the moment).
If you are sharing the DB and rely on base path then you can ignore this and wait for a fix.

The process should be reversible, except for a few very unusual situations.
  1. Open XnViewMP, go to Options -> Catalog and note down the "Base path" value. In my case it said "G:\photos\"
  2. Still in Options, under Integration->Paths. Click the button labelled "Open the Catalog Location"
  3. in XnViewMP, browse to a folder outside the base path - something like "C:\" should be safe. This is important to stop duplications in case you need to restart XnView.
  4. Go back to Options->Catalog and delete the contents of base path, save changes. Note: When you change the base path, XnView does not propagate these changes through the DB - you need to do it yourself. That is what this post is about.
  5. Shut XnViewMP down.
  6. From the Explorer window, Make a backup copy (or 2) of the DB file: XnView.db and leave this explorer window open in the background. If you are using non-default locations/names then you should know which file to use.
  7. Install SQLite Browser if you do not have it already. Run the program and open the database - XnView.db. You can copy back one of the backup copies if something goes wrong.
  8. Select the Browse Data tab and select Table:Folders from the drop-down list. Click on Pathname column to sort by that column. All entries starting with "@" are using a base directory specification, and these are what we need to fix.
  9. First, check that you do not have any folders that really have an '@' in them. Copy the following SQL command into the "Execute SQL" tab and press one of the "play" buttons. The box below should remain empty, and the status box below that should say "0 Rows returned ... " If there are some rows shown then I would suggest you do not continue, unless you can work out the appropriate sql yourself.

    Code: Select all

    select * from Folders where Pathname like "_%@%";
  10. Delete the first script, and copy/run the following sql, first replacing your directory base name where mine says "G:/photos/". Important: use '/' and not '\' as the folder separator, even on MS-Windows. Make sure you include the trailing '/'.

    Code: Select all

    update Folders set Pathname=replace(Pathname, "@", "G:/photos/") where Pathname like "@%";
    
  11. If you see the status UNIQUE constraint failed: Folders.Pathname: ... then it indicates that, like mine, your DB was a bit of a mess. I was able to identify where my experimentation had resulted in situations where the same path was present with the '@' form as well as the full name. Luckily, I was able to clean this up, by going back to XnView db management and deleting ones that I could tell were accidental duplications.
  12. Go back to the Browse Data tab and scroll through the lists, checking that the '@'s have gone and been replaced by correct paths. If this is the case then you can write changes to update the DB and try running XnView with the new DB.
Now that I look back at this list, I suspect that if you really need all this information then perhaps you should not try this. If you understand what I did then much of the detail is superfluous.
TechMich
Posts: 1
Joined: Mon Nov 26, 2018 9:53 pm

Re: MP 0.72 search fails if using image base path

Post by TechMich »

Hey,
i know the thread is older then three years now and the bug should be fixed in 0.82, but i have the same problem in 0.92. :?
Is there a workaround to keep the base path?
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

Hi,
the fact that this is still listed under "new" probably means the bug has not been addressed.
I have stopped using base path, so have not noticed it.

I guess the way to make search still work is to use "My Computer" as the root of the search - or whatever is the appropriate selection for your OS. However that might give you too many hits.

One experiment that might work to limit search to a subfolder - if you use the file browser to select where you want to start the search, then manually edit the text in the box to replace the base path folder with "@"

Cameron.
User avatar
michel038
XnThusiast
Posts: 1273
Joined: Tue Sep 27, 2016 8:18 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by michel038 »

I don't know if this bug was fixed, but there is a new bug in V0.92

Searches are ok when selecting categories in "categories filter" tab, but the option "use catalog" in search function gives always 0 files found, even if base path has default value.

see
viewtopic.php?f=62&t=38100
and
viewtopic.php?f=62&t=38152 (French : same problem for any criteria when "use catalog" is enabled)
User avatar
xnview
Author of XnView
Posts: 43326
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by xnview »

TechMich wrote: Mon Nov 26, 2018 10:03 pm Hey,
i know the thread is older then three years now and the bug should be fixed in 0.82, but i have the same problem in 0.92. :?
Is there a workaround to keep the base path?
When using 'base path' and searching at root path like d:\?
Pierre.
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

michel038 wrote: Wed Nov 28, 2018 1:16 pm I don't know if this bug was fixed, but there is a new bug in V0.92

Searches are ok when selecting categories in "categories filter" tab, but the option "use catalog" in search function gives always 0 files found, even if base path has default value.

see
viewtopic.php?f=62&t=38100
and
viewtopic.php?f=62&t=38152 (French : same problem for any criteria when "use catalog" is enabled)
And the presence of that bug (search using catalog always returns zero matches) makes it impossible to test the base-path bug. I was testing for words in keywords, iptc:captions or XMP:description.
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

I have just tested 0.93 beta1 that Pierre sent me.

Summary:
  1. The new bug reported by michel038 (nothing is found by search "using catalog") has been fixed
  2. The problem with base path still exists - with slight differences from my first report.
Edit: Update - I have also copied v0.93b1 over my main 0.92 installation, with the same results, akthough on that system I do not use a base path.

Details:
I used a laptop that had version 0.90 installed (x64 windows 7), and many folders that had not been scanned into the database.
  • I created a base path that was one folder down my tree : E:/Photos/Australia. None of these had already been scanned into the DB.
  • I used the settings->catalog->scan folder from my base path.
  • I used the optimise operation on the db to clean everything up.
  • I copied a file with a known uncommon caption from the newly scanned area to a folder outside the base path, and manually browsed there to get the DB updated.
  • search with "use catalog", starting at E:/Photos for an entry in the caption finds the one outside the base path, but not the one within the base path
  • search without "use catalog" finds both files as expected
  • the bug is repeated if I simply search for the file name (and the test runs a lot faster)
  • I then replaced the xnviewmp.exe with V0.93 beta 1 and the search results were essentially the same, with slight differences as noted below
Other aspects:
  • The folder entry UI has changed maybe between 0.90 and 0.93b1 to allow a "recently used folders" dropdown. This has changed things in two other ways:
    1. I can no longer select "My Computer" from the folder selection box but,
    2. I can type arbitrary text into the start folder location.
  • If I type "@" into the start folder location then it finds nothing
  • One of my folders under the base path is "Long_Trips". If I type "@Long_Trips" into the start folder location then the search with "use catalog" works as expected (almost)...
  • Searching from "@Long_Trips" for specific words in the caption returned 35 files. They are usually listed with the "@Long_Trips\" text at the start of the search result path, however for some of the files, the "@Long_Trips" is expanded to the full path. I cannot see the pattern - in 4 tests returning different numbers of files, the expansion occurred for 2/2 results, 8/8, 2/35 and 2/74.
  • In the case where all 8 images were named in full, I redid the search starting from "E:/Photos" and it found nothing when using the catalog.
User avatar
xnview
Author of XnView
Posts: 43326
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by xnview »

CameronD wrote: Mon Dec 03, 2018 2:09 am [*]I created a base path that was one folder down my tree : E:/Photos/Australia. None of these had already been scanned into the DB.
[*] search with "use catalog", starting at E:/Photos for an entry in the caption finds the one outside the base path, but not the one within the
Ok, right, mixing absolute & relative path has some problem on search
[*]The folder entry UI has changed maybe between 0.90 and 0.93b1 to allow a "recently used folders" dropdown. This has changed things in two other ways:
  1. I can no longer select "My Computer" from the folder selection box but,
  2. I can type arbitrary text into the start folder location.
I don't remember that it was possible
Pierre.
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

xnview wrote: Mon Dec 03, 2018 10:03 am
CameronD wrote: Mon Dec 03, 2018 2:09 am [*]I created a base path that was one folder down my tree : E:/Photos/Australia. None of these had already been scanned into the DB.
[*] search with "use catalog", starting at E:/Photos for an entry in the caption finds the one outside the base path, but not the one within the
Ok, right, mixing absolute & relative path has some problem on search
I assume this means you can now reproduce it under these circumstances.

I have always found it unavoidable that I will eventually browse images outside my main folder path, whether in %TMP% or on some network drive.
The catalog will always eventually have mixed and relative paths.
[*]The folder entry UI has changed maybe between 0.90 and 0.93b1 to allow a "recently used folders" dropdown. This has changed things in two other ways:
  1. I can no longer select "My Computer" from the folder selection box but,
  2. I can type arbitrary text into the start folder location.
I don't remember that it was possible
If you are referring to selecting "My Computer" - that was in my initial post from version 0.72. I have to say I cannot even remember it myself, but I am finding that happens a lot these days :( .
limar500
Posts: 10
Joined: Sat Sep 15, 2012 1:24 pm

Re: MP 0.72 search fails if using image base path

Post by limar500 »

michel038 wrote: Wed Nov 28, 2018 1:16 pm I don't know if this bug was fixed, but there is a new bug in V0.92

Searches are ok when selecting categories in "categories filter" tab, but the option "use catalog" in search function gives always 0 files found, even if base path has default value.

see
viewtopic.php?f=62&t=38100
and
viewtopic.php?f=62&t=38152 (French : same problem for any criteria when "use catalog" is enabled)
I'm having the same problem described by michel038 with V0.92 as well.
waiting for V0.93 to be out for this.
I have soooooo many files that without an index like "categories" it takes forever to find anything.
User avatar
xnview
Author of XnView
Posts: 43326
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by xnview »

See issue for current status and some details.
Pierre.
User avatar
xnview
Author of XnView
Posts: 43326
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by xnview »

This problem is supposed to be fixed in XnView MP 0.94. Please check and confirm the bug fix here.
Pierre.
CameronD
Posts: 308
Joined: Wed Aug 01, 2007 1:28 pm
Location: Australia

Re: MP 0.72 search fails if using image base path

Post by CameronD »

The good news is that I can no longer reproduce any of the previous problems.
I am not sure how thoroughly I could test things, since there are a large number of permutations, but the issues I reported seem to be fixed.

The not quite so good news is that there might have been another bug introduced - or it was always there but I never noticed it.
To reproduce,
* start with a clean DB, and default base path "/"
* view or scan some folders under the future base path, so the folders are recorded with a full pathname in the DB.
* create a base path
* use ''catalog->add folder" to scan a few more folders under the base path (just choose a small sample for speed, but include files that will match your search criterion)
* also manually view a few other folders with images that will satisfy your search
* then 'search->using catalogue and select "browse"
* search again and there will be duplicate entries, one with a full path and another entry with the "@" base path prefix.

Examining the folders table in the DB shows that both entries are in the DB, but the settings->catalogue cached directory list does not show both forms.

I think I can also reproduce it by simply manually viewing the previously scanned folders - a second copy then appears in the folders table in the DB.

One solution might be to check this when the user first enters a base path. It should be a simple matter to update the folder path names if the user approves. I recall in past discussions that this might be used in network-shared situations, so it would need a bit of caution.
User avatar
xnview
Author of XnView
Posts: 43326
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: MP 0.72 search fails if using image base path

Post by xnview »

CameronD wrote: Sat Sep 21, 2019 5:08 am The good news is that I can no longer reproduce any of the previous problems.
I am not sure how thoroughly I could test things, since there are a large number of permutations, but the issues I reported seem to be fixed.
this issue was in previous version too. 'base path' should not be changed after scanning folders.
Pierre.
Post Reply