Out of memory when recursive viewing 20.000+ videos

Ask for help and post your question on how to use XnView MP.

Moderators: XnTriq, xnview

User avatar
xnview
Author of XnView
Posts: 29581
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Out of memory when recursive viewing 20.000+ videos

Post by xnview » Tue Nov 08, 2016 8:52 am

jadO wrote: yes it's normal if you have 'create for whole folder'
Why is this normal? I don't see a reason for loading all thumbnails until everything crashes.
But please explain me why you can't disabled 'create for whole folder' setting?
Edit: Also my findings showed, that simple scrolling through a large number of thumbnails increases the memory dramatically. Memory management seems to be absent.
There should be strict memory control. I only see 2 reasons to load a thumbnail: When it gets displayed or processed. When it does not get displayed anymore its memory should get freed and after it has been processed it should free its memory too.
On old computer, this can be slow, but this can be an enhancement when 'create for whole folder' setting is OFF
Pierre.

jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Fri Nov 11, 2016 9:02 am

It seems like the only working solution is as follows:

1. Open folder
2. Eventually select recursive folder option
3. Select all files
4. Select rebuilt thumbnails from the menu

This seems to go through all files, creates thumbnails for them and keeps a low memory footprint about 300 mb in my case.

Basically this is what I would have expected from the 'create for whole folder' setting.
I still don't see a point in loading as many thumbnails to crash XnView MP.

User avatar
xnview
Author of XnView
Posts: 29581
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Out of memory when recursive viewing 20.000+ videos

Post by xnview » Fri Nov 11, 2016 12:38 pm

jadO wrote: 4. Select rebuilt thumbnails from the menu
This seems to go through all files, creates thumbnails for them and keeps a low memory footprint about 300 mb in my case.
'create for whole folder' is OFF?
Pierre.

jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Fri Nov 11, 2016 1:52 pm

xnview wrote:'create for whole folder' is OFF?
Yes, but I don't know if the result is what I want.

I had it running now for a couple of hours and it seemed like it was creating thumgbnails for each files. But now when I scrolled through them I see a lot of maybe most of them not being created.

I don't know what this thing is doing.

Kaminari
Posts: 7
Joined: Wed Sep 07, 2016 9:50 pm

Re: Out of memory when recursive viewing 20.000+ videos

Post by Kaminari » Mon Nov 14, 2016 9:17 pm

helmut wrote:I don't know anyone except you who has 20.000 or more video files on his computer. So I still don't see that this is a standard use case. What's is your use case to scroll through 20.000 or more videos?
The memory leak problem doesn't only affect video files, but also pictures.

I work with professional photographers and one of my folders contains more than 50.000 files (let me spell that: FIFTY THOUSANDS), not even counting subfolders. That's a pretty much standard use case for my job.

Recursive thumbnail creation litterally kills the OS. On the fly ("on demand") thumbnail creation is slow and for my needs is not an option. Creating thousands of thumbnails in one pass doesn't necessarily mean having to store all of them simultaneously in RAM. Obviously there should be a better way to implement such a feature. Directory Opus can do it without hogging the whole memory.

User avatar
xnview
Author of XnView
Posts: 29581
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Out of memory when recursive viewing 20.000+ videos

Post by xnview » Tue Nov 15, 2016 3:26 pm

Kaminari wrote:
helmut wrote:I don't know anyone except you who has 20.000 or more video files on his computer. So I still don't see that this is a standard use case. What's is your use case to scroll through 20.000 or more videos?
The memory leak problem doesn't only affect video files, but also pictures.
Memory leak or out of memory?
On the fly ("on demand") thumbnail creation is slow and for my needs is not an option.
Even if thumbnails is stored in catalog?
Pierre.

jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Wed Nov 30, 2016 9:44 pm

Kaminari wrote:The memory leak problem doesn't only affect video files, but also pictures.

I work with professional photographers and one of my folders contains more than 50.000 files (let me spell that: FIFTY THOUSANDS), not even counting subfolders. That's a pretty much standard use case for my job.

Recursive thumbnail creation litterally kills the OS. On the fly ("on demand") thumbnail creation is slow and for my needs is not an option. Creating thousands of thumbnails in one pass doesn't necessarily mean having to store all of them simultaneously in RAM. Obviously there should be a better way to implement such a feature. Directory Opus can do it without hogging the whole memory.
Thank you for your confirmation of this issue.


jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Wed Jan 18, 2017 8:46 pm



jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Wed Jan 18, 2017 11:05 pm

XnTriq wrote::-( And the final release, too?
Yes.

User avatar
XnTriq
Moderator & Librarian
Posts: 5238
Joined: Sun Sep 25, 2005 3:00 am
Location: Ref Desk

Re: Out of memory when recursive viewing 20.000+ videos

Post by XnTriq » Wed Jan 18, 2017 11:30 pm

jadO wrote:
XnTriq wrote::-( And the final release, too?
Yes.
OK. Thanks for reporting back, jadO.

jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Thu Jan 19, 2017 5:48 am

XnTriq wrote:
jadO wrote:
XnTriq wrote::-( And the final release, too?
Yes.
OK. Thanks for reporting back, jadO.
No problem. I hope my answer was clear that the issue persists even in 0.84 RC. The links you have posted seem to address the very same issue.

User avatar
xnview
Author of XnView
Posts: 29581
Joined: Mon Oct 13, 2003 7:31 am
Location: France
Contact:

Re: Out of memory when recursive viewing 20.000+ videos

Post by xnview » Thu Jan 19, 2017 9:39 am

jadO wrote:
XnTriq wrote::-( And the final release, too?
Yes.
it's only for create catalog in settings
Pierre.

jadO
Posts: 338
Joined: Wed Apr 29, 2015 6:36 am

Re: Out of memory when recursive viewing 20.000+ videos

Post by jadO » Fri Jan 20, 2017 6:44 pm

xnview wrote:
jadO wrote:
XnTriq wrote::-( And the final release, too?
Yes.
it's only for create catalog in settings
I don't understand what you are trying to say.

Post Reply