Page 1 of 1

XnView MP crashing PC with low virtual memory warning

Posted: Sat Jun 03, 2017 3:39 am
by Optic
Hi there,

I'm using XnView MP 0.86 on Windows 10 on a Dell XPS 15 (9550) laptop (4K display) with 16 GB RAM and a 500 GB SSD (Samsung PM951)

I am using XnView MP as a image organiser and really love the fast thumbnail browsing and ability to generate thumbnails in cache manually. :)

My images are TIFF (LZW compressed), RGB24 - created by FFmpeg and range from 2-5 MB each. So the image folders are very large, e.g. one folder contains 141,516 images and the manually rebuild thumbnails for that folder is 1.58 GiB using Lossy - High quality (WebP) with 640x360 as thumbnail size.

I have set: Memory usage for database engine to 1,000MiB

I am running into an issue where lately XnView MP will cause the system to become very slow and unresponsive and eventually crash the machine (automatically restarts). It looks like XnView MP is loading all the thumbnails into memory as the progress bar at the bottom right corner progresses, but it doesn't usually reach 100% (most of the time I don't stay inside the folder long enough) but the longer I stay in the folder the higher the chance the PC will crash.

I check Event Viewer and see this:

Code: Select all

Log Name:      System
Source:        Microsoft-Windows-Resource-Exhaustion-Detector
Date:          19/05/2017 12:03:56 AM
Event ID:      2004
Task Category: Resource Exhaustion Diagnosis Events
Level:         Warning
Keywords:      Events related to exhaustion of system commit limit (virtual memory).
User:          SYSTEM
Computer:      ***
Description:
Windows successfully diagnosed a low virtual memory condition. The following programs consumed the most virtual memory: xnviewmp.exe (224) consumed 63196856320 bytes, OneDrive.exe (9160) consumed 568487936 bytes, and MsMpEng.exe (4116) consumed 185860096 bytes.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Resource-Exhaustion-Detector" Guid="{9988748E-C2E8-4054-85F6-0C3E1CAD2470}" />
    <EventID>2004</EventID>
    <Version>0</Version>
    <Level>3</Level>
    <Task>3</Task>
    <Opcode>33</Opcode>
    <Keywords>0x8000000020000000</Keywords>
    <TimeCreated SystemTime="2017-05-18T16:03:56.619437400Z" />
    <EventRecordID>5642</EventRecordID>
    <Correlation ActivityID="{DB8646C6-D86A-491F-AA75-4BFAC533D264}" />
    <Execution ProcessID="5700" ThreadID="8660" />
    <Channel>System</Channel>
    <Computer>***</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <UserData>
    <MemoryExhaustionInfo xmlns="http://www.microsoft.com/Windows/Resource/Exhaustion/Detector/Events">
      <SystemInfo>
        <SystemCommitLimit>68564443136</SystemCommitLimit>
        <SystemCommitCharge>68518096896</SystemCommitCharge>
        <ProcessCommitCharge>66270928896</ProcessCommitCharge>
        <PagedPoolUsage>386490368</PagedPoolUsage>
        <PhysicalMemorySize>17024835584</PhysicalMemorySize>
        <PhysicalMemoryUsage>16672030720</PhysicalMemoryUsage>
        <NonPagedPoolUsage>540282880</NonPagedPoolUsage>
        <Processes>200</Processes>
      </SystemInfo>
      <ProcessInfo>
        <Process_1>
          <Name>xnviewmp.exe</Name>
          <ID>224</ID>
          <CreationTime>2017-05-18T15:40:59.193485400Z</CreationTime>
          <CommitCharge>63196856320</CommitCharge>
          <HandleCount>549</HandleCount>
          <Version>0.86.0.0</Version>
          <TypeInfo>201</TypeInfo>
        </Process_1>
        <Process_2>
          <Name>OneDrive.exe</Name>
          <ID>9160</ID>
          <CreationTime>2017-05-18T11:20:05.555147400Z</CreationTime>
          <CommitCharge>568487936</CommitCharge>
          <HandleCount>740</HandleCount>
          <Version>17.3.6816.313</Version>
          <TypeInfo>210</TypeInfo>
        </Process_2>
        <Process_3>
          <Name>MsMpEng.exe</Name>
          <ID>4116</ID>
          <CreationTime>2017-05-18T11:19:21.610625400Z</CreationTime>
          <CommitCharge>185860096</CommitCharge>
          <HandleCount>1061</HandleCount>
          <Version>0.0.0.0</Version>
          <TypeInfo>67</TypeInfo>
        </Process_3>
        <Process_4>
          <Name>NVIDIA Share.exe</Name>
          <ID>12332</ID>
          <CreationTime>2017-05-18T11:20:08.148171200Z</CreationTime>
          <CommitCharge>184401920</CommitCharge>
          <HandleCount>655</HandleCount>
          <Version>56.2924.1538.1</Version>
          <TypeInfo>152</TypeInfo>
        </Process_4>
        <Process_5>
          <Name>
          </Name>
          <ID>0</ID>
          <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime>
          <CommitCharge>0</CommitCharge>
          <HandleCount>0</HandleCount>
          <Version>0.0.0.0</Version>
          <TypeInfo>0</TypeInfo>
        </Process_5>
        <Process_6>
          <Name>
          </Name>
          <ID>0</ID>
          <CreationTime>1601-01-01T00:00:00.000000000Z</CreationTime>
          <CommitCharge>0</CommitCharge>
          <HandleCount>0</HandleCount>
          <Version>0.0.0.0</Version>
          <TypeInfo>0</TypeInfo>
        </Process_6>
      </ProcessInfo>
      <PagedPoolInfo>
        <Tag_1>
          <Name>CM31</Name>
          <PoolUsed>145498112</PoolUsed>
        </Tag_1>
        <Tag_2>
          <Name>MmSt</Name>
          <PoolUsed>46099840</PoolUsed>
        </Tag_2>
        <Tag_3>
          <Name>CMAl</Name>
          <PoolUsed>27881472</PoolUsed>
        </Tag_3>
      </PagedPoolInfo>
      <NonPagedPoolInfo>
        <Tag_1>
          <Name>ismc</Name>
          <PoolUsed>134455296</PoolUsed>
        </Tag_1>
        <Tag_2>
          <Name>Rtu5</Name>
          <PoolUsed>85784288</PoolUsed>
        </Tag_2>
        <Tag_3>
          <Name>smNp</Name>
          <PoolUsed>52932608</PoolUsed>
        </Tag_3>
      </NonPagedPoolInfo>
      <ExhaustionEventInfo>
        <Time>2017-05-18T16:03:54.651940500Z</Time>
      </ExhaustionEventInfo>
    </MemoryExhaustionInfo>
  </UserData>
</Event>
Followed by:
Application popup: Windows - Out of Virtual Memory : Your system is low on virtual memory. To ensure that Windows runs properly, increase the size of your virtual memory paging file. For more information, see Help.

Then it restarts with this in the log (Kernel-Power):
The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.


Any suggestions?

I would like to keep my original files as TIFF for archive purposes.

I can add more RAM to my laptop to make it 32 GB - would that help?

Re: XnView MP crashing PC with low virtual memory warning

Posted: Sat Jun 03, 2017 1:31 pm
by Xnout!
Don't add more RAM, try this first (not sure if instructions equate to Win10, but you can look for equivalencies as you go along):
Microsoft Community wrote:If you receive warnings that your virtual memory is low, you'll need to increase the minimum size of your paging file. Windows sets the initial minimum size of the paging file at the amount of random access memory (RAM) installed on your computer plus 300 megabytes (MB), and the maximum size at 3 times the amount of RAM installed on your computer. If you see warnings at these recommended levels, then increase the minimum and maximum sizes.

1. Open System by clicking the Start button, clicking Control Panel, clicking System and Maintenance, and then clicking System.

2. In the left pane, click Advanced system settings. If you are prompted for an administrator password or confirmation, type the password or provide confirmation.

3. On the Advanced tab, under Performance, click Settings.

4. Click the Advanced tab, and then, under Virtual memory, click Change.

5. Clear the Automatically manage paging file size for all drives check box.

6. Under Drive [Volume Label], click the drive that contains the paging file you want to change.

7. Click Custom size, type a new size in megabytes in the Initial size (MB) or Maximum size (MB) box, click Set, and then click OK.

Increases in size usually don't require a restart, but if you decrease the size, you'll need to restart your computer for the changes to take effect. We recommend that you don't disable or delete the paging file.

For more information, please refer the link mentioned below:

http://windows.microsoft.com/en-US/wind ... ual-memory]
There is also the "flush memory" shortcut trick. Google it. If you have trouble finding it let me know (flushes RAM).

If those things only work for a short while then you are likely doing a lot of serious, intensive work, not only with XnView
but a mix of memory intensive programs, no? In that case, upgrading RAM is a possibility.

Re: XnView MP crashing PC with low virtual memory warning

Posted: Sun Jun 04, 2017 1:25 pm
by Optic
Thanks for your reply Xnout,

I think the issue is resolved now as I just unticked the following setting in XNView MP:
Browser > Thumbnail > Create thumbnails for whole folder

And now the green progress bar loads very fast now and RAM usage is now consistent, not maxing out!

So maybe XnView MP was trying to re-generate thumbnails on the fly (which exhausted my RAM and pagefile because I had so many images in the folder) even when thumbnails were already rebuild manually in Catalog settings?

Re: XnView MP crashing PC with low virtual memory warning

Posted: Tue Jun 06, 2017 2:22 am
by Optic
Hi there,

I had a look at:
http://newsgroup.xnview.com/viewtopic.php?t=21149

And just realised setting Create thumbnails for whole folder is Lazy Load behaviour. With it ticked, when all thumbnails are loaded in memory then browsing thumbnails is instant..

With it off, the speed is still fast but not instant.

My question now is:
Why is XnView MP exhausting all my memory with option Create thumbnails for whole folder enabled - when the folder pre-generated thumbnails in Catalog settings reports only 1.58 GiB (141,516 images, 640x360 - Lossy - High quality (WebP))

Shouldn't XnView MP only be using 1.58 GiB maximum for this folder, when it loads the pre-generated thumbnails into memory?

Re: XnView MP crashing PC with low virtual memory warning

Posted: Wed Jun 07, 2017 7:49 am
by xnview
Optic wrote: Shouldn't XnView MP only be using 1.58 GiB maximum for this folder, when it loads the pre-generated thumbnails into memory?
No, the catalog use compress image, thumbnails are decompressed in ram

Re: XnView MP crashing PC with low virtual memory warning

Posted: Thu Jun 08, 2017 6:53 am
by Optic
Thanks Pierre - that makes sense now.

So then the only way to reduce memory usage when thumbnails are decompressed (loaded) into memory is to reduce the thumbnail size. I will try reducing thumbnail size from 640x360 to 320x180 and use sharpen thumbnail option..

I had it to this size because it looked really nice on 4k retina screen and thought the compressed version is loaded in memory.

Thanks for your help,

Re: XnView MP crashing PC with low virtual memory warning

Posted: Sun Jun 11, 2017 10:03 pm
by mudo_j
Hello!

I might be having the same issue (Windows 7, 16 GB RAM). When the program is building thumbnails for hundreds of images, with "Create thumbnails for whole folder" enabled, the value of Private Bytes jumps up to 16 GB, while other memory-related parameters look nothing special.

Image

Also, when I am exporting jpeg images from my raw editors (LightZone, RawTherapee) to the folder, displayed in XnView MP, every added picture rises the Private Bytes usage for about 4 GB. When 16 GB are reached, Windows starts to complain, raw editor sometimes crashes etc. XnView MP releases the memory, if I switch the folder.

I can reproduce this on two Windows 7 machines, with 16 and 8 GB of RAM.
XnView MP 0.84, 0.85, 0.86, 64-bit, portable or installed.

Have a nice day!