Out of Memory error

[witness]

I keep getting out of memory errors when I try to build a patch or add a new version to my project. I have 4GB of RAM, plenty of extra hard drive space and no other apps running that consume large amounts of memory. When I look in the Processes tab in my task manager, it shows multiple versions of VP20Design.exe running even though I only have the one instance actually open. When I add up all the other apps running, they are using about 400 MB of RAM. VP20Design.exe is pulling between 300 and 700 MB for each of its instances. Anyone else running into this kind of problem? Anyone know a solution?

[Lorne]

There should only be one instance of VP20Design.exe in the list of processes. VP20Design.exe never spawns another instance of itself.

Also, adding a version to the project shouldn't require much memory; it isn't a memory-intensive operation.

I just tested with version 2.0.4.0 and I get a single instance of VP20Design.exe. After adding a dozen versions and many files, the peak memory use was still under 34 MB.

What version of Windows are you running? (I performed this test on XP Pro, although I've never seen the behaviour you described on any operating system.)

Have you rebooted the system? Try that, and launch Visual Patch with nothing else running.

You could also try launching VP20Design.exe from a command prompt, in case you're getting multiple instances from multiple mouse clicks.

Do the out of memory errors occur even if you launch Visual Patch immediately after rebooting, with nothing else installed?

I'm wondering if it could have something to do with having 4 GB of RAM, especially if you're on a 32-bit operating system. It might be worth trying it on a system with 2 GB or less, to see if that makes any difference. We haven't had any reports of problems on systems with that much RAM installed, but you never know, with Windows there are a lot of specific configuration details that come into play.

As for the amount of memory used, 4 GB is a lot of memory to play with, so memory shouldn't be too much of a concern for you at all -- assuming it isn't causing memory allocation errors for some reason. Unless you're running a 64-bit version of Windows, Visual Patch could never use more than 2 gigs anyway. (Well, possibly 3 gigs if you had XP configured in a very unusual way.)

A bit of background on memory use when building patches
When building a patch for very large files, the more memory you can use, the better; Visual Patch will gladly gobble up as much free physical RAM as it can address on your system while it's building patches for really large files. (By really large, I mean above 600 MB.) With that said, we've tested building patches for gigabyte-sized files on systems with 128 MB of RAM. It takes longer to build, and you end up with a slightly larger patch file, but that's the tradeoff for operating in constrained memory.

For performance reasons, building a patch will try to use the largest contiguous block of address space it can allocate.

If it needs to operate in constrained memory, it will do so just fine, although the patches may end up a bit larger...which is why we recommend closing all other applications when building patches for very large files.

The actual amount of memory required varies by the size of the files you're building patches for, and a few other factors, so there isn't any simple way to predict the amount of memory needed. But for the smallest possible patches when very large files are involved, close everything else down, and build the patch right after starting Visual Patch 2.0 so the patching engine isn't further constrained by fragmented address space.

[witness]

Sample of error

Here is a screen capture of what I see when VP20 is trying to open my project.

[Lorne]

That screenshot shows it building the project. It shouldn't be building the project when you start Visual Patch, unless you have it set up to do an unattended build for some reason.

How are you opening the project file? Are you double-clicking it in explorer, or are you launching Visual Patch and then loading the project into it?

Edit: I have a theory about this, but before I look into it further, what operating system are you running?

[Adam]

What OS is that running on?

For a test try building a patch between 2 versions rather than a full history (all version) to see if that sidesteps the issue. That info could help us to resolve this issue.

Also in task manager show the "Virtual Memory Size" column.

Adam Kapilik

[witness]

Feedback

My system is Windows XP Pro. The application I am using your products for is an HTML product demo that contians about 30,000 files (a mix of html, images, flash, other web type files). All told, the product is about 1.7 GB. My typical patches are between 10 and 50 MB. The strange part about the duplicate VP20Design.exe files in my performance monitor is that the application never actually spawns another version of itself. If I remember correctly, the duplicate has shown up after a patch build with the memory error appears and then I close and reopen VP20. I am trying to replicate the steps now so I can provide a more detailed explination.

[witness]

Feedback

Lorne: That screenshot shows it building the project. It shouldn't be building the project when you start Visual Patch, unless you have it set up to do an unattended build for some reason.

This screen was captured right after (a few minutes) I clicked the Build icon. I am not trying to open or start any other aps.


How are you opening the project file? Are you double-clicking it in explorer, or are you launching Visual Patch and then loading the project into it?

I open the project file through File/Open not by double clicking.

Edit: I have a theory about this, but before I look into it further, what operating system are you running?



ADAM:
What OS is that running on?
Running on Windows XP Pro

For a test try building a patch between 2 versions rather than a full history (all version) to see if that sidesteps the issue. That info could help us to resolve this issue.

If I run a patch on two versions it doesn't run out of memory.

Also in task manager show the "Virtual Memory Size" column.

Attached is a new screen shot of another error and the VM size column shown.

[Lorne]

As strange as it sounds, I suspect the problem may be that you have too much memory free. I think the amount of memory you have free is causing the DeltaMAX differencing engine to make some bad decisions about how much memory it can use -- in this case is at the expense of Visual Patch itself.

If that's the case, it's a bug, and I offer my apologies. In most cases DeltaMAX can't get anywhere near enough memory to difference very large files without sacrificing performance and compression to some extent, so it's designed to use as much free memory as it can, within reason. Unfortunately the "within reason" part isn't reasonable enough in your case.

The Windows virtual memory system means that every 32-bit process essentially has 2 gigabytes of memory to work with -- even if you had 8 GB installed, each application could still only see 2 GB.

(Technically it's possible to increase that to 3 GB, but for various reasons systems are rarely if ever configured that way.)

So even though you have 4 GB installed, Visual Patch can only use up to 2 GB, tops.

I think the differencing engine is taking advantage of the huge amounts of memory you have free at the expense of Visual Patch itself. It isn't leaving enough memory available for things like the file processing lists to grow.

The code tries to anticipate this situation by leaving some of its 2 GB of addressable memory free for Visual Patch to use "no matter what," but in this case the amount it leaves free isn't enough for the number of files involved.

Potential workarounds
I can think of a couple things to try to get around this right now:

1. Go to Publish > Settings, click on the Optimizations tab, and set the Low Memory Scenario option to "Maximize build speed." Save the project, reboot, and try building the project again.
2. If that doesn't work, try to bring your free memory to under 2 GB. I've attached a little utility I wrote that you can use to "gobble" up memory, or you could just open a lot of applications in the background.

[witness]

I'll give those items a try and let you know how things work out. Thanks for your help. This would be the first time I have ever had a problem because of too much RAM

[Lorne]

(I've logged this as REF: 14174.)

[Adam]

Ahh MemGobbler saves the day again. Post your results when you get a chance because the outcome of your situation could help us to know exactly what to fix.

Adam Kapilik