3
$\begingroup$

I am trying to do soundscape analysis (LTSA, PSDs, TOLs) using PAMGuide in MATLAB. However, after loading about 50-100 files/455 files (depending on my frequency range), MATLAB crashes. Given that the files are very large (about 300MB), I suspect a memory issue. Does anyone have a similar issue and solution?

Thanks in advance!

$\endgroup$
2
  • 3
    $\begingroup$ Welcome @MelanieSmith. Could you give us a bit of info about how much memory your computer has? I have not had this issue with PAMguide. You can look at your memory usage using the Windows Task Manger > Performance tab and see how your memory is changing as the files load. $\endgroup$
    – selene
    Oct 17, 2022 at 16:31
  • 1
    $\begingroup$ Hi Selene, my computer has 16 GB of memory, which gave me an error for array size. I then changed my settings so MATLAB can use virtual memory. MATLAB memory usage increases as I load more files and crashes around 40-50 GB. As I load more and more files, the time it takes to analyze those files also increases. The first file is analyzed in about 20 seconds and the last file that is analyzed before it crashes takes about 400 seconds. All the files are the same length, size, etc. $\endgroup$ Oct 25, 2022 at 0:52

2 Answers 2

2
$\begingroup$

I know, the OP is a year old, but here is my answer:

I guess your parameter settings are suboptimal.

I just tried PAMGuide on a dataset I had on my disk

120 files, 30 s each, 192 kHz sampling

was running PAMguide (with Matlab) with default configuration

PSD, 1 s window, no time average, PlotType Both

processing lasted >900 s, plot never showed up and matlab effectively froze PC (16GB laptop). Had to power off and on again to continue.

Changed parameter in PAMguide to

PSD, 1024 samples window, 10s time average, PlotType Both

and all worked fine

Analysis complete in 119.5887 s.
Plotting... done in 1.8633 s.
Computing noise level statistics... done in 0.048294 s.
Plotting... done in 1.0546 s. 

Suggest, to reduce processing window to a much lower level.

The 1 s window gives you 1Hz resolution and in my case 192000 frequency bins, but screens have typically only about 1000 pixels.

Same is with time averaging: if you have a day of data to visualize, then with 1 s average you would have 86400 samples, but screens are very likely less than 2000 pixels wide, so 10 to 30 s averaging brings your data to a reasonable size.

OK, your may have to play with the parameters, but start first with the smallest window and the longest time averaging. In my case in order to get PSD statistics I needed to reduce time averaging to 2.5 s, which needed nearly the same amount of time for analysis and plotting

Analysis complete in 122.0571 s.
Plotting... done in 0.58137 s.
Computing noise level statistics... done in 0.10046 s.
Plotting... done in 0.88013 s.
Writing concatenated output array...done in 0.72288 s.

When reducing the analysis window, make sure you always have averaging window specified, otherwise you will get far too many time samples and will blow-up your memory.

Comment to your crash: very likely was the page file (the file where virtual memory is off-loaded) was too small and exhausted, generating a crash.

$\endgroup$
1
$\begingroup$

I'm not used to PAM, but MATLAB has some functionalities to work with large data sets, like datastore and tall arrays. You can check available options on this support page. Also, since the whole process may take a while, it may be useful to spend a few more seconds when running setting up a parallel processing pool, which will speed up the process in the long run. A simple way and example is when using parfor (a parallel for).

However, it was mentioned in the comment that "The first file is analyzed in about 20 seconds and the last file that is analyzed before it crashes takes about 400 seconds". For what I understood, it doesn't seem you need to load all the files to analyse the data. So maybe in this case it's just about rearranging the programming logic to load a file, analyse it, save results, clear memory, repeat.

(This "answer" should be a simple comment, but I don't have the reputation needed yet. Just hope it helps.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.