13
$\begingroup$

I’m looking for a way to review very high resolution video alongside a scrolling spectrogram of the associated audio track. Some DAWs that have spectrograms (Adobe Audition,Reaper) allow for simultaneous video but the video won’t playback at full frame rate or resolution. VLC can handle the video and has audio visualizations, but not a spectrogram. Anyone know of a tool?

It isn’t enough to route the audio to a separate program running a real time display. I need to be able to scroll around the video and see the spectrogram as I do.

As an alternative, I’ve been making a movie of a spectrogram using screen recording, then overlaying this on the video in Adobe Premiere. This actually works very well for playback but it’s a pain to make the movies. I just have to screen record them and manually sync them up. Surely someone has a tool that can generate a video of a scrolling spectrogram without screen recording and faster than real-time. Thoughts?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ For anyone else who's interested, Boris [link] (boris.unito.it) looks like a very user-friendly way to do this and it seems it would handle the high-res video smoothly. Sadly they don't support Mac anymore so I'm unable to make it work. Linux or Windows, it might be the answer. $\endgroup$
    – Marc D
    Commented Aug 18, 2022 at 20:21
  • $\begingroup$ I edited the title of your question a bit to make it more detailed/specific and better fit the SE Q/A format. I hope to have maintained the essence of your question, but if not, please comment here or edit back. For more info see: How do I write a good title? $\endgroup$
    – selene
    Commented Aug 25, 2022 at 16:41
  • $\begingroup$ sometimes comments can be deleted over time or get hidden, your suggestion would make a great answer for others looking to solve the same problem. You are welcome to answer your own question on SE, so would you mind turning your above comment about Boris into an answer? $\endgroup$
    – selene
    Commented Aug 25, 2022 at 16:44
  • $\begingroup$ Happy to do so but how do I change the comment into an answer? Do I repost it as an answer? $\endgroup$
    – Marc D
    Commented Aug 26, 2022 at 17:34
  • $\begingroup$ yup exactly...You can just scroll down to the bottom and add it as an answer (there should be a white answer box you can type in or a blue button that says "answer your own question" or something like that. Then if folks try it and it helps them, they can vote for it! $\endgroup$
    – selene
    Commented Aug 26, 2022 at 17:52

5 Answers 5

Reset to default
6
$\begingroup$

I asked a question in the similar topic couples of weeks ago: Annotate acoustic and visual events of a multimodal signal.

To get real good quality scrolling spectrogram in video I use dynaSpec package in R that allows you to generate MP4 video of a spectrogram: https://marce10.github.io/dynaSpec/.

When I do a video recording of a vocalizing animal and I want to review both the visual behaviour and the spectrogram of the vocalization, I first create a scrolling spectrogram using dynaSpec. Then, I combine the video recording with the scrolling spectrogram using Davinci Resolve software (free version).

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ This is a great answer to the second option, which is creating a scrolling video and adding it with video editing software. I'd prefer some software that just loaded the video and played a spectrogram in real-time. But your solution to the creation of scrolling spectrograms is a definite improvement over the way I create spectrogram movies. Right now I'm stuck screen recording. Now all I have to do is learn R. $\endgroup$
    – Marc D
    Commented Aug 15, 2022 at 15:39
  • 1
    $\begingroup$ I have a collaborator trying to make this work but he's running into difficulties. I'll report back if he gets it up and running. Meanwhile, I'm just screen recording Raven windows so I can move forward. That may end up being the easiest solution that is accessible to me after all. $\endgroup$
    – Marc D
    Commented Aug 18, 2022 at 20:23
  • 1
    $\begingroup$ My collaborator got the R package working, but it is painfully slow. He did a three-second file and it took 3 minutes. We set up an hour-long file and let it go overnight. It says it still has more than 7 days. Does this jive with anyone else's experience with this package? Or is something wrong on our side? $\endgroup$
    – Marc D
    Commented Aug 19, 2022 at 15:23
  • $\begingroup$ Hi @MarcD , I haven't used this package to date, but R can be notoriously slow with some (seemingly) straightforward processes. I'm assuming you're using a high performance computer to implement the package? $\endgroup$
    – Tristan Louth-Robins
    Commented Aug 22, 2022 at 21:46
  • $\begingroup$ Hello, it is right that R is very slow to create such MP4 file. I think the package was created just for making a short video of scrolling spectrogram. Even with my gaming computer that is very powerfull, it takes ages to create scrolling spectro, while creating long videos with Davinci Resolve as example, is much faster. $\endgroup$
    – Mathieu Mahamoud Issa
    Commented Aug 23, 2022 at 9:42
5
$\begingroup$

There is a similar question on this Video SE. One suggestion/answer uses ffplay (linux?) to combine video with scrolling spectrogram.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. I wish I could try that but I think this method is technically out of reach for me. No Linux knowledge. $\endgroup$
    – Marc D
    Commented Aug 15, 2022 at 15:41
4
$\begingroup$

I’m not sure that this will fully solve your problem, but for the alternative you have listed (generate scrolling spectrogram + overlay w/ video in adobe premiere), ffmpeg is a useful tool to consider. The “showspectrum” function from ffmpeg is a great option for producing scrolling spectrogram videos from a .wav by running a (relatively) simple shell script. This tool gives you lots of flexibility with spectrogram parameters, aesthetics, etc. (a good example is found here). You can create a similar .sh script for your needs and run from the command line to generate your scrolling spectrogram video (or many such videos) without screen recording.

This of course would still require syncing with video, but will at least automate and greatly speed up your generation of scrolling spectrogram videos. As another commenter suggested, dynaSpec (R package) also appears to be a useful tool for this purpose but I do not personally have experience working with that package.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you. It's not clear to me how much time that would save me vs. screen recording, especially as I'd need to learn ffmpeg first. I think I'm just going to record the Raven window. It is realtime, but with my long recordings, I can just let it run. The screen recorder I have can just stop after a set period of time, so I don't have to stick around. I may look back at that option when I have more data, but for a few days worth of recording, I'll probably just go with what I know. $\endgroup$
    – Marc D
    Commented Aug 18, 2022 at 20:15
3
$\begingroup$

I do not know how adaptable this set of tools is beyond the specific CATS animal-borne tagging application, but this paper provides a set of code/tools that pairs video, acoustic, and accelerometer data:

https://animalbiotelemetry.biomedcentral.com/track/pdf/10.1186/s40317-021-00256-w.pdf

The paper points to the GitHub repositories for the code and a workshop with tutorials and wikis to guide new users.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Wow! That looks powerful and interesting. It could be overkill and would be a steep curve for me. Maybe for the next phase of my project where I hope to have more data and some little more technical support who could pull it together. Thanks. $\endgroup$
    – Marc D
    Commented Aug 18, 2022 at 20:16
3
$\begingroup$

If you end up working with any of the tools in the GitHub repository mentioned by Selene and run into any problems/questions, let me know and we can work through them together!

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks so much for the offer. It looks like an amazing tool. I wonder if it is overkill given that I only have video and spectrograms. Also, if it would handle the 5.7 k video. Anyway, I think it is too much for me to pull off at this stage. It's a preliminary study with limited hours of data. I think I'm going to bite the bullet and screen record. Maybe in the next phase with more data. $\endgroup$
    – Marc D
    Commented Aug 18, 2022 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.