I have used Cetacean Research's SQ26-H1 Portable Underwater Sound Recording System for listening for fish sounds from a boat, similar to what you're proposing with whales, and I believe the system should be at least close to within your budget. I was able to get a student discount for mine, so I can't recall the usual full cost of it.
The paper discussing the relatively new HydroMoth prototype (Lamont et al. 2022) also presents the results of a systematic review of recent acoustic studies and the costs of the recorders used by their authors, to quantify the lack of low-cost underwater recording equipment, especially when compared to terrestrial recording equipment. So you're not alone in struggling to find affordable options!