Your wall outlet is AC but a battery is DC, so a pure wire setup is not so much a safety thing as it is just incompatible. A good fire starter.
So you’ll have to convert it, which makes for a big, bulky plug.
On top of that, you’ll need prior knowledge of the battery layout, including whether they’re parallel or serial. Usually parallel, but not a guarantee.
All of this isn’t insurmountable, but is enough to make it not the norm. They do, in fact, already exist
There are better ones out there, that one in particular had some iffy reviews about voltages, but yeah they definitely exist.
I usually see them labeled as “power supply adapter for [AA,AAA,etc]”.