Black boxes were developed in Australia in the early 1950s. It was, simply, just painted black.
The original version was a recorder designed with physical magnetic tape, with microphones placed randomly around the cockpit. It was encased in a fireproof box, and paint itself is used in every industry to protect bare metal and stop rust. That's just the color they painted it when it was developed.
Black Boxes are actually known in the aerospace industry as Flight Data and Cockpit Voice Recorders. The Cockpit Voice Recorder records foud channels of audio for a duration of two hours, while the Flight Data Recorder records 25 hours of data… and may record several thousand parameters.
As to why the media calls recorders Black Boxes, the name may have its origin in early engineering design philosophies, where boxes that contained electronic components were termed "black boxes" or possibly by their original color which was black. All voice and data recorders are painted a bright orange which helps search and recovery teams identify the recorders when searching an accident scene.