I admit it's a bit of black magic doing it this way but it may be a viable solution if you can't do it inside maya. I didn't check if/how it's doable within maya but I guess you did thorughly.
Anyway I took a look on hex dumps of several .swatches files and the file format is quite simple and not too hard to deduce. Although I still have a couple of blackhole bytes. But you could get away with it if you're writing to an existing swatch file.
It consists of simple file header followed by number of swatch header+data blocks:
CODE
// file header
BYTE[] "MayaIcons"
DWORD unknown(?)
BYTE[] "Swatches"
DWORD unknown(?) // probably number of swatches in file
//swatch header
WORD swatch_name_string_length
WORD width
WORD height
WORD unknown(?) // probably flags of some sort
BYTE[] swatch_name
// swatch pixel data
BYTE[4] RGBA
BYTE[4] RGBA
BYTE[4] RGBA
BYTE[4] RGBA
.
.
.
// next swatch header+data block
So when you write to the existing file you just need to alter the width, height and actual rgba data, and copy everything else.
The only problem I see with this method is how to force hypershade to refresh swatches after they are written. It seems that maya loads the swatches when scene is opened and that's it. Which would mean that only way to refresh the swatches is to save/reload the scene. Not too pretty from regular user's point of view.