A pile of clothes behaves like one big bouncy thing because its made of many smaller floppy bouncy things.
I'd try building the pile in 8-10 layers of 'clothes', each layer deformed by a soft-body lattice (with goal object, of course). This would make individaul items (like shirt sleeves, etc) 'flop' individually.
Then take the stack of lattices and place them inside one great big soft-body lattice. That will make the pile of clothes act as a whole, while still retaining some of the individual movement of each layer.
Judicious adjustment of goal weights will make the top layers more floppy and bouncy, and the lower layers stick to the characters arms.
HTH
Growler