I think it's way simpler to pack all your stuff into a soft sided bag and put that inside your hard pannier, rather than padding the pannier itself. If you really want it quiet, you can put compression straps around the bag to keep everything tight. I've been using these inside my Pelican knockoff cases for several years now, and they work great.
The problem with lining your cases with anything absorbent like felt is, if they ever get wet, they'll take a long time to dry; in a humid climate like we have in NC, that means mold and mildew.
The bags I bought are listed as "Travel Bag Lightweight Duffel Gym Bag Waterproof Foldable Portable Luggage Bag Men Women (32L, Blue) " on Amazon. They're not bad for nine bucks.
View attachment 64226