Make a decoction of coriander seeds. The website I got the info from says 10 grams in 1 liter of water, but I personally just eyeball it.
Anyways, gargle the decoction as needed. It won't take all the pain away but it will make it manageable. You may have to constantly gargle—I have to do so about every 5-15 min dep. on the severity of the pain, but most other things I try do absolutely nothing or only work for a few seconds or less than 5 min.