Armrests are heat formed foam injected thick vinyl ,There is no fabric back. That looks like UV burn and it is not dirt/grease and won't clean off . Time to redye them . after removing them from the door use a good cleaner, Then clean again using lacquer thinner to remove any gloss still left ,Then use Bulldog adhesion promoter to help bind the final color , 1-2 very light coats works better than 1 heavy one .In this case less is better .
https://www.ebay.com/itm/Klean-Strip...38.m2548.l4275
Then use Sem 15103 color coat to finish
https://www.ebay.com/itm/SEM-15103-S...cAAOSwyYFZ-vmd
http://www.tcpglobal.com/SEM-VCS-CC_3.html#.WgIig3ZOnIU
This is the closest color , an exact color match can be done by your local auto interior shop or check in your area for a automotive vinyl repair specialist in your yellow pages .