Hrqls: This was not a tough decision: Drop. If you accept you will probably lose and may get gammoned. I'd say if you played 10 games you would win once, be gammoned once and lose without being gammoned 8x.
In fact I wouldn't have doubled if I were your opponent and played for the gammon, I think his position was too good for a double.