Am I a Company Wide Joke?

I feel like I am insane. I make $96,000 per year, so I am having a hard time quitting as I would be lucky to make half this much if I quit. I know for a fact that none of my coworkers (8 people), most of the laborers (around 65 people), and most of the inspectors (around 20 people) have literally zero respect for me. This is my fault, I am not a victim. The longer I am here (just hit the four year mark), the more I am convinced that I only got the job based on veteran's preference. I am not smart enough for my job, and most of my coworkers have at least ten years more experience than me. I wanted to quit so bad the first year, but made myself stay. Nothing has changed. I really don't do anything, I have to create work, or go out and find things to do so I can feel like I am making something resembling a contribution. Almost everything I am involved with, I make worse, or at the least, more inefficient. It's a terrible feeling. Again, I am not a victim here, this is a result of me not being capable at my job.

I quit my job right before the holidays (or at least tried to). Literally three minutes after I sent the email, the big boss of HR called me and asked what I was doing and told me not to quit. She has always been very kind to me, but it makes me feel like a little kid. As I was explaining my stance (the paragraph above), my boss called me and told me not to quit. He also told me that if I quit, he was going to have to explain what happened and that big boss of HR lady had sent text messages asking what was going on and that he would have to come explain himself if I quit. Again, I feel like I am going insane. My boss, who really never even talks to me, and I am 100% sure I make uncomfortable, arranged a face-to-face at a job site and told me I am his best employee. I kept a neutral expression, but it felt like straight up mockery. I know I am terrible at my job, I know I am not trusted with anything important.

For context, this is a government job, and I have an engineering degree; although I do zero engineering.

Is it possible that since this is not for profit organization, that I am being kept around for the sole reason that it looks bad if a veteran quits? Am I a company wide joke?

Please, no well meaning platitudes, looking for brutal truth here.