That's what I figured. I did a little more research and found that the internal check valves are notorious for failing. Some people have gone the route of using an external fuel check valve. So I look at that option and they cost more than a new fuel pump.
I'm just gonna replace the pump. Only cost about $20 and it's an easy job.