Also I do have some investments that require the higher levels of tax prep software or a person just to understand (mostly the whole foreign tax credit stuff on my VTIAX holding) but for the most part, the government knows what we owe or get back, why can't this just be automated and we only file if we disagree?
Probably because it might lead to the ultra-wealthy paying their fair share.
Since two moderators have already chimed in on this, I feel I should point out that the latter doesn't seem to follow from the former. The self-filing method seems silly to me too. But if the government knows how much you should pay, it should also know how much the ultra-wealthy should pay, so I don't see how they get away paying less under the self-filing method.
I suppose you could argue that the wealthy may have "hidden" income that the government doesn't know about - but if that's the case, the self-filing method would result in MORE revenue to the government. Why? Because in the government-bill method, if the government doesn't bill you for your income, then you know they don't know about it and you can get away with not paying taxes on it. In the self-file method at least you have to wonder if the government might know about your "hidden" income and you might declare it just to be safe.
Now, if the self-file method were a true "honor system" then you'd be right, but since by your former assertion it's really more a "trap system" the idea that it's designed to let rich people get off seems...unlikely.