Spoke to a guy about buying a new transmission to swap in and when I explained my problem he said that's normal with acura automatic transmissions. He suggested that if it's becoming an annoyance to just flush the transmission and make sure to use honda ATF to help keep it smooth.
So I guess i'll just drive it as it is and flush the transmission when i can. If the tranny goes i guess i'll just deal with it when the time comes.
Thought i'd post my findings here for future searchers. I'll update if anything changes.
here's a picture of my 4runner, for the hell of it.