File this under "wish I had already known."
A few versions ago WordPress added the ability to upgrade from within, all you had to do was enter your FTP connection info and it would take care of the rest. Turns out part of that was that you couldn't store your FTP connection info in wp-config.php and WP would do the upgrade when you asked without prompting for a password.
Add this somewhere to your wp-config.php file. I just updated my main WP site along with a network site and it went without a hitch.
define('FTP_USER', 'your ftp username');
define('FTP_PASS', 'your ftp password');
define('FTP_HOST', 'ftp host - probably localhost');
Now when you update plugins, themes, or the WordPress core it won't ask you for your connection info. It'll just do it.
Thanks to DigWp.com
for the tip.