The chat tab remains open even when I set settings to closed.

This is a temporary browser cache issue if you leave the facebook chat box open and revisit your website in a short time period.

Your new visitors will see a closed chat box tab. You can test in a private browser window to confirm there is no problem.