When I let it sit there user can work with outlook, when ik click cancel (OK button is grayed out) i have to click on Excange needs password. from within oulook i can set the out of office reply i am not able to run the test email outlook configuration (holding donw CTRL and right click on outlook in system tray) At that moment the popup appears.
If you've done that already and use the Outlook web version for your emails, you can set an out of office reply in Outlook for web, too. Enable the box that says Send automatic replies at the top.