Error validating login on the trim workgroup server
You may find details on how to do it here: https://technet.microsoft.com/en-us/library/cc730960MSMQ runs under the (less privileged) Network Service account instead of (all powerful) Local System account starting from version 4.0 (Vista/Server 2008) So to answer @mkus question more directly to "set" domain mode you just install MSMQ on domain controller and make sure that proper permissions in place for Network Service account.
I always try to keep myself up-to-date with latest release of SQL Server.
Also couple of links to clarify issues around MSMQ objects permissions and when/why you need to set them:
In short starting from MSMQ 4.0 Everyone and Anonymous Logon were removed from default MSMQ objects ACLs as precaution against Do S attacks (though there are exlusions to this change and Workgroup mode is one of those).
I was able to upgrade to next version without above error: Have you even seen such upgrade blockers?
Reference: Pinal Dave (https://blog.sqlauthority.com)Pinal Dave is a SQL Server Performance Tuning Expert and an independent consultant.I ran into some problems, no matter how much I installed or uninstalled MSMQ or restarted, I still wasn't able to use the queue.