Today I tried to map a network drive to a folder shared via SMB from Mac OS X. I enabled sharing in System Preferences and set up the user account appropriately. But whenever trying to connect from Vista, I ended up with “Systemfehler 1326” (“System error 1326 has occurred. Logon failure: unknown user name or bad password”) complaining about invalid username or password.
First I suspected a problem with my longish password that contains special characters, but that was not it. Turns out it is a compatibility problem/feature between the Samba configuration in OS X (the component responsible for sharing folders via the SMB protocol) and Vista’s default security settings.
First a solution for the impatient:
On Vista launch regedit.exe and navigate to “HKLM/SYSTEM/CurrentControlSet/Control/Lsa”. Check the value of “LmCompatibilityLevel” and set it to 1 – it defaults to 3.
For a list of settings for this key, see Microsoft Knowledge Base Entry 239869
On my system I did not have to reboot, I could connect to the Mac share immediately.
LM Compatibility Level 3 means the client will only try NTLMv2 authentication. This will not work against OS X in the default configuration, which only offer NTLMv1. By setting this to 1 you tell Vista to use v2 if the server supports it, but fall back to v1 if not.
While this is a quick and rather simple fix, it degrades security. By default Vista only connects to SMB servers that support the NTLMv2 authentication mechanism, because it is superior to the older variant from a cryptographic point of view. See http://davenport.sourceforge.net/ntlm.html and the Wikipedia entry on NTLM for more details.
In general you should prefer increasing security instead of loosening restrictions. To do so, you should configure XP and Windows 2000 to the same level 3 setting as Vista (the registry key is the same) and also set up Mac OS X to support NTLMv2.
Edit /var/db/smb.conf (using sudo vim) and make sure, the following two lines are present:
ntlm auth = no lanman auth = no
If not, add or edit them to appear like this. Do not change anything else in that file!
When you are done, relaunch the Samba daemons:
sudo launchctl stop org.samba.smbd sudo launchctl stop org.samba.nmbd sudo launchctl start org.samba.nmbd sudo launchctl start org.samba.smbd
From now on, Mac OS will only accept NTLMv2 connections, matching the higher security standards and refuse v1 clients – so make sure, you configure all your XPs accordingly.