Sunday, April 12, 2009

Solved problem: Access Mac OS X SMB Shares from Vista – no more System Error 1326

Today I tried to map a network drive to a folder shared via SMB from Mac OS X. I enabled sharing in System Preferences and set up the user account appropriately. But whenever trying to connect from Vista, I ended up with “Systemfehler 1326” (“System error 1326 has occurred. Logon failure: unknown user name or bad password”) complaining about invalid username or password.

First I suspected a problem with my longish password that contains special characters, but that was not it. Turns out it is a compatibility problem/feature between the Samba configuration in OS X (the component responsible for sharing folders via the SMB protocol) and Vista’s default security settings.

First a solution for the impatient:

On Vista launch regedit.exe and navigate to “HKLM/SYSTEM/CurrentControlSet/Control/Lsa”. Check the value of “LmCompatibilityLevel” and set it to 1 – it defaults to 3.

For a list of settings for this key, see Microsoft Knowledge Base Entry 239869

On my system I did not have to reboot, I could connect to the Mac share immediately.

LM Compatibility Level 3 means the client will only try NTLMv2 authentication. This will not work against OS X in the default configuration, which only offer NTLMv1. By setting this to 1 you tell Vista to use v2 if the server supports it, but fall back to v1 if not.

While this is a quick and rather simple fix, it degrades security. By default Vista only connects to SMB servers that support the NTLMv2 authentication mechanism, because it is superior to the older variant from a cryptographic point of view. See http://davenport.sourceforge.net/ntlm.html and the Wikipedia entry on NTLM for more details.

In general you should prefer increasing security instead of loosening restrictions. To do so, you should configure XP and Windows 2000 to the same level 3 setting as Vista (the registry key is the same) and also set up Mac OS X to support NTLMv2.

Edit /var/db/smb.conf (using sudo vim) and make sure, the following two lines are present:

ntlm auth = no
lanman auth = no

If not, add or edit them to appear like this. Do not change anything else in that file!

When you are done, relaunch the Samba daemons:

sudo launchctl stop org.samba.smbd
sudo launchctl stop org.samba.nmbd
sudo launchctl start org.samba.nmbd
sudo launchctl start org.samba.smbd

From now on, Mac OS will only accept NTLMv2 connections, matching the higher security standards and refuse v1 clients – so make sure, you configure all your XPs accordingly.

7 comments:

Bennie said...

You could just have added:

client ntlmv2 auth = yes

to the [global] section. Samba had ntlmv2 authentication for years.

Daniel Schneller said...

Yes, and the way I configured both Vista/XP and Mac OS X is to even *only* accept NTLMv2. I explicitly did not want any compatibility mode, but instead included instructions on how to enable NTLMv2 in both Windows versions exclusively.

Atreus said...

Thanks for a really dandy post. You solved a headache I've been ignoring for ages. :)

Jason said...

Any idea where to add these comments in Lion Server?

steve call said...

Great just did not work for Windows vista, the problem is that the admin user works ok all i do is type in the ip address and it works, but for a second user with their own account it dose not. invalid username or password knowing full well what the password is. So user 1 gets access but user 2 dose not.
Steve

Unknown said...

If you use Windows 2008 server, you need to change LmCompatibilityLevel to 3.

Fernando Echeverria said...

Thanks! Your post was many years ago, but it's still relevant, since it helped me solve this problem, which was driving me nuts for days! I didn't find this kind of advice anywhere else.
Just like Unknown said, in my case (Windows 7 to OS X Yosemite) the solution was exactly the opposite as yours: The setting was at 1 and I had to change it to 3. After that, it worked immediately.