Bizarre NFS problem


Results 1 to 9 of 9

Thread: Bizarre NFS problem

  1. #1
    Join Date
    Oct 2006
    Posts
    41

    Bizarre NFS problem

    I'm having a hell of a time trying to understand and resolve two NFS issues and I'm hoping that the knowlegeable folk on this forum are able to help me understand and resolve the issues (I'm guessing they're related).

    Background:
    I am running Ubuntu 10.10 on my workstation and have Ubuntu Server 10.10 installed on a HP Proliant MicroServer.

    A NFS share is configured on the MicroServer and successfully mounts from workstation PC/ Ubuntu 10.10

    The issues:
    • Copying files to an NFS share on the microserver is very, very slow.
    • Whilst files copied to the NFS share are visible to the workstation, they don't appear when using 'ls' on the server.
    • despite the folder owner/ user account on the Microserver and the workstation having the same name and sharing the same UID & GID (1000), copies made from the workstation have a different UID & GID
      Code:
      workstation$ ls -la
      total 28
      drwxr-xr-x 7 4294967294 4294967294 4096 2011-04-19 22:04 .
      drwxr-xr-x 6 root       root       4096 2011-04-21 19:54 ..
      drwxrwxrwx 4 4294967294 4294967294 4096 2011-04-21 19:55 disk1
      drwxrwxrwx 2 4294967294 4294967294 4096 2011-04-19 22:04 disk2
      drwxrwxrwx 2 4294967294 4294967294 4096 2011-04-19 22:04 disk3
      drwxrwxrwx 2 4294967294 4294967294 4096 2011-04-19 22:04 disk4
      Code:
      microserver$ ls -la
      total 28
      drwxr-xr-x 7 egd  egd  4096 2011-04-19 22:04 .
      drwxr-xr-x 4 root root 4096 2011-04-19 07:27 ..
      drwxr-xr-x 3 egd  egd  4096 2011-04-21 08:24 disk1
      drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:13 disk2
      drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:16 disk3
      drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:17 disk4


    Configuration:
    HP Proliant MicroServer:

    Network config:
    Code:
    microserver$ ifconfig
    eth0      Link encap:Ethernet  HWaddr 68:b5:99:79:ff:90  
              inet addr:192.168.168.252  Bcast:192.168.168.255  Mask:255.255.255.0
              inet6 addr: fe80::6ab5:99ff:fe79:ff90/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:170867 errors:0 dropped:0 overruns:0 frame:0
              TX packets:86755 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000 
              RX bytes:255720726 (255.7 MB)  TX bytes:6477663 (6.4 MB)
              Interrupt:18
    4 x 2TB drives formatted ext4, mounted to /media/audiolib/disk1...4 via fstab:
    Code:
    /dev/sda1  /media/audiolib/disk1  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdb1  /media/audiolib/disk2  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdc1  /media/audiolib/disk3  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdd1  /media/audiolib/disk4  ext4  user,noatime,noexec,nodev  0  0
    folder/ file permissions for /media/audiolib
    Code:
    microserver$ sudo ls -la
    drwxr-xr-x  7 egd  egd  4096 2011-04-19 22:04 audiolib
    folder/ file permissions for /media/audiolib/disk1...4
    Code:
    drwxr-xr-x 3 egd  egd  4096 2011-04-21 08:24 disk1
    drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:13 disk2
    drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:16 disk3
    drwxr-xr-x 3 egd  egd  4096 2011-04-19 21:17 disk4
    NFS share configuration:
    Code:
    microserver$ cat /etc/exports 
    # /etc/exports: the access control list for filesystems which may be exported
    #		to NFS clients.  See exports(5).
    #
    # Example for NFSv2 and NFSv3:
    # /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
    #
    # Example for NFSv4:
    # /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
    # /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
    #
    /media/audiolib 192.168.168.1/24(rw,root_squash,sync,nohide)

    Workstation:
    Network config:
    Code:
    workstation$ ifconfig
    eth0      Link encap:Ethernet  HWaddr 00:1e:8c:44:b5:06  
              inet addr:192.168.168.1  Bcast:192.168.168.255  Mask:255.255.255.0
              inet6 addr: fe80::21e:8cff:fe44:b506/64 Scope:Link
              UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
              RX packets:93652 errors:0 dropped:0 overruns:0 frame:0
              TX packets:24996 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000 
              RX bytes:12985610 (12.9 MB)  TX bytes:245907927 (245.9 MB)
              Interrupt:17
    NFS share mounted using
    Code:
    sudo mount 192.168.168.252:/media/audiolib /media/canonflash/
    Code:
    workstation$ mount
    192.168.168.252:/media/audiolib on /media/canonflash type nfs (rw,vers=4,addr=192.168.168.252,clientaddr=192.168.168.1)
    Code:
    egd@workstation:/media$ ls -la
    total 24
    drwxr-xr-x  6 root       root       4096 2011-04-21 19:54 .
    drwxr-xr-x 22 root       root       4096 2011-04-02 11:18 ..
    drwxr-xr-x  2 egd        egd        4096 2010-05-09 10:13 0CDB-0F33
    drwxr-xr-x  2 egd        egd        4096 2010-05-03 20:05 AudioLIB
    drwxr-xr-x  7 4294967294 4294967294 4096 2011-04-19 22:04 canonflash
    drwxr-xr-x  7 egd        egd        4096 2011-01-25 19:27 egd_backup
    -rw-r--r--  1 root       root          0 2011-04-12 07:04 .hal-mtab
    Last edited by UbuntuBantu; 04-21-2011 at 03:03 PM.

  2. #2
    Join Date
    Oct 2000
    Location
    Sweden
    Posts
    494
    I'll give it a shot - ok, I've had a few glasses of wine tonight (liquid - not digital! ) so please don't kill me if I'm wrong!
    This seems to me to be an issue with username/uid.
    You need to have the same user on client and server - that is, the user must have same uid on both computers, even if it's the same username that really means nothing. Username is only for us humans it is not used by computers, computer uses number=uid!
    (Well, actually you can map a user on client to a different uid on server, but I don't think you've done that.)
    Look at how it's displayed when you issue "ls -la" on workstation:
    Code:
    drwxr-xr-x  2 egd        egd        4096 2010-05-03 20:05 AudioLIB
    drwxr-xr-x  7 4294967294 4294967294 4096 2011-04-19 22:04 canonflash
    drwxr-xr-x  7 egd        egd        4096 2011-01-25 19:27 egd_backup
    The user with uid&gid 4294967294 is not recognized - otherwise the name would be displayed instead of the number.

    So my suggestion is to make sure you have a user with same uid on both client & server.
    In pingvino veritas!

  3. #3
    Join Date
    Mar 2004
    Location
    Gatineau Quebec
    Posts
    823
    I've had a few diet 7Up & vodka

    what x said.
    Linux user #367409

  4. #4
    Join Date
    Oct 2006
    Posts
    41
    So how does one resolve it?

  5. #5
    Join Date
    Oct 2006
    Posts
    41

    Ok, I've made some config changes, but the problems persist

    Changed NFS config as follows:
    Microserver:
    Code:
    microserver:/$ cat /etc/exports 
    # /etc/exports: the access control list for filesystems which may be exported
    #		to NFS clients.  See exports(5).
    #
    # share all hard drives mounted to /diskpool
    /diskpool 192.168.168.0/24(rw,no_subtree_check,async)
    Code:
    microserver:/$ cat /etc/fstab
    # /etc/fstab: static file system information.
    /dev/sda1  /diskpool/d1  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdb1  /diskpool/d2  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdc1  /diskpool/d3  ext4  user,noatime,noexec,nodev  0  0
    /dev/sdd1  /diskpool/d4  ext4  user,noatime,noexec,nodev  0  0
    /dev/sde1  /diskpool/d5  ext4  user,noatime,noexec,nodev  0  0
    Code:
    microserver:/$ ls -la
    total 102
    drwxr-xr-x  22 root root  4096 2011-04-22 09:58 .
    drwxr-xr-x  22 root root  4096 2011-04-22 09:58 ..
    drwxr-xr-x   2 root root  4096 2011-04-19 22:50 bin
    drwxr-xr-x   4 root root  2048 2011-04-18 22:16 boot
    drwxr-xr-x  18 root root  3860 2011-04-22 11:50 dev
    drwxr-xr-x   7 egd  egd   4096 2011-04-22 09:59 diskpool
    Code:
    microserver:/diskpool$ ls -la
    total 28
    drwxr-xr-x  7 egd  egd  4096 2011-04-22 09:59 .
    drwxr-xr-x 22 root root 4096 2011-04-22 09:58 ..
    drwxr-xr-x  3 egd  egd  4096 2011-04-22 11:28 d1
    drwxr-xr-x  3 egd  egd  4096 2011-04-22 11:32 d2
    drwxr-xr-x  3 egd  egd  4096 2011-04-22 11:42 d3
    drwxr-xr-x  3 egd  egd  4096 2011-04-22 11:42 d4
    drwxr-xr-x  3 egd  egd  4096 2011-04-22 11:48 d5
    Workstatation:
    Code:
    p5e64wspro:/media$ ls -la
    total 28
    drwxr-xr-x  7 root root 4096 2011-04-22 12:15 .
    drwxr-xr-x 22 root root 4096 2011-04-02 11:18 ..
    drwxr-xr-x  7 egd  egd  4096 2011-04-22 09:59 diskpool
    Code:
    p5e64wspro:/media/diskpool/d1$ sudo mount 192.168.168.252:/diskpool /media/diskpool/
    Code:
    p5e64wspro:/media/diskpool$ ls -la
    total 28
    drwxr-xr-x 7 egd  egd  4096 2011-04-22 09:59 .
    drwxr-xr-x 7 root root 4096 2011-04-22 12:15 ..
    drwxr-xr-x 3 egd  egd  4096 2011-04-22 12:21 d1
    drwxr-xr-x 2 egd  egd  4096 2011-04-22 09:59 d2
    drwxr-xr-x 2 egd  egd  4096 2011-04-22 09:59 d3
    drwxr-xr-x 2 egd  egd  4096 2011-04-22 09:59 d4
    drwxr-xr-x 2 egd  egd  4096 2011-04-22 09:59 d5
    Code:
    p5e64wspro:/media/diskpool/d1$ ls -la
    total 12
    drwxr-xr-x 3 egd egd 4096 2011-04-22 12:41 .
    drwxr-xr-x 7 egd egd 4096 2011-04-22 09:59 ..
    drwxr-xr-x 2 egd egd 4096 2011-04-19 21:22 Fleetwood Mac - Rumours

    Now listing same folder via the Microserver:
    Code:
    microserver:/diskpool/d1$ ls -la
    total 24
    drwxr-xr-x 3 egd egd  4096 2011-04-22 11:28 .
    drwxr-xr-x 7 egd egd  4096 2011-04-22 09:59 ..
    drwx------ 2 egd egd 16384 2011-04-22 11:28 lost+found
    The copy operation is damned slow and why don't the files show up on the damned server?
    Last edited by UbuntuBantu; 04-22-2011 at 06:51 AM.

  6. #6
    Join Date
    Oct 2000
    Location
    Sweden
    Posts
    494
    Aha, I think I see the problem:
    /diskpool/d1 - d5 are all different partitions, they are therefore not accessible from client via nfs.
    You have to explicitly export those directories and then mount them on client - maybe there's a config option in nfs to allow partitions to "follow" when mounting parent dir, nothing that I know of though.
    Last edited by x; 04-22-2011 at 06:57 AM. Reason: Adding missed detail
    In pingvino veritas!

  7. #7
    Join Date
    Oct 2006
    Posts
    41
    Quote Originally Posted by x View Post
    Aha, I think I see the problem:
    /diskpool/d1 - d5 are all different partitions, they are therefore not accessible from client via nfs.
    You have to explicitly export those directories and then mount them on client - maybe there's a config option in nfs to allow partitions to "follow" when mounting parent dir, nothing that I know of though.
    Thx, I solved it a few minutes prior to your post by doing exactly what you'd just said, explicitly exporting each drive/partition and mounting each individually.

    So, the lesson for anyone wanting to use NFS to deal with a pool of drives is as follows:

    say you have 5 drives on a server you're wanting to share via NFS, and they're mounted on the server as /drivepool/d1...d5.
    One would think that you'd only need to export /drivepool on the server side and mount it on the client end as follows:

    Server export: /diskpool 192.168.168.0/24(rw,no_subtree_check,async)
    Client mount: $ sudo mount 192.168.168.252:/diskpool/ /media/diskpool/

    That works (insofar as you can see the drives and access their content from the client), but writing to any of the drives is dog slow and only Linus knows where the files are actually written to, cause they're only visible on the client side, not on the server (somehwere I have at least 10 copies of FM - Rumours hidden in my filesystem -- judging by write speed it's probably on my memory stick).

    The only way (I know of) to get this working as you'd think it would is to explicitly export every drive mountpoint as follows:
    Server export:
    /diskpool/d1 192.168.168.0/24(rw,no_subtree_check,async)
    /diskpool/d2 192.168.168.0/24(rw,no_subtree_check,async)
    /diskpool/d3 192.168.168.0/24(rw,no_subtree_check,async)
    /diskpool/d4 192.168.168.0/24(rw,no_subtree_check,async)
    /diskpool/d5 192.168.168.0/24(rw,no_subtree_check,async)

    Client mount:
    $ sudo mount 192.168.168.252:/diskpool/d1 /media/diskpool/d1
    $ sudo mount 192.168.168.252:/diskpool/d2 /media/diskpool/d2
    $ sudo mount 192.168.168.252:/diskpool/d3 /media/diskpool/d3
    $ sudo mount 192.168.168.252:/diskpool/d4 /media/diskpool/d4
    $ sudo mount 192.168.168.252:/diskpool/d5 /media/diskpool/d5

    Thank goodness for fstab!

  8. #8
    Join Date
    Jul 2003
    Location
    Spokane, Washington
    Posts
    580
    I use NFS in my network. I don't have seperate partitions, and in fact export different subdirectories in the same encrypted partition for different purposes. However, there might be some options about submounts (though I thought the default was to export the whole tree and an option was needed to not export the submounts). 'man exports' or 'man /etc/exports' should say more, or maybe 'man nfsmount' or something like that.

    Regarding uid/gid consistency, I use NIS for network logins to keep users and groups syncronized. In a small enough network (up to 5 workstations?) it could be managed by hand, but I use NIS even with only 2-3 workstations. (I'll take this moment to mention that I like the way Windows networking uses different namespaces for accounts on different machines, and a centralized authorization certifier for access control, but I don't think it has to be implemented nearly as complicatedly as Microsoft has.)

  9. #9
    Join Date
    Oct 2000
    Location
    Sweden
    Posts
    494
    I believe the files you copied earlier are on the server, if you unmount /diskpool/d1 on server you'll probably find your lost files.
    In pingvino veritas!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •