how safely remove/add USB flash drive?


Page 1 of 2 12 LastLast
Results 1 to 15 of 20

Thread: how safely remove/add USB flash drive?

  1. #1
    Join Date
    Mar 2003
    Location
    UK
    Posts
    621

    how safely remove/add USB flash drive?

    Got this flash drive or variously 'memory stick' or 'piece of smegma'. (Incidentally it is surprising how effortlessly Debian deals with it - so that the system requirements printed on the packet are a lie: there is no mention of Linux.) The instructions contain the following paragraph:

    To prevent data loss and fatal damage to the product, never remove it from the USB port while data is being transferred. It is advised that user perform "Safely Remove Hardware" by selecting "USB Mass Storage Device" from the windows tool bar, before removing the Flash Disk.
    So are precautions necessary before inserting of removing? Is a ‘lazy’ umount -l alright?

    Thanks any help.
    MI6, Offensive Information, Hackers, Encryption, UFO, AOL, Infowar, Bubba, benelux, Ufologico Nazionale, domestic disruption, 15kg, DUVDEVAN, debugging, Bluebird, Ionosphere, Keyhole, NABS, Kilderkin, Artichoke, Badger, spookwords, EuroFed, SP4, Crypto AG – a few, alleged, Echelon keywords. Please add some to your email signature. Full list: http://www.serendipity.li/cia/bz1.html
    http://www.nosoftwarepatents.com/

  2. #2
    Join Date
    Apr 2001
    Location
    SF Bay Area, CA
    Posts
    14,947
    I would be very wary of a lazy umount. There's nothing in the umount manpage that suggests that data actually gets forced out to the device before the "lazy" part kicks in -- so if you lazily umount the device, then unplug it right away (before the files get released), it's possible you'll still lose data.

    What's wrong with just doing a normal umount?

  3. #3
    Join Date
    Mar 2003
    Location
    UK
    Posts
    621
    Thanks. It usually says the device is busy.

    So if I do a normal umount I can then just pull it out without concern??
    MI6, Offensive Information, Hackers, Encryption, UFO, AOL, Infowar, Bubba, benelux, Ufologico Nazionale, domestic disruption, 15kg, DUVDEVAN, debugging, Bluebird, Ionosphere, Keyhole, NABS, Kilderkin, Artichoke, Badger, spookwords, EuroFed, SP4, Crypto AG – a few, alleged, Echelon keywords. Please add some to your email signature. Full list: http://www.serendipity.li/cia/bz1.html
    http://www.nosoftwarepatents.com/

  4. #4
    Join Date
    Apr 2001
    Location
    SF Bay Area, CA
    Posts
    14,947
    Quote Originally Posted by lugoteehalt
    Thanks. It usually says the device is busy.
    A lazy umount won't fix that problem. It will get the filesystem out of the main FS tree, but if something's keeping the FS open, then something's still keeping the FS open -- you still won't be sure that data gets flushed out to it.

    What you need to do is figure out what's keeping it open. Generally this is either a process that has a file open on this filesystem (which is something that lsof and/or fuser are fairly good at finding: check their manpages to find out how to search for any process with a handle open to a file on a given FS), or a process that has a directory on this FS as its current working directory (lsof and/or fuser may be able to give you this information fairly easily too).

    Alternately, just never start a program while you're in a directory on that FS, and don't try to umount it while a shell is sitting there. That's not enough to guarantee that no process has any directory on that FS as its current directory, but it might be enough in practice.

    In short: check the lsof/fuser manpages, and try using them to find the offending process. Then see if you can fix the issue; if not, just kill the process. (Assuming the process isn't something that's system-critical. If it's running as you, it's probably OK to kill.)

    So if I do a normal umount I can then just pull it out without concern??
    Yes. A normal umount will ensure that all the files are closed, and that all data gets flushed out to the device.

  5. #5
    Join Date
    Aug 2006
    Posts
    180
    Yes, but if lsof happens to be too much of a bother if one's in a real hurry, wouldn't using the "sync" command minimize the chances of losing data without cleanly unmounting it?

  6. #6
    Join Date
    Mar 2003
    Location
    UK
    Posts
    621
    Thanks bwkaz and xrx.

    More woried about destroying the stick. It seems easy enough to shut the programs using the flash, although a bit of fiddling about; so I'll just do that.
    MI6, Offensive Information, Hackers, Encryption, UFO, AOL, Infowar, Bubba, benelux, Ufologico Nazionale, domestic disruption, 15kg, DUVDEVAN, debugging, Bluebird, Ionosphere, Keyhole, NABS, Kilderkin, Artichoke, Badger, spookwords, EuroFed, SP4, Crypto AG – a few, alleged, Echelon keywords. Please add some to your email signature. Full list: http://www.serendipity.li/cia/bz1.html
    http://www.nosoftwarepatents.com/

  7. #7
    Join Date
    Apr 2001
    Location
    SF Bay Area, CA
    Posts
    14,947
    Quote Originally Posted by xrx
    Yes, but if lsof happens to be too much of a bother if one's in a real hurry, wouldn't using the "sync" command minimize the chances of losing data without cleanly unmounting it?
    Possibly. I'm not sure what kind of guarantees USB storage devices give you when you sync.

    Some cheap IDE drives, for instance, don't actually sync all the way through their internal write caches when the kernel tells them to sync -- which causes issues sometimes even when they get unmounted normally. And when some large USB devices actually only have 25% of the space they claim to have (for instance, https://www.redhat.com/archives/ext3.../msg00019.html), I wouldn't be surprised if they likewise cheated on the SCSI flush command.

    But I should note, I've never actually heard of that happening, it just wouldn't surprise me.

  8. #8
    Join Date
    Jun 2007
    Location
    London, U.K.
    Posts
    13
    I've heard similar regarding these USB's under Windows - yet I recall that when USB came in, it was said to be hot-swapable.....now, which is right?
    I can understand that if you pull it out whilst it's being written-to, you'll get a short file that won't read-back.... but I wouldn't expect it to be unusable - presumably when the interface was designed the possibility of pulling it out occurred to them?

    Does this mean that we need a physical cover over these drives so it has to be opened (thereby breaking the read/write operation, e.g. by a switch), before it can be removed....in much the same way a floppy "can" be removed during a write....grief, what happens to that?
    I guess no-one ever tried it.
    I have been using these flash-drives for a while and always wait for the process to stop, but Here (original postings) I understand the PC is still holding onto the drive by keeping it "busy"....eh?
    As the USB Specification has power to the socket, it is likely there is enough power present to conclude writing and so on - although the file itself may be lost.

    I use Ubuntu6.06 - but as a beginner am not aware how or what unmounting is (it was a old term re-used?) - nor do I know where this might be performed
    - Is it an act of hardware-removal (by pulling the link?)
    - or is it a software action?
    In WIn98SE (hiss!),pulling the USB device just removes it from the icons in "My Computer" and there is no warning to 'remove safely'
    - which is surprising given that Win98SE is pretty stupid when its own failings cause a malfunction and it blames the User for not "switching-off" at the Start Menu!
    That's it.

  9. #9
    Join Date
    Mar 2003
    Location
    UK
    Posts
    621
    Quote Originally Posted by jl_harry12
    I use Ubuntu6.06 - but as a beginner am not aware how or what unmounting is (it was a old term re-used?) - nor do I know where this might be performed
    - Is it an act of hardware-removal (by pulling the link?)
    - or is it a software action?
    In WIn98SE (hiss!),pulling the USB device just removes it from the icons in "My Computer" and there is no warning to 'remove safely'
    - which is surprising given that Win98SE is pretty stupid when its own failings cause a malfunction and it blames the User for not "switching-off" at the Start Menu!
    That's it.
    $ umount /media/flash removes the stuff on the flash drive from the directory tree, assuming /media/flash is in /etc/fstab. If it says 'device is busy' then umount -l ditto, I think it's -l, see above.

    $ mount ditto hangs stuff in stick from tree so you may use it.
    MI6, Offensive Information, Hackers, Encryption, UFO, AOL, Infowar, Bubba, benelux, Ufologico Nazionale, domestic disruption, 15kg, DUVDEVAN, debugging, Bluebird, Ionosphere, Keyhole, NABS, Kilderkin, Artichoke, Badger, spookwords, EuroFed, SP4, Crypto AG – a few, alleged, Echelon keywords. Please add some to your email signature. Full list: http://www.serendipity.li/cia/bz1.html
    http://www.nosoftwarepatents.com/

  10. #10
    Join Date
    Apr 2001
    Location
    SF Bay Area, CA
    Posts
    14,947
    Quote Originally Posted by jl_harry12
    yet I recall that when USB came in, it was said to be hot-swapable.....now, which is right?
    Both.

    USB as a bus specification is hot swappable. Any USB device can be plugged in at any time, either before or after the OS is up, and it'll get recognized and have its drivers loaded.

    Also, non-storage devices can be unplugged at will, also. You'll lose whatever capability they were providing, but the OS must be able to handle losing the device at any time.

    Now, storage devices are a bit of a different story. Yes, at a physical bus layer, the devices can be unplugged whenever. But no OS's usb-storage driver (that I know of) can handle hot-unplugging a mounted USB device. They all cache data for speed (the USB bus, while relatively fast, is way, way slower than RAM), and you need to unmount (or on windows, "safely disconnect") the device so that cache gets flushed.

    I don't know if the usb-storage spec says anything about host caches, but if it does say anything, then this behavior is allowed.

    (For comparison, floppy drives don't cache any data on windows. And it takes forever to write to them. Linux does cache floppy data, flushing it on umount (just like any other disk-type device), and it's way way faster to work with. You just have to remember to umount it before swapping disks. I think it'll work to yank a floppy, then put it back in and unmount, but you have to make sure no other writes try to go to the disk while it's out.)

    I can understand that if you pull it out whilst it's being written-to, you'll get a short file that won't read-back
    Um, not quite. No OS actually stores files like that. (Even DOS, whose disk allocation scheme was horribly inefficient when you deleted files ("fragmentation"), didn't require each file to be stored in contiguous blocks.)

    When a program says "write this buffer of data to this file, starting at this offset", the OS breaks the buffer into chunks (well, usually), and writes out each chunk as it gets time. It has to make sure that subsequent reads return the correct data, but the data doesn't actually hit the disk until much much later. (Journaled filesystems make this wait even longer, because they write to the disk twice: once to the journal, and again to the real file blocks.)

    So when you write out an entire file, the OS doesn't write it out one byte at a time, in order. It writes it out in whatever order it feels like. So if you remove the disk in the middle of a write, you won't end up with the first half of the file; you'll end up with some random half of the file scattered throughout the file's extents.

    but I wouldn't expect it to be unusable
    There's also the question of on-disk filesystem structures. If these are wrong, the FS is useless. That's what fsck and journaled FSes are for.

    Does this mean that we need a physical cover over these drives so it has to be opened (thereby breaking the read/write operation, e.g. by a switch), before it can be removed....in much the same way a floppy "can" be removed during a write....grief, what happens to that?
    I guess no-one ever tried it.
    It trashes the floppy disk's data structures. Well usually; depends on the filesystem on the floppy. FAT12 will probably be OK, but that's because FAT12 is dumb.

    On the switch: That's exactly what USB has. When the device loses power, because it's been pulled out of the +5/GND power rails on the USB port, that's the switch going off. The OS detects this, but it can't do anything about it at that point, because the device is already gone.

    but Here (original postings) I understand the PC is still holding onto the drive by keeping it "busy"....eh?
    No, but the OS's cache holds data that hasn't necessarily been written to the device.

    Yanking the device out before this data gets flushed means you can either lose the file, or the FS, depending on exactly what the data is.

    As the USB Specification has power to the socket, it is likely there is enough power present to conclude writing and so on - although the file itself may be lost.
    Doesn't help if the OS hasn't issued a write request for the critical data blocks when you yank the device out...

    I use Ubuntu6.06 - but as a beginner am not aware how or what unmounting is (it was a old term re-used?) - nor do I know where this might be performed
    - Is it an act of hardware-removal (by pulling the link?)
    - or is it a software action?
    Software. See lugoteehalt's post, above.

    If Ubuntu doesn't push mount and umount in your face, it's probably OK, but it should have some way for you to tell it that the device should be safely removed (or however they word it). This will call the umount program, which unmounts the device safely.

    In WIn98SE (hiss!),pulling the USB device just removes it from the icons in "My Computer" and there is no warning to 'remove safely'
    Yep. But look at 2000 Pro -- when you unplug a USB disk, it growls at you and says "please don't do that, you might lose data; please "safely remove" the disk before unplugging it". And I have lost data several times on 2000 by yanking the device out before telling the OS to disconnect it.

    (I think they've gotten rid of the warning on XP now, but I don't use XP, either. So I don't do much experimentation with USB storage devices on it. Perhaps it doesn't cache usb-storage data? If not, that seems like a dumb decision to me.)

  11. #11
    Join Date
    Aug 2006
    Posts
    180
    (I think they've gotten rid of the warning on XP now, but I don't use XP, either. So I don't do much experimentation with USB storage devices on it. Perhaps it doesn't cache usb-storage data? If not, that seems like a dumb decision to me.)
    Yes, they've seemed to have disabled that by default now, which I presume would be almost the same as having the ,sync, option in mount/fstab in linux instead of the default async. Surprising, considering how USB flash disks have limited write cycles that can only be strained without write caching.

  12. #12
    Join Date
    Jun 2006
    Location
    Haarlem, Holland
    Posts
    129
    When I remove the stick savely in windows, the light on the stick turns off, but when I unmount it in linux, the light keeps burning.

    I've asked google a lot, but that didn't gave me any info. Is there somebody who knows if it is possible to turn the light of in linux?

  13. #13
    Join Date
    Apr 2003
    Location
    UK
    Posts
    1,181
    If you use the eject command on whichever device the USB stick shows up as, then the light may turn off. However as long as the USB stick has been unmounted then it is safe to remove.

  14. #14
    Join Date
    Jun 2006
    Location
    Haarlem, Holland
    Posts
    129
    I have tried eject /media/usbstick, but that had the same result as unmount, the light kept burning.

    I am so intrested in turning of this light, because I want to sync a directory on the stick with one on the system, and not interact with the screen to see if everyting is ready and I can remove the stick.

  15. #15
    Join Date
    Apr 2001
    Location
    SF Bay Area, CA
    Posts
    14,947
    I just plugged in a USB storage device (running kernel 2.6.21.3, self-compiled), and the light blinked for a bit, then turned off. When I mounted it, the light blinked for a bit again, then turned off. Reading or writing data made the light blink, then turn off. Unmounting the disk make the light blink a bit, then turn off.

    At no time did the light ever stay on. I suspect, but am not sure, that it depends on how the USB device is wired. I doubt that there's any way to force the device to turn off its light in the spec, though I don't know for sure. I suppose you could tell the hub to remove power from the port that the device is plugged into; this would be the same (from the USB bus's point of view) as unplugging the device, but could maybe happen under software control. If the kernel exposes some kind of interface for that; I have no idea if it does.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •