Page 1 of 2 12 LastLast
Results 1 to 20 of 32

Thread: How can lightroom possibly corrupt files during import?

  1. #1
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4

    How can lightroom possibly corrupt files during import?

    Using...

    Lightroom 3.4.1
    Windows 7 (64bit) - fully up to date with patches
    System Rating 5.9
    Processor i7 CPU - 930@2.8Ghz - no overclocking
    Memory - 6 GB
    Display Adaptor - Nvidia GeForce GTX 460
    No yellow bits in device manager :-)

    Canon 5D Mk II and 7D
    Sandisk Extreme 60MB/s 32GB & 8GB
    Sandisk Card reader - directly plugged

    This happens all the time - but for example - yesterday I took 350 photos, when importing them directly from the CF card with Lightroom, seven of them were corrupt.

    However, if I manually copy the files to the HD and then import into lightroom - no corruption.

    I have used a hex comparison tool to compare the good and bad files and sure enough there are a couple of picked bits here and there.

    I'm a bit confused. I have some basic questions...

    How can Lightroom introduce corruption at this stage? Seems implausible?

    If there was a corruption - why on earth wasn't it detected at transfer time? Surely there is some sort of basic checking, CRC at a minimum - yes/no?

    Thanks,
    RzzB
    Last edited by RzzB; 3 Jul 2011 at 4:04 pm.

  2. #2
    Lightroom Guru
    Join Date
    Oct 2007
    Location
    Worcester, MA
    Posts
    2,187
    Memory
    3gb - 4gb
    CPU Cores
    2
    Video Card
    nVidia
    LR Version
    5.3
    Camera
    Pentax
    File Format
    Raw
    Printer
    Offsite Photo Lab
    Likely bad CF card reader, cable or memory. Try different reader of direct from camera connection.

    Don
    Don Ricklin, MacBook Pro 15.4" 2.4Ghz Duo 2 Core, 4Gb RAM, 250Gb SATA HD, 10.6.4 , LR V5.3, Assorted Firewire & USB2 SATA Back-up Drives.
    Gear: Pentax K-7, Sigma 17-70 2.8-4.5; 75-300 and 175-500, Zigview R
    'I was older then, I'm younger than that now' .... 'Oh, you mean Nancy'!

  3. #3
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Don,

    Thanks for your response.

    This occurs with different card readers, connected direct and through a hub. The host machine is very stable so I doubt that I have a memory problem of this magnitude.

    Let me be very clear ...

    When I transfer the files using Windows Explorer there are NO corrupted files.

    When I transfer the files using Lightroom I get corrupted files.

    It's as if Lightroom is using a different transfer mechanism that is causing corruption.

    What is worrying me more is - why aren't the corruptions detected and an error raised? I believe USB transfers have a CRC checking / retry mechanism built in for most transfer modes. Is Lightroom perhaps using a modes that has this turned off?

    Thanks,
    RzzB

  4. #4
    Lightroom Guru
    Join Date
    Oct 2007
    Location
    Worcester, MA
    Posts
    2,187
    Memory
    3gb - 4gb
    CPU Cores
    2
    Video Card
    nVidia
    LR Version
    5.3
    Camera
    Pentax
    File Format
    Raw
    Printer
    Offsite Photo Lab
    Okay, time to post screen shots of some of these files in LR so we can see what corruption you are talking about.

    Don
    Don Ricklin, MacBook Pro 15.4" 2.4Ghz Duo 2 Core, 4Gb RAM, 250Gb SATA HD, 10.6.4 , LR V5.3, Assorted Firewire & USB2 SATA Back-up Drives.
    Gear: Pentax K-7, Sigma 17-70 2.8-4.5; 75-300 and 175-500, Zigview R
    'I was older then, I'm younger than that now' .... 'Oh, you mean Nancy'!

  5. #5
    Lightroom Queen / Owner
    Join Date
    Sep 2007
    Location
    Southampton, UK
    Posts
    15,651
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    16gb
    CPU Cores
    4
    Video Card
    nVidia
    LR Version
    5.5
    Camera
    Canon
    File Format
    Raw
    Printer
    None
    Hi Rzzb, welcome to the forum!

    I can certainly understand your concern, and that is a very odd problem. It's not one that I've heard any other reports of, so I would still look at hardware amongst the other possibilities.

    I assume you've ruled out the other possibilities, such as a hard drive, and you've run a low level memory test such as Memtest? Perhaps LR is accessing memory that has a minor glitch, whereas Explorer isn't hitting it? LR's very good at finding the most hidden minor memory glitches, and it could cause corruption, so it would be worth ruling out.

    And how are you discovering that the transferred files are corrupted? Is LR telling you, or are you spotting it in the previews? Or just in the hex comparison that you're seeing it?
    Victoria


    Lots of questions? Adobe Lightroom 5 - The Missing FAQ Book available now - PDF, ePub, Kindle & Paperback Discount code: LRF2011

    Website: Lightroom Queen Blog - Follow me on Twitter as @LightroomQueen - Find me on Facebook

  6. #6
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Many thanks for your responses guys.

    I'm detecting the corruption because some of the images in the preview window are a mess - see the attached. When I inspect these files (hex comparison) with the versions that I transferred manually I can see that there are some picked/dropped bits. I can see no pattern sensitivity - they look pretty random.

    I have done nothing to rule out hard drive problem, but I can't see how this could be associated with a hard drive problem. Not sure what I should do? Thoughts?

    I haven't run memtest recently. I did run all sorts of test when I originally built the machine (6 months ago) - Memtest, Burnin, Prime95 - just to make sure I had a stable system. But I agree - it would be sensible to run Memtest again to be sure. I will do this in the near future and report back - might be in a couple of days.

    I do agree that this feels like a hardware problem, but it seems to be exacerbated by Lightroom.

    Possibly memory - I'll do some testing as mentioned above.

    The other thing I need to do it double check all the failing scenarios - just to make sure. ie Cables, readers, hubs etc. I'll probably do this over the weekend.

    What concerns me is that none of the software involved is detecting the corruption and raising an error.

    Many thanks for your help,
    RzzB
    Attached Thumbnails Attached Thumbnails Capture 2.JPG   Capture 1.JPG  

  7. #7
    Moderator
    Join Date
    Oct 2007
    Location
    Paradise
    Posts
    4,623
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    8gb
    CPU Cores
    4
    LR Version
    5.5
    Camera
    Nikon
    File Format
    Raw
    Printer
    Epson
    What is the build number of Lightroom? Check in system info in the Help drop down menu.
    This looks similar to the issue with 3.4.
    Geoff Walker,
    New Zealand.


  8. #8
    Lightroom Guru
    Join Date
    May 2011
    Location
    Canada
    Posts
    1,539
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    Memory
    8gb
    CPU Cores
    4
    Video Card
    nVidia
    LR Version
    5.6
    Camera
    Nikon, Fuji, Olympus, Canon :)
    File Format
    Raw
    Printer
    Offsite Photo Lab
    Geoff,

    Wasn't that problem with jpegs from a non-Canon?

    Hal

  9. #9
    Moderator
    Join Date
    Oct 2007
    Location
    Paradise
    Posts
    4,623
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    8gb
    CPU Cores
    4
    LR Version
    5.5
    Camera
    Nikon
    File Format
    Raw
    Printer
    Epson
    I have no idea Hal, but it looked like something similar I think.
    Geoff Walker,
    New Zealand.


  10. #10
    Lightroom Guru
    Join Date
    Jun 2009
    Location
    Zurich, Switzerland
    Posts
    3,207
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    Memory
    More than 8gb
    CPU Cores
    6
    Video Card
    ATI
    LR Version
    4.2
    Camera
    Canon
    File Format
    Raw
    Printer
    HP
    The issue was noticed with images from a HP PhotoSmart R607, but you never know ....

    Beat
    Beat Gossweiler

    Canon 1Ds MkII | Canon 5D MkII | CANON EF 24-105mm F/4.0 L IS | CANON EF 100-400mm F/4.5-5.6L IS | CANON EF 100mm F/2.8 Makro
    HP Color Laserjet CP3505X

  11. #11
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Thanks for your interest gents.

    Lightroom version is 3.4.1 - see first post for other info.

    One thing I haven't mentioned is that these are all raw CR2 files. I only shoot in raw.

    Thanks,
    RzzB

  12. #12
    Moderator
    Join Date
    Oct 2007
    Location
    Paradise
    Posts
    4,623
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    8gb
    CPU Cores
    4
    LR Version
    5.5
    Camera
    Nikon
    File Format
    Raw
    Printer
    Epson
    I have another example here from a forum that looks very similar to yours and funnily enough from a Canon 40D.
    I seem to recall it being something to do with the drive or connections.
    Did you move them from one drive to another when importing? Where do you store your files - on an eternal drive???
    Geoff Walker,
    New Zealand.


  13. #13
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Quote Originally Posted by Kiwigeoff View Post
    I have another example here from a forum that looks very similar to yours and funnily enough from a Canon 40D.
    I seem to recall it being something to do with the drive or connections.
    Did you move them from one drive to another when importing? Where do you store your files - on an eternal drive???
    Geoff,

    Many thanks for your response - could you post a link to the other example please. My cameras are a 7D and 5D but similar in respect of being Canon.

    The importing is done from a CF card using a card reader directly to my D: drive which is a 1T mirrored raid drive. I do have external backup storage in the form of Synology DS210 boxes but I never import to them.

    I have a non Raid drive in my machine - perhaps I'll create a new catalogue and import to that drive just to rule out any Lightroom/Raid problems.

    Thanks for the thought...

    Regards,
    RzzB

  14. #14
    Moderator
    Join Date
    Oct 2007
    Location
    Paradise
    Posts
    4,623
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    8gb
    CPU Cores
    4
    LR Version
    5.5
    Camera
    Nikon
    File Format
    Raw
    Printer
    Epson
    I have the file on my computer and it looks like this:
    Click image for larger version. 

Name:	IMG_9058.jpg 
Views:	4 
Size:	70.1 KB 
ID:	764
    Interesting it was on a D drive as well. I'm trying to recall what the solution was but am too busy at the moment to search. I'd think your idea to test on your C drive would be a good one!!
    Geoff Walker,
    New Zealand.


  15. #15
    New Member
    Join Date
    Jul 2011
    Posts
    6
    Experience
    Advanced
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Quote Originally Posted by RzzB View Post
    Geoff,
    ...... snip.....
    I have a non Raid drive in my machine - perhaps I'll create a new catalogue and import to that drive just to rule out any Lightroom/Raid problems.
    Hmmm... I'm now beginning to regret starting this thread :-(


    I tried using the non-raid drive - and hey presto no corruption!


    However, I retried to the raid drive also and guess what - it worked ok there too.


    In the last couple of days I have transferred the same 350 images using all sorts of hardware and I have seen just one corrupt file.


    So - something has changed - and I have been racking my brain trying to work out what.


    Oh well - I think I now have to wait till I see the corruptions again and look at what is happening on my machine at the time.


    Thanks,
    RzzB

  16. #16
    Lightroom Queen / Owner
    Join Date
    Sep 2007
    Location
    Southampton, UK
    Posts
    15,651
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    16gb
    CPU Cores
    4
    Video Card
    nVidia
    LR Version
    5.5
    Camera
    Canon
    File Format
    Raw
    Printer
    None
    That sounds like an excellent plan RzzB. Funny how things fix themselves when you say them out loud!
    Victoria


    Lots of questions? Adobe Lightroom 5 - The Missing FAQ Book available now - PDF, ePub, Kindle & Paperback Discount code: LRF2011

    Website: Lightroom Queen Blog - Follow me on Twitter as @LightroomQueen - Find me on Facebook

  17. #17
    Lightroom Guru
    Join Date
    Nov 2007
    Location
    Port Deposit, Maryland USA
    Posts
    6,293
    Experience
    Intermediate
    Primary OS
    Windows 7 64-bit
    Memory
    4gb - 8gb
    CPU Cores
    8
    Video Card
    ATI
    LR Version
    4.2
    Camera
    Canon
    File Format
    Raw
    Printer
    HP
    Good news.

    I'd say we don't like to come across as Lightroom fanboys, but I don't think I've ever seen convincing evidence that Lr can cause raw file corruption at the import or editing stages. Lr never touches the raw files at all (with one arcane exception) and uses the OS services exclusively for file operations.

    But if you just flat out say that, it sounds like defending an Adobe who can do no wrong, which is patently false.
    ......brad s Win7x64 (HPE) / i7-920 / 6GB / 2.67 GHz / tons o'drives / Canon/Sigma gear

  18. #18
    Junior Member
    Join Date
    Jul 2011
    Posts
    11
    Experience
    Intermediate
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Quote Originally Posted by RzzB View Post
    ... I tried using the non-raid drive - and hey presto no corruption!
    However, I retried to the raid drive also and guess what - it worked ok there too.
    In the last couple of days I have transferred the same 350 images using all sorts of hardware and I have seen just one corrupt file.
    ...
    It seems this is the same problem I have:
    when importing 400 cr2 files directly from the camera into LR, one (1) file gets corrupted. It does not happen every time.
    And: when I transfer the files using Windows Explorer there are NO corrupted files.

    Lightroom 3.4.1
    Windows 7 (64bit)
    Canon 5D Mk II
    Sandisk Extreme 60MB/s 16GB

    Checked memory and hard drive.
    The corrupt file seams to appear after 6-8 GB of the 10 GB has been transfered.
    I have also used a hex comparison tool to compare the good and bad files, and there is a certain pattern: a multiple of the 6th and 14th byte are replaced... somewhere in the last 20-40% of the file content.
    Last edited by Roomservice; 17 Jul 2011 at 4:31 pm.

  19. #19
    Junior Member
    Join Date
    Jul 2011
    Posts
    11
    Experience
    Intermediate
    Primary OS
    Windows 7 64-bit
    LR Version
    3.4
    Maybe the file corrupt problem is related to this problem:

    Today I took some test shots with camera setting RAW+jpg, to another empty memory card.
    When importing them to LR it hanged. I made that import repeatedly: sometimes LR freezes, sometimes not. When it freezes you can see in windows explorer that it has lost the connection to the card. Does Lightroom cause the card to disconnect?

    But there are no problems when copying with Windows explorer.
    So RAM, hard disk, memory card, camera and usb cable seems ok (and RAM + hard disk are tested).
    But when Lightroom is involved...

    Any suggestions would be appreciated

  20. #20
    Lightroom Guru
    Join Date
    Jun 2009
    Location
    Bellaire, TX USA
    Posts
    7,384
    Experience
    Power User
    Primary OS
    Mac 10.9.x Mavericks
    Memory
    16gb
    CPU Cores
    4
    Video Card
    AMD Radeon HD 6770M
    LR Version
    5.5
    Camera
    Nikon D800
    File Format
    Raw
    Printer
    Canon
    Roomservice, welcome to the forum. Unless you have actually used a second USB cable, you can not eliminate 'Badd Cable" from the list of possible causes. By using two different cards, you probably have eliminated the "Bad Card" cause (Or validated that you have two bad cards). The camera needs a fresh battery to begin the transfer, Using a real card reader instead of the camear can help isolate the camera/battery as a cause.

    If you can successfully copy images to the computer and Import them into LR from the computer location, you can then remove the camera/card/cable as a source of the problem.

    I would also like to note that with flash memory, it very important to give the camera time to write its buffered image data to the card before shutting the camera off or ejecting the card from the camera. Either operation while the file is still being written can result is a corrupt image.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •