wondering how many photos you can have in your LR catalog before its an issue

[bjc]

lately i notice that when i take my photos off my cf card it takes a really long time to do this... doesnt seem to matter what size the card is or how many photos i have taken ...it just takes a long time and i started thinking maybe i have too much in the catalog? do you usually take things out of LR periodically or when you think you are done with the photos... they would still be on my ehd and i could always import them back into LR..is this the cause of the rreally slow importing from my cf card? would love some comments or suggestions from those in the know..thanks

 

[b_gossweiler]

The number of images in your catalog mostlikely is not the cause of what you are experiencing. Victoria here just mentioned her working catalog with 100K images, and the main one with 600K ....

Which part of importing takes a lot of time? The copying from card (before the preview rendering starts) or the preview rendering (after you see the grid view again?

Beat

 

[clee01l]

You will find several people chiming in with hundreds of thousands of image in their catalog.. The issue that you describe has nothing to do with the number of issues in your catalog but more to do with the size of your ACR cache (if you are shooting RAW), the Preview cache settings on the Catalog Settings {File Handling} tab, and the available free space on your primary HD for LR to work. LR makes use of a lot of temporary storage when loading RAW data from cards.

Another contributor is how you are accessing the CF card? A USB card reader is preferable over the camera plugged in to the PC a built-in card reader has advantages over that and the speed of the card itself can have an affect on the transfer speed.
Also your HD speed (5400rpm vs. 7200rpm) and whether the images are being transferred to a local drive or one connected by a cable.

I'd be interested in knowing some detailed H/W specs. In addition to the above, How many cores, How much RAM?

 

[dj_paige]

lately i notice that when i take my photos off my cf card it takes a really long time to do this... doesnt seem to matter what size the card is or how many photos i have taken ...it just takes a long time and i started thinking maybe i have too much in the catalog?

Have you optimized your catalog lately? File->Optimize Catalog. That's the first thing to try when things start running slowly. Too many photos in the catalog is rarely a cause of slowness.

do you usually take things out of LR periodically or when you think you are done with the photos... they would still be on my ehd and i could always import them back into LR

No. This defeats the purpose of the Lightroom Library Module. Advice: never re-import. Just don't even think about doing it any more, it is not a good idea.

 

[bjc]

answers to questions

i have not optimized..didnt even know about that..and yes it is when i am importing the start to finish of the importing takes much longer than i remember it used to...yes i am shooting raw with canon 7D ... and yes i am using a card reader that is built into the computer

so what do you suggest i do? i guess i should do the optimize first right? i will try that

and glad to hear that i dont have to remove any photos from my LR

edit...i just went to optimize and it said last optimized 7/13/11 so i guess it optimizes when it does the backup when you go to close and it says you should backup? so i guess that isnt the issue

any other ideas..thanks

 

[clee01l]

The remaining unanswered questions:

• How much free space remains on your primary drive where the temp folder resides i.e what percent is it?
• How fast (rpm) is/are the drive(s) that stores ACR Cache, LR Previews and the Master copies of the original.
• How much RAM do you have? How many Cores?
• What are your preview cache settings in the Catalog settings panel?
• What is the maximum size of ACR Cache (in Preferences)
• What is the capacity of the CF Card that you are importing?
• At what speed is that card rated?

 

[bjc]

• How much free space remains on your primary drive where the temp folder resides i.e what percent is it? it shows 1.22 TB free of 1.35 TB on my C drive that is where my temp folder is ..and btw i store all my photos on ehd in case that matters
• How fast (rpm) is/are the drive(s) that stores ACR Cache, LR Previews and the Master copies of the original. dont know the answer to this question? how do i find it? i can try to ask someone
• How much RAM do you have? How many Cores? dont know this one either...how do i find it?
• What are your preview cache settings in the Catalog settings panel? not sure i understand the question but i went to catalog settings preview cache its says 1440 px medium and discard after 30 days
• What is the maximum size of ACR Cache (in Preferences) not sure i understand the question but i went to preferences and see something that says camera raw cache settings 10.0 GB is this what you mean?
• What is the capacity of the CF Card that you are importing? it seems to happen with all sizes cf card ....i usually use an 8 GB but it happens with my 2 GB also
• At what speed is that card rated? not sure how you know the speed of the card? can you tell me? the cards i have are san disk extreme III, san disk ultra II

thanks so much for helping
[/QUOTE]

 

[Jim Wilde]

Can I just interject a quick question? I noted in one of your posts that you are using a 7D, just wondered how long you've been using it and if the 'slow imports' started around the time you got the 7D. IIRC the 7D is an 18 megapixel camera, which means pretty big files, which generally means slower imports from previous camera (I upgraded from a 10mp 40D to a 21mp 5DII and the import speed difference is definitely noticeable). Just wondered is that could be a factor?

 

[Hal P Anderson]

Could you give us an actual time that an import takes? Tell us how many images and how long it takes.

A lot of Cletus's questions can be answered by going to the Help/System Info... menu item.
There's a copy button there, so you can copy and then paste into your reply. Be sure to blank out your serial number before you post.

Hal

 

[bjc]

info from LR

Lightroom version: 3.4.1 [749592]
Operating system: Windows 7 Home Premium Edition
Version: 6.1 [7601]
Application architecture: x64
System architecture: x64
Physical processor count: 6
Processor speed: 3.2 GHz
Built-in memory: 8183.8 MB
Real memory available to Lightroom: 8183.8 MB
Real memory used by Lightroom: 188.1 MB (2.2%)
Virtual memory used by Lightroom: 195.0 MB
Memory cache size: 0.0 MB
System DPI setting: 96 DPI
Desktop composition enabled: No
Displays: 1) 1920x1080
Serial Number:

Application folder: C:\Program Files\Adobe\Adobe Photoshop Lightroom 3.4
Library Path: N:\Lightroom CATALOG edh\Lightroom 3 Catalog.lrcat
Settings Folder: C:\Users\Bobbie\AppData\Roaming\Adobe\Lightroom

 

[bjc]

and cant say how long the upload is...will try to take some new photos and clock it..but i know i leave my computer and go do a thing or two and it still may be uploading ...not sure..will have to get back to you on this one

and not sure if it started from when i got the 7D but i dont think so i have had the camera awhile

 

[clee01l]

Thanks bjc. What I've been trying to determine is if there are any H/W bottlenecks. From the information provided I can not see any. Still to be determined is the speed of your HD 5400rpm is the low end of what's common , but 7200rpm is desired. This should be listed in the specs that came with the HD/computer.

What's left is the interesting point that Jim Wilde has brought up. If you have just recently started shooting RAW then the time spent in ACR has to be factored in If you have always shot RAW then there is nothing new here. Another could be the number of images that come off of the card. If before, you were shooting fewer phptps before import, it could just seem longer.

Based upon your current settings, I would make the following recommendation: Increase your ACR Cache to 30-50GB. You have lots of free space and while it won't improve import performance, it will meand that LR will need to revisit ACR less often to regenerate image data.

Somethings your might consider if yo have multiple drives would be to segregate your ACR Cache, Catalog and master files onto separate drives. They may be competing with each other fro R/W time on the one HD. This is not a remedy that should be required and any improvement would be slight.

One other thing that you can do . LR is a multi threaded, multitasking application. LR can continue to import iamges as a background task while you move on to your workflow step 2 as soon as the first image appears in the "Previous Import" collection. My workflow begins: Import, Cull, Assign additional Keywords, titles and Captions... I begin reviewing images for rejects and picks as soon as I have an image to work with. I can't even tell you how long it take to import a 16GB card full of 25Mb DNG images.

Speaking of DNG, (My camera writes DNGs btw) Are you also converting CR2s to DNG on Import? If so, Why? Also are you taking advantage of LRs option to make a temporary second copy to another drive on import? Have you always had this option checked or unchecked?

 

[bjc]

nothing really helps

i have always shot raw..and i dont do dups... and it doesnt matter if i am importing a small number or large number..so i guess i dont know what to do now

 

[bjc]

btw..how do i increase the cache to 30-50 GB as you recommended..thanks for telling me how to do that

 

[clee01l]

btw..how do i increase the cache to 30-50 GB as you recommended..thanks for telling me how to do that

Menu: {Edit}{Preferences}{File Handling (tab)} Camera RAW Cache Settings section. (This is where you found the 10GB that you reported.)

As for your slowness problem, I'm almost out of ideas. If you do not create two copies on import or convert your CR2s to DNG, Everything should flow as quick as the OS can manage to read the card and write to the HD. There is one other possibility and that is virus scanning software. I would make sure virus scanning is turned off for the file type CR2 and your whole catalog folder including the preview cache folder as well as the ACR cache folder. You do not need to be scanning those and if your camera is sending out virii you have bigger problems that how fast you camera card loads.

 

[Jim Wilde]

Could you give us an actual time that an import takes? Tell us how many images and how long it takes.

I think until we actually get some objective data then I really don't see how we can help too much. We now know we are dealing with Raw files from a Canon 7D, but we still need some more specifics, i.e. during the next import you do could you fill in the following blanks:

1. Number of images in the Import.
2. Which CF card you are using.
3. Once you have finished setting up the Import, we need the total time from the second you press the 'Import' button to the time that the 'Copy and Import' phase is completed (watch the progress indicator at the top left of the screen).
4. We then need the time from the end of the 'Copy and Import' phase to the end of the Preview generation phase.

If you could get us that info it would certainly help us understand if there is a problem.

 

[bjc]

ok

thanks will get back to you after the next import

 

[bjc]

i just did two imports..same card... sandisk extreme III 8 GB

the first one was 65 photos and from start of import to end it tool 10 minutes

the second was 16 photos and from start of import to end it also was 10 minutes

thanks for your thoughts

 

[dj_paige]

Was the 2nd import started before the first import had finished generating thumbnails?

 

[bjc]

just did two imports from the same card...i didnt time the first one but it was very long... this is an 8 GB sandisk card used with a canon 7D shooting raw... i started the import at 2:36 and it still isnt done yet, and its 2:53 now so far there are 67 photos... and i think it is about 2/3 done according to the meter under copy and import photos... is this a normal time ..it seems to be so much longer than it used to be....what is the issue? any thoughts please or info you need

edit i am now finally done... its 3:25 took an hour and there are 125 photos..IS THIS NORMAL??????


ps...just took 20 photos from my iphone and it took a nano second..of course they arent big files but thought i would add that info

 

[Jim Wilde]

Would still like to have the timings for both stages of the Import....the first stage does the 'copy and import', i.e. the pictures are copied from the CF card to your (external) hard drive and added to the LR catalog. The second stage then renders the previews. Would like to know the time for each stage (and also need to know what type of previews you render in import).

Having said that, I would certainly agree that something is NOT normal with your imports. I just setup a test import, trying to replicate your setup as best I can....I imported 120 pictures from a CF card, copying them to a USB2 connected hard drive (7200rpm), and rendered 1:1 previews. The pictures were from my 5DMkII, thus a bit larger than those from your 7D. The first copy stage took 3 minutes 50 seconds, the preview rendering took a further 6 minutes 35 seconds, total just over 10 minutes. So your 1 hour certainly seems excessive.

Problem will be identifying the cause, and this may well have to be trial and error. Probably the first thing I would do is try importing directly from the camera using the USB cable, and compare the timings. We had a similar problem with another member a few months back and it turned out to be a very slow card reader, so suggest you try bypassing that first.

 

[Replytoken]

I am wondering if the problem lies with the card reader or USB port. Perhaps it is connecting with the PC at USB 1.0 rates? I would try another USB port as well as another card reader.

--Ken

 

[bjc]

hoping someone can see this post about how long it is taking me to upload from cf card..see above post ..any ideas?

 

[bjc]

update...my problem with slow upload is on my windows 7 computer.. and ehd... i am now uploading same cf card,new photo shoot, same ehd , but old vista computer... and it took 4 minutes to do 42 raw photos shot on same canon 7D..so what is the story ? help please

havent had time to try to upload directly from camera..and have to find the chord to do it... not at that windows 7 computer for a few days..but will try that

 

[Jim Wilde]

OK, let us know how that goes. Alternatively, try a different card reader as Ken suggested if you have access to one.

I think we're all thinking along the same lines, and that is focussing on your card reader.....so bypassing that allows us to test that theory.