underwatermacro

Photo Storage, Backup, and Workflow

Posted on by in underwatermacro and tagged , , , , , , , , .

Diving photographers and photographers in general take a lot of photos and videos and have different backup and long-term storage needs than the average person. This is especially true when you start recording 4K video.

I have lots of friends that have tragically lost photos over time, it’s usually because of the following reasons:

  • Laptop died.
  • External USB drive was dropped and doesn’t work anymore.
  • Ran out of space and had to delete files.
  • Accidentally deleted files.
  • Phone dropped in the ocean.

I’m also amazed how many professional and semi-professional photographers simply do not take care of their photos and have accidents where they lose years of their digital life.

But I’m an IT nerd and my life revolves around saving data and keeping computers from ruining my life (I know, it’s a cynical view of things).

So here is how I manage files, backups, and disks….

Equipment

Macbook Air mid-2015, 500GB SSD. The laptop that goes everywhere with me.

2TB Samsung T5 solid-state USB external drive. Portable backup, it also goes everywhere with me.

Synology DS 1815+ 8-bay Network Attached Storage (NAS) and 8x8TB drives (NAS-specific Seagate Ironwolf drives) in a Synology Hybrid RAID with 2 redundant drives, and formatted for BTRFS. The big beast in my spare bedroom with 42TB of useable storage. I realize this is a huge setup for most of you, but a nice 2-drive with SHR/RAID 1 or a 4-drive unit with SHR/RAID 5 works swimmingly for the average home user.

1 external 8TB USB drive for backing up the NAS.

USB hub with USB gigabyte ethernet adapter for Macbook, Cat 5e/6 ethernet cables, and a couple gigabit switches. I do have a 24-port managed gigabit switch for the NAS so that it can bond 4 ethernet ports into one super-fast network port.

Custom-built monster Linux desktop with M2 drive for root and 2x8TB hard drives in RAID1 for /home.

Second Macbook Air for my wife.

Considerations

All my cameras use SD cards. I have a reader built into the 2 Macbooks. I have a USB reader for my desktop. At some point in the future I’ll upgrade Macbook and have to figure out how to play this game with USB-C connectors.

I need a travel backup. SSD are the only USB/external drives that I have seen survive a busy travel schedule. Conventional spinning hard drives don’t take abuse and they’re heavy. I’ve killed even the “durable” hard drives.  I used to use a Samsung T3 250GB SSD but ran out of room because my Macbook has 500GB of storage. So I upgraded to the T5 2TB.

Time Machine Settings

No recent backup to NAS because I’ve been on the road since August 9th. I turned off “Back up Automatically” to make Time Machine Editor work.

MacOS uses a low priority for Time Machine backups so that it doesn’t slow down programs that are you are using. With photos and especially video, it takes a long time to make a backup. You can set the priority for Time Machine so that it works faster but possibly makes your computer seem slower and use an application to schedule backups for nighttime. This way, you get fast backups but only when you’re not using your Mac. Obviously, then you need to keep your Mac on overnight. =)

Time Machine Editor

Time Machine Editor Configuration. Only run at night when I’n not using my Mac.

At home, I need to share files across several computers. Network drives help with this. As a side benefit, it takes data off of my laptop or desktop hard drive and puts it somewhere more durable.

Synology NAS supports something called Synology Hybrid RAID (SHR). This allows you to upgrade the size of your storage by replacing the hard drives with larger drives (also as drives get cheaper over time). However, you have to do this one at a time and then allow the data to be synced to the new drive. For 8 drives, this could take a couple of weeks to fully replace all the drives. My NAS can also accept an expander unit of 5 drives.

Time Machine can backup to a NAS very easily. However, it’s slow over wifi. A USB to gigabyte ethernet dongle helps you a ton, and I have one of these at home permanently connected to a USB hub for my laptop.

You need some kind of naming scheme to let you find photos. I use “YYYY.MM Description” as folder names, so “2019.04 Malapascua” and “2019.07 Folly Cove” exist on my NAS.

My NAS has the Hyper Backup software that can backup to USB or another NAS. Think of it as Time Machine for NAS. I need this because when I transfer photos to NAS, the NAS becomes the single backup for everything. With the drive redundancy, I’m mostly OK, I’m making backups to protect against user error (oops, I deleted everything) and any kind of home disaster like fire, flood, etc. When I had 2 homes, I had a second NAS and would backup from NAS to NAS over a home-to-home (H2H) VPN. I like this approach but I only have one home nowadays. I looked at Synology C2 or Amazon Glacier as potentials for backing up my NAS and just set up Synology C2 while I was writing this.

My NAS also backs up photos and video from my phone and makes them available to any kind of laptop/desktop on the local network. It also makes photos on the NAS available to my phone via DS Photo software, so if I’m on the road and need to retrieve something from NAS I can.

Accessing NAS files via DS Photo.

I backup my Linux desktop and personal webserver using Synology Active Backup.

Workflow

Shoot and Transfer. If I’m traveling, I take the SD card out of my camera, insert it into the Macbook’s reader, and transfer it into a new folder named “YYYY.MM Description” under “Pictures”. Yes, I’m gangster and put videos in this directory too even though I have a “Movies” directory. If it’s a long trip or if I have different cameras to store images from (for a long trip I might have my G9, TG5, my wife’s TG4, a Paralenz, a GoPro, a drone, and even a 360° camera), I’ll also use subdirectories with “DD Camera Name”, so something like “2019.04 Malapascua/08 G9”.

Backup! Immediately after I transfer photos to my laptop, I plug in my Samsung T5 and do a backup with Time Machine. I won’t format my SD card until this backup is complete because until then I would have photos on only one device: my laptop hard drive.

Speaking of formatting SD cards, I do a full format before every dive to avoid any kind of filesystem corruption. But of course, after that first backup.

I will edit photos on my laptop during the trip. I have recently switched from Lightroom to darktable because it works on Mac and Linux. More importantly, it writes changes to a file called a “sidecar” in the same directory as the original so that my changes stay with the file across computers. You just have to remember to also transfer across the .xmp files when you transfer photos.

I plug in my T5 every other night on the road or so and let Time Machine backup to it while I sleep. That way my edits and other things on the laptop get backed up too. Night backups while you sleep is an awesome thing.

When I get home, I connect my laptop to the USB hub with ethernet adapter. I remove all of my pictures from darktable. Don’t worry, the changes are still on hard drive in the sidecar file. Then I copy the directory with all of my trip pictures and videos to a “photos” directory on the NAS. I transfer the entire directory so that my “photos” directory has some kind of order inside of it. I check that all the files were transferred to NAS by counting the number and loading the last handful of files to make sure they can be accessed via NAS.

photos drive

My nicely-organized photos directory.

I plug my T5 drive into the laptop and leave it there while my laptop is at home. That way, the laptop backs up to both the NAS and the T5 at night. I think Time Machine backs up first to the drive with the oldest backup, so overnight with both available it alternates backup drives.

If I dont need to use my laptop right away, I run Time Machine as soon as I transfer files to NAS to backup to either NAS or T5. It’s better if I let the laptop run overnight so that it does a backup to both locations. After I have verified that I have backups to both locations (usually the next day), I can safely delete the photos and videos from my laptop hard drive.

I can access photos and videos on the NAS from any computer as a network drive. On my Macbook, I mount the directory as a Windows share. On my Linux desktop, I mount via NFS with cache. In darktable, you can create a local copy of files that you are working on and then the local copy will be deleted when you remove the file from darktable. This saves you a lot of headache where NAS connections time out.

For using Lightroom on a network drive, you have 2 options:

  • Load the files into your Lightroom catalog when they’re on the laptop. That way, it’s faster reading off the local drive. Then you can do one of two things: #1 use Lightroom to transfer the directory to NAS; #2 transfer the files to NAS then have Lightroom “find lost files” on the NAS.
  • Transfer the files to NAS and then load them into the Lightroom catalog via the network share. This is slower but then you don’t have to worry about getting Lightroom to find them on NAS later.

I have a USB drive connected to the NAS to make a backup of the unique files that live on the NAS. Ie, photos and videos. I don’t make a backup of backups on the NAS like in the Time Machine directory… I already have 2 backups plus the original. One huge tip on Hyperbackup is that you need to unclick the “sleep USB drive after backup is complete” option because otherwise you won’t be able to make recurring backups to the USB unless you unplug/replug it.

20190826_1333446412836257628397925.jpg

USB drives can get diver gear tags, too. =) =) =)

See you Underwater!!

–Mike

Underwater Strobes

Posted on by in underwatermacro and tagged , , , , , , , , , , , , , , .

Let’s start with a simple fact: you can’t take photos without light. I know, this isn’t a huge surprise for most people. But this has a lot of implication for underwater macro photography. What light you have is usually weak and it’s a funny blue-green color. There are just a handful of ways to counter this: shallow dives with an adjusted white balance, a handheld dive torch, a camera-mounted wide-angle video light (or several), and strobes.

My Inventory

I own 3 different strobes.

Sea and Sea YS-01. This is a small strobe with not a lot of power. I own only one of these and have for quite awhile. It’s great for underwater macro with a compact like my trusty TG-5 and TG-4. Its size is good to add on to my camera setup without adding much weight or bulk, and as my blog readers will know, small components are the name of the game when I take the compact on a dive. However, the strength of these isn’t so good for wide-angle.

Sea and Sea DS-02. These are my usual strobes for diving with my Lumix G9. They have medium-strong power. I’ve had to replace a couple of them over time because of corrosion in the battery compartment. They work awesome for macro if I use a dome diffuser and put then right up along the sides of the lens. These work well for wide-angle if you use two of them and you’re 2 meters or so away from the subject.

Scubalamp P53 Pro. These are my new video lights with a strobe capability. I added these earlier this year. They’ve been a bit of a trial to use just as strobes, but they are great video lights. They are heavy and need some flotation in water which isn’t much of a problem with a wide-angle setup and a huge glass dome but they’re very negatively buoyant on a macro rig.

Diffusers

Diffusers are a mixed bag of results, so you need to give some thought on when to use them. They’re a piece of plastic that fits on the front of the strobe. The idea is that they bounce the light as it goes through them.

So instead of having all the light come from one direction, it’s like having a bunch of mini strobes in many directions. This reduces the dark shadows and bright highlights in a photo that’s not quite as dramatic. This also had a side-effect of reducing some of the backscatter in your photos.

Diffusers also increase the angle that is illuminated by the strobe. I think it’s easier to shoot with diffusers since you don’t have to be as precise with where you aim the strobe.

However, diffusers have one downside: they reduce the total amount of light reaching the subject. This is around 1-2 camera stops which is a small limitation in how you shoot.

Diffusers come in flat or round shapes. The round shapes bounce the light in even more directions.

I have a set of Carbon Arm round diffusers for my DS-02 and I use them for macro. I don’t user a diffuser with the YS-01 although I’ve tempted to buy a dome diffuser for it. The P53Pro comes with a flat disk diffuser that fits under the front ring and I use those.

Signaling Strobes

There are 2 ways to signal strobes to fire: fibre optics and electronic cables.

I haven’t used electronic cables because they require that you have a hole, called a bulkhead, in the housing to put the electrical connection.

All of my shooting is done with fibre optics. They’re just a long piece of fiberglass with a plastic coating and some standard plugs. When light shines on one end, it shines out the other end.

For cameras that have a built-in flash–TG4/5/6 and my older EM10MKII– housings have holes right in front of the camera flash to hold the fibre optics.

For cameras that don’t have a built-in flash, you use a miniature LED flash, called a flash trigger, that fits into the camera’s hot shoe.

The strobes themselves have a plug for the fibre optic and a sensor that can tell when light comes through the fibre optic.

Strobe Layout

Single Strobe

For macro, you can easily use one strobe. You can experiment with distance from the camera body and light angle. This is a pretty easy setup to shoot with.

For cameras with a port, place the strobe on a medium-length (10-20cm) arm in the center of the housing so that it reached out over the end of the lens port. Point the strobe down.

For compact cameras, you can mount the strobe directly on the housing or on a small (5-15cm) arm for a little bit more flexibility.

Twin Strobes

I put my strobes up against the port at 9-o-clock and 3-0-clock and facing inwards a bit, maybe 20°. The diffusers are about even with the end of the port. This is more about getting the arms to work than the strobe and I don’t put too much thought into it unless I have extreme amounts of backscatter or I’m doing some weird style of shooting.

If I need to fit into a smaller area, I’ll move them up to 10-o-clock and 2-o-clock like Mickey Mouse ears. That lets me slide the lens front into smaller areas sometimes.

If I’m shooting a long-distance macro shot like little fish (blennies being a huge favorite), I’ll move the strobes out away from the lens port maybe 15-20cm. That limits the backscatter.

Free-Range Strobes

I know several photographers that mount strobes on small weighted tripods so that they can place them anywhere they want. They either keep the strobe connected via fibre optic or they have a fibre optic cable that plugs into the strobe and has an exposed fiberglass end that will catch the light from their on-camera strobe.

Troubleshooting

I’ve had days where I was not happy with my strobes. There are many things that can go wrong, and when they do go wrong, you shoot black photos. I have a lot of these on my network storage drive. I’ve seen my wife lay her camera down on the sandy bottom and swim away from it and I’ve felt like that myself.

Is the Strobe Flashing?

Hold a hand in front of the strobe to reduce noise from other lights and take a test shot. You should see the strobe fire into your hand. This is part of my pre-dive camera setup routine. If the strobe doesn’t fire, then the rest of the troubleshooting tests apply.

If the strobe fires but your photos are still black, then it’s one of 5 things:

Camera is set to use TTL but strobe is not. Easy fix is to set the stove to TTL mode and see if that exposes the photo properly. Harder fix is to check the camera settings to turn off TTL. TG4/5/6 calls this “RC” (remote control). Lumix calls this “Flash Mode”. Also one warning here: I don’t know of any flash triggers for Micro Four Thirds that do optical TTL.

Camera is set to “second curtain”. This is where the flash fires at the end of the exposure. In some cases this will mean that the strobe fires too late. Try first curtain and see if that works.

Shutter speed is too fast to “flash sync”. Set shutter speed to something like 1/125. Most strobes can’t sync faster than 1/250.

Camera exposure is too dark. Bump up the ISO to 400, appetite to F8, and shutter speed to 1/125.

Lens cap is on. We’ve all done it before.

Is the Camera Flashing?

If the strobe doesn’t fire, the first thing to check is if the camera is making a flash. Most of the time you can do this by removing a fibre optic cable and watching the now-empty hole while you take a test shot.

If the camera won’t flash, then there are several reasons why.

Camera is set to “quiet mode”. This turns off the shutter noise and the flash. Great for wildlife photography, bad for underwater macro.

Flash trigger is not turned on, doesn’t have batteries, or isn’t seated properly in the hot shoe.

Flash is set to “automatic”. Setting it to “fill-in” forces the flash to fire regardless of how bright the photo exposure is.

Is the Fibre Optic Broken?

This is very common, they don’t like much abuse. Check the connectors for obvious damage. Pull the plug off of the stove, fire a test shot, and see if you get a flash coming it of the cable.

Always carry extra fibre optic cables. Swap them out and see if that fixes the problem.

New Batteries?

Strobes seem to get schizophrenic when their batteries are low. My DS-02 need the flash power turned down when the batteries are half used, otherwise they don’t fire at all. Changing batteries fixes this.

The Backup

I always have a focus light of some kind on my camera and I try to carry a cheap video dive torch when I dive. In instances where I am having troubles with my strobes and I can’t figure it out after a couple of minutes, I’ll switch to using torches and keep the dive going. Things like that are usually better to sort out on the surface. Best to keep calm and carry on with taking shots.

See you underwater!!!

–Mike

Screwing with my Dive Flag

Posted on by in underwatermacro and tagged , .

I now live in Massachusetts. Like many states in the US, divers are required to carry a dive flag. Mine is on a stick connected to a big yellow lobster buoy. There is a simple hand spool attached to it. Yay boats can see me, but is also hard to give away one hand to hold a spool while you shoot tiny things on the bottom of the ocean.

After a couple of incidents where I had to surface and chase my dive flag because it tried to run away, I stopped in to my local dive shop (Undersea Divers in Danvers) and asked if they had any advice. The answer that I got was to use a dog leash screw. They use them with open water students to “anchor” the dive flag and practice ascending and descending along a line. Pretty cool and cheap solution for me.

These look like a giant corkscrew and they are used by screwing into the ground and then attaching your dog’s long leash to it. It keeps your dog from running away, it should also keep your dive flag from running away. The one thing is that they’re obviously not made of stainless steel so they do rust out over time.

The “albatross” I have to carry on every dive. There is a different type that is a big round insert tube with a storage space in the middle.

Giant corkscrew with a bolt-snap connected via zip-tie. Also works if you meet up with a huge bottle of wine in the middle of your dive.

Pet supplies store packaging. Well-grounded!

For pet use only. Why do I have images in my head involving staking out your children at the park?

See you underwater!!!

–Mike