Aka VMware Consolidated Backup (The Good), EMC Networker (The Bad) and HP SMVI Recover (The Ugly). Put these three guys together and you're done with long, boring evenings! For those who don't know SMVI Recover, it's Storage Mirroring Recover for Virtual Infrastructure, the heir of Storage Mirroring for Virtual Infrastructure (the marketing folks at HP like to make our lives miserable sometimes, thank god they have competent technicians!).
The issue? Backing up powered-off virtual machines (replicas created by SMVI) to our tape library using VCB on EMC Networker. SMVI was working, VCB was configured properly by our vendor, why wouldn't it work? Read below, dear visitor!
Existing setup
– HP SMVI (Storage Mirroring Recover for Virtual Infrastructure), replicates machines from Production to Contigency by using snapshots at specified intervals (time or size change)
– EMC Networker, backups to disk/tape. EMC relies on VMware VCB (VMware Consolidated Backup), a VMware addon, to perform VM related tasks
Issue
EMC Networker needs to create a snapshot for the VCB method to work. When the SMVI protection runs, and a machine is backed up by EMC networker, SMVI inevitably creates a snapshot because during the time the backup is in progress, either the 30 minutes or the 256 MB replication threshold is triggered. SMVI then replicates the data according to the specified criteria, then deletes ALL snapshots, killing off EMC VCB Snapshot and causing the VM backup to fail.
Software Limitations
SMVI : a machine protected by HP SMVI should have ZERO snapshots, because SMVI is not able to distinguish which snapshots are his and which are not. Therefore, once replication succeeds, all snapshots are deleted. SMVI does not support time windows for replication. Either you replicate or you don't. It is however possible to externally start/stop a protection using undocumented commands startprotection.exe and stopprotection.exe provided by HP Competency Center.
EMC Networker: when using VCB Backup, it's impossible to give a pre- and post- backup command to be executed. Therefore it's impossible to STOP a machine protection before backup and RESTART it once the backup is done. We are therefore unable to elegantly stop/start a protection per currently backed-up virtual, which would maximalize the protection window for our VMs.
Workaround
Since both programs refuse to cooperate, we have to bring in the power of VBS and BAT scripts.
The major issue with this workaround is that passwords must be provided plain text.
We have the following setup:
Server1 / Backup Server (EMC Networker, connected Tape Library)
- C:\scripts\stopall.bat, Scheduled, runs at 08:10PM every day
- C:\scripts\startall.bat, Scheduled, runs at 06:50AM every day
Both scripts use PSEXEC to remotely execute their respective VBS counterpart remotely on Server2 by using appropriate credentials. Furthermore, these jobs are scheduled using a domain service account to ensure they will run consistently. It is necessary to logon at least ONCE with the service account and to launch PSEXEC to Accept the EULA otherwise the PSEXEC process will get stuck and won't run.
Server2 / HP Storage Mirroring Recover for Virtual Infrastructure
- C:\scripts\stopall.vbs, Remotely launched by job on Server1
- C:\scripts\startall.vbs, Remotely launched by job on Server1
Both scripts either stop ALL protections or start ALL protections using the protection parameters. Whenever a new machine is added to SMVI, these two files must be amended.
Here's an output of the startall.vbs file.Due to long file names and enclosure of the Protection parameter in double quotes, string concatenation must be used.
Set WshShell = WScript.CreateObject("WScript.Shell")
App="""C:\Program Files\HP\Storage Mirroring Recover for Virtual Infrastructure\Bin\startprotection.exe"""
Param1="username=domain\service_account password=password protection="
Param2="""Protection Name"""
WshShell.Run App & Param1 & Param2
Notes
Please note that Param2 must be in quotes (hence the need for """ at beginning and end of the string), also note that this must be exactly written as you see in the SMVI console.
Repeat the last 3 lines for each protection
Change the App string with stopprotection.exe instead of startprotection.exe in the stopall.vbs