EMC Avamar Tape-Out Option
Closed     Case # 10051     Affiliated Job:  New Trier Township District 2031
Opened:  Tuesday, March 1, 2011     Closed:  Friday, March 25, 2011
Total Hit Count:  23435     Last Hit:  Thursday, October 19, 2017 2:33:57 AM
Unique Hit Count:  4781     Last Unique Hit:  Thursday, October 19, 2017 2:33:57 AM
Case Type(s):  Server
Case Notes(s):  All cases are posted for review purposes only. Any implementations should be performed at your own risk.

Project:
Avamar Tape Out Option

It's only been a couple weeks since EMC implemented our Avamar to Networker/Backup to Tape option... In our environment we decided to archive within Avamar 30 dailies, 3 monthlies and archive the month ends out to tape for a more extended period of retention while not reducing our storage within Avamar.

There are a couple variations to select from for the Tape out Option.
-   ATO "Avamar Tape-Out:" Re-Hydrate the data to a server attached to a tape library with Networker installed, Avamar pushes the data to the server and then calls Networker to archive the data to tape
-   ADT "Avamar Data to Tape(?):" Variations of how to take data directly out of Avamar and store this already data duped data out to tape.

The first option is what we selected, I am explained that the other option "ADT" is a much more involved process of restoring data back. The "ATO" option, which we have now used for a full year, has worked extremely well. The initial setup, confusing and incomplete - we have worked to fit our needs and customized based on our environment. The biggest variance between these options - ATO is very portable to restore the data and very quick; I am explained ADT pushes data back into Avamar, is more involved and time consuming to restore data. On the flip side, ADT requires less tapes since the data is data duped; however, unlike what we are doing, ATO offers the ability to perform incrementals rather than fulls; however, since we perform ATO on VM image level backups, SQL databases and only once a month, full ATO works best for us.

SQL or Exchange agent level backups - they are re-hydrated and then stored to tape. Originally we were explained by EMC that to restore from tape a Avamar client based backup of SQL or Exchange, would require the assistance of EMC; however, we later proved this incorrect. On several test restores, I have taken the f-0 etc files and restored them manually - they appear to be like SQL dump files - throw on a .bak (not required, but then SQL sees them while browsing) and SQL will restore them right back into the environment. We have not tested nor had the need to restore an Exchange database; however, I have had direct confirmation that this as well may be restored without the need to involve EMC.

So I wanted to document some of the process I have learned while troubleshooting various issues we have encountered during our initial implementation. The consultant from EMC created the overall process - the server used for ATO was an old system we beefed up with large cheap storage and our previous tape library. Originally, we were setup with a single "batch" that called each of the groups; however, we found, for our situation calling each one individually over a span of 7 days worked out best. If the next group is called during a maintenance window - the group would fail - we were explained this was because some data is written to the Avamar grid; therefore, breaking the groups out and calling each right after a maintenance window over a span of 7 days proved to work best for our retention plan.

Action(s) Performed:
Total Action(s): 3
Action # Recorded Date Type Hit(s) User Expand Details
10178 3/25/2011 10:59:02 AM Server 1868 contact@danieljchu.com Utility Node: From a file level configuration view on our Utility N  More ...
10177 3/25/2011 10:59:02 AM Server 2181 contact@danieljchu.com Backup Client: Some of our servers being backed up to the grid - on  More ...
10176 3/25/2011 10:59:02 AM Server 1873 contact@danieljchu.com Networker Server: We kept receiving errors with regard to the SSL c  Collapse ...
Last Hit: Wednesday, October 18, 2017 10:49:22 PM

Networker Server:
We kept receiving errors with regard to the SSL communications between the Avamar grid and the TAPEOUT server; it was explained to clear these types of errors we could add the below to the avtar.cmd file located at "C:\Program Files\avs\var" on the Networker server, again if this file does not exist, create it... This will turn off SSL communication between the Avamar grid and the TAPEOUT (Networker) server.
-   --encrypt=proprietary

You may also add these two parameters to enable verbose debugging on the log files:
-   --debug
-   --verbose

We also added a cleanup script which uses robocopy to clear out the restored files after they have been committed to tape. The ATO process has this built in, but if NTFS permissions do not allow this because of access rights, I have found using robocopy will force delete files even without appropriate permissions. I do not know if you can just create this file or if it needs to be configured somewhere, but our EMC rep already had a cleanup.bat located at "C:\Program Files\avs\etc\scripts" which we added the command:
-   robocopy "C:\_Batch\EmptyFolder" "E:\ato" /E /COPYALL /MIR /R:0 /W:0

This command uses (which you should create) a source path to a empty folder should mirror the results where the data is restored to "E:\ato" - since the source folder is empty it will clear out the ato folder even if permissions do not allow...

In order to recover data or view data stored on tapes execute the following, Start - Run - "winworkr" - this utility will allow you to Browse "Versions" of backup catalogs.
-   Select the drive the data was restored to,
-   Then click the View option on the menu bar,
-   Select "Versions" to select the backup set,
-   Select the version on the "Versions" window and click "Change Browse Time"

Networker in itself is pretty cryptic - so you have to guesstimate the backup set based on dates/times...

Outcome:
[Contributed: 2/21/2012] In conclusion, I want to stress how happy we have been both with Avamar and the ATO option. We have been fortunate, over the past year, to have only required data restored out of Avamar directly; however, we have performed trial restores off tape and the process has been quite simple. Restoring SQL, files and helping to keep storage in our Avamar grid just below the 80% threshold - the ATO option has helped us retain a proper retention policy.



Profile IMG: Footer Left Profile IMG: Footer Right