Engineer.iX Dedundancio

 

 

This program was designed for "Clean up" and removing of redundancies in Windows directories in preparation of a following file backup. It helps you to find and remove doubles of files, respectively searching and destroying of files with identical content. In order to reach this goal every file will be registered and all following files with the same content will be eleminated. Sometimes you may want to put the same file in different folder to make it easier to find the file. Then you can let the program replace a double file by a desktop link to the same file that was found first.

 

 

For distinguishing the file identity Dedundancio uses the SHA1-Hash algorithm, other algorithm are not yet planned but if you need then just ask for extension support.

On the server (HP Proliant G5) of our reference customer we need around 5 hours to process ca. 260 GB in 300.000 files. The performance of Dedundancio could be increased significantly using multithreading. If you need it faster or if you have a special wish or a fancy idea, do not hesitate and best contact us directly.

 

 

Typical Use Cases

Case 1 - Elimination of identical files for saving disk space

This program originally was designed for supporting a customer's backup system. The backup routine was designed like following: All user files will be saved weekly and daily from the servers to the backup tapes. The weekly tapes will be saved and stored on a secure place. The daily tapes will be overwritten after one week. In general the customer wants a loss of restorability of max. 1 hour. (what means that files have to be saved at least every hour)

The used backup system of a famous canadian software producer was pretty inapropriate to do the hourly backup because the "Backup To Disk" consumed too much server resources and slowed down the system so much that the users started to complain about regularly access problems during their normal file work. Another worse thing was that the restore process took too much time.

In order to track changes during the week (the problem is when a file is created on Monday and removed on Friday it would not appear on the weekly backup) and in order to make a fast recovery of files that were changed during the day, we decided to use Microsoft's ROBOCOPY for copying the user files whichs archive attributes were changed since the last day backup. (The files' archive attributes will be set back by the daily tape backup.) This procedure had one bad side effect. Every hour the same files are copied additionally to the lately changed ones. For saving resources we decided to run a cleanup job - before the weekly backup job - in order to eleminate all recurring files. For this we developed Dedundancio.

Case 2 - Elimination of identical files to avoid version branching

In some case - especially on shared group folders - it may be important to make sure that there is only one copy of a file at same time, in order to make sure that there is no branching and parallel modification of a file under the same version number. In this case you can use Dedundancio to replace every double of a file by a link to the first appearance of a file.

 

 

Last Update on: 14 Jun 2011

Copyright © 2004-2011, Engineer.iX Software Services. All rights reserved. Legal Info, SiteMap