How do I script a 'recurrsion copy' command? - recursion

I have a project where I need to prove the virtue of a degausser to erase completely all traces of files on a standard HDD. I want to propagate the subject HDD with repetitive information, such as a single word "information".
I am thinking, in this way, I can conduct a search of the degaussed drive very quickly to prove the excellence of the process. As a part of my security requirements, programs like dBan and such are less desirable than a process that will destroy the media's ability to function for storage and any retrieval. The environment the degausser would be used in, should it prove it's integrity, also limits physical destruction of the storage media.
The .bat file I wish to create should contain a recursion that will effect the size of the file being copied so as to incrementally increase the amount of data transfer thereby decreasing the time to fill the drive.
I've used the following and so far the process slows to a crawl after a very (relative) short time. I've experimented with the use of '%' before and after ERRORLEVEL and it seems with the % symbol the program fails early in process. Anyone have any ideas? Here's the script I've been using and the specs of the computer I'm trying this process through:
P4, 1GB Ram, 2.8GHz
echo off
Rem Setting home folder
C:
Rem Setting Home folder again
cd\
:copy repeat
rem Using date and time so it formats the file IT_20130708_Time in hours minutes and milliseconds
set hr=%time:~0,2%
if "%hr:~0,1%" equ " " set hr=0%hr:~1,1%
COPY "IT.txt" "C:\testcopy\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy2\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy3\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy4\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy5\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy6\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy7\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy8\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy9\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
COPY "IT.txt" "C:\testcopy10\IT_%date:~-4,4%%date:~-10,2%%date:~-7,2%_%hr%%time:~3,2%%time:~6,2%%RANDOM%.txt"
rem if it Does not error GOTO the start aka :copy repeat
IF ERRORLEVEL ==0 GOTO copy repeat
rem end this file if error
goto end
:end
The intent in the above is to continue copying the next group of files even if there is a failure in the copy operation. Also, the process needs to be accomplished in as short a time as possible. I sincerely appreciate all suggestions, thank you for your time and considerations.

One issue you will find (FAT32, myabe NTFS too) is that with large numbers of files in a folder, it takes longer to create new files. You should look at creating a new folder every say 50,000 files.
The size of the file being copied is a factor as small files are very inefficient - so this creates a ~27 MB file to copy with INFORMATION repeated inside it.
There are a finite number of root directory entries in some drive formats so this creates one root folder and then subdirectories inside that.
This might work for you:
#echo off
for /f "delims=" %%a in ('wmic OS Get localdatetime ^| find "."') do set dt=%%a
set stamp=%dt:~0,8%-%dt:~8,6%
del it.txt 2>nul
echo creating string
for /L %%a in (1,1,500) do call set var=%%var%%INFORMATION
echo creating it.txt file (45 seconds...)
for /L %%a in (1,1,5000) do (set /p =%var%>>IT.TXT <nul)
for /L %%a in (1,1,2000000000) do call :next %%a
pause
goto :EOF
:next
set folder=00000000000%1
set folder=%folder:~-10%
set folder=c:\testcopy\testcopy-%folder%
md %folder% 2>nul
set c=0
:loop
set /a c=c+1
echo %folder%-%c%
copy it.txt %folder%\IT_%stamp%_%random%%random%%random%%random%.txt >nul
if %c% EQU 50000 goto :EOF
if not errorlevel 1 goto loop
goto :EOF

Related

Probleme with batch - script

i have 1000 photos with extension .jpg and every name of photo is a code like this 12345.jpg.
And i have 1000 folders with names like this: text_text_text_code(000000)
the job is put every photo in every single folder, i can solve this, but not working very well.
I found the problem, but i don't know how to solve.
in name folder i have text_text_text_code(000000), i need the code to stop when it arrives at character (.
#echo off
chcp 1250
echo.
echo.
cd "C:\Users\folderexample\Desktop\123"
for /r "C:\Users\folderexample\Desktop\imagem" %%a in (*.jpg) do (
for /f %%b in ('dir /b *"%%~na"*') do copy "%%a" "%%~fb"
)
I would like the search to go to the character ( and go to next search up until finish photos.

creating a folder name with a batch file

I am trying to create a folder using batch file. The folder name should be in a format - yyyymmdd-hhmm .I got started with the below code but I get yyyymmdd- as one folder and hhmm as another folder. But when I tried it after 13.00 hrs I get yyyymmdd-hhmm format. Why is there a different behaviour during 9:45 in the morning. I don't know. Any help appreciated.
For /f "tokens=2-4 delims=/ " %%a in ('date /t') do (set mydate=%%c%%a%%b)
For /f "tokens=1-2 delims=/:" %%a in ("%TIME%") do (set mytime=%%a%%b)
mkdir %mydate%-%mytime%
I get 1 folder -> 20160810- and another folder -> 945.
"I get 1 folder -> 20160810- and another folder -> 945."
That's because of the space, so mkdir sees two parameters and so creates two folders.
Either put qoutes around the new foldername
mkdir "%mydate%-%mytime%"`
or (maybe better) replace the space with a zero:
mkdir %mydate%-%mytime: =0%
putting qoutes around anyway doesn't harm:
mkdir "%mydate%-%mytime: =0%"
(btw: there is a way to get a date-time-string independent of local settings)

DOS extract directory from find command

I am writing a dos script and I want to get the path of a file that is known so that the path can be used within the script to change to that directory to use the file specified and ouput log files to the same directory.
The script adds some directories to the path and changes to the required directory to execute a command using input files in the same directory. The command generates a number of files that are saved to the same directory.
Here is what i have so far
#ECHO OFF
:: Check argument count
set argC=0
for %%x in (%*) do Set /A argC+=1
IF %argC% LSS 3 (echo WARNING must include the parent Directory, the filename and the timestep for the simulation)
:: Assign meaningfull names to the input arguments
set parentDirectory=%1
set filename=%2
set scenarioTimestep=%3
:: Check validaty of the input arguments
:: TODO: implement check for directory, filename exists, and possibly limits to the timestep
IF "%parentDirectory%"=="" (
set parentDirectory=P:Parent\Directory
)
IF "%filename%"=="" (
set filename=ship2.xmf
)
IF "%scenarioTimestep%"=="" (
set scenarioTimestep=0.1
)
echo parent Directory: %parentDirectory%
echo filename: %filename%
echo timestep: %scenarioTimestep%
set MSTNFYURI=file:mst.log
set MSTNFYLEVEL=debug
set MSTNFYFLUSH=1
set XSFNFYURI=file:xsf.log
set XSFNFYLEVEL=debug
set XSFNFYFLUSH=1
set parentNFYURI=file:parent.log
set parentNFYLEVEL=debug
set parentNFYFLUSH=1
:: Add the parent directories to the path
set PATH=%parentDirectory%\bin\;%parentDirectory%\bin\ext\;%parentDirectory%\bin\gtkmm\;%parentDirectory%\bin\osg\;%PATH%
:: Change to the target directoy
set tagetDirectory=%parentDirectory%\examples\testing_inputs
cd %tagetDirectory%
echo command will be: ft -c %filename% -T %scenarioTimestep%
::ft -c %filename% -T %scenarioTimestep%
#ECHO ON
What i want to be able to do is instead of using the hard coded directory path examples\testing_inputs for targetDirectoy, i want to be able to search for the filename supplied and change directory to that path.
I know i can get the information displayed using
"dir filename.ext /s"
DOS ouptut
Volume in drive C is OS
Volume Serial Number is XXXX-XXXX
Directory of C:\Users\Me\parent\examples\testing_input
15/11/2012 02:51 PM <size> filename
...
...
How do i extract the directory form this info to be used within the script? Also if there is more than one file of the same name, how can i select the path based on the timestamp of the file?
for /f %%F in ('dir /B /S /A:-D filename.ext ') do set file_path=%%F
pushd %file_path%\..
dir_path=%CD%
popd
echo %file_path%
echo %dir_path%
is this what you looking for?
EDIT: Check dbenham's comment.

ASP.net: Read HTML and ASP.net Pages to Strings?

I am trying to write an aspx page which will crawl through a directory and find all the files contained within. I think I have that part down.
Is it possible to read to a string without first creating .txt files from the html and asp pages I'm reading through? I don't want to create a ton of new files and then end up having to delete them later.
Ultimately, I'm trying to develop a tool to search through an entire directory and find all the image tags which have empty alt attributes or no alt attributes. I wrote some jQuery which can find the tags, and I have also written the part that searches through a directory.
If you have a file on your filesystem, you can simply read it - if you know it is a textual format, you need to use a stream with the correct encoding to do this.
Since you are reading and querying HTML, I suggest using a library that is specifically written for this task - the HTML Agility Pack - you can give it the path to the HTML file and then query it for all img elements. The source download comes with sample projects that will show you how to achieve this and other tasks.
Link:
http://msdn.microsoft.com/en-us/library/system.io.streamreader.aspx
Example:
http://www.csharp-examples.net/load-text-file-to-string/
(this example shows how to work with .txt files, but I believe if you can put any other extension)
Getting all files:
http://www.csharp-examples.net/get-files-from-directory/
Edit: and don't forgot about encoding.
sure, why not save to environment variables, no fuss, no mess. so try something like this:
will take apart an html or asp file and save to an array of variables, i have shown you how to put it back together as well.
let me know how if this is a solution for you
#echo off
setlocal EnableDelayedExpansion EnableExtensions
echo.
set count=0
if exist newfile.html del newfile.html
:: to unassemble
for /f "tokens=*" %%a in (filename.html) do (
echo %%a
set /a count=count + 1
set htmllinenum!count!=%%a
)& set finalcount=!count!
:: to assemble
for /l %%a in (1,1,%finalcount%) do (
echo !htmllinenum%%a!>>newfile.html
)
notepad newfile.tmp
set count=0
if exist newfile.asp del newfile.asp
:: to unassemble
for /f "tokens=*" %%a in (filename.asp) do (
echo %%a
set /a count=count + 1
set asplinenum!count!=%%a
)& set finalcount=!count!
:: to assemble
for /l %%a in (1,1,%finalcount%) do (
echo !asplinenum%%a!>>newfile.asp
)
notepad newfile.asp

Batch renaming / moving / hashing of files

I have a highly structured hierarchical directory containing multiple files that need to be moved into a flat structure and renamed at the same time. The original path and name must be logged along with the new path and name and eventually loaded into a database. Finally, each renamed file must get a unique, unguessable (IE: encrypted or hashed) file name. When the renamed file is moved into the new directory structure, I also want to limit the # of files in each directory, so each directory would be created with a sequential number for its name and then the files would be loaded into it until a maximum number of files was reached (eg: 255) before rolling into a new directory with the next sequential number for its name.
Is there a tool / software that does this? I did some initial research and nothing came up with the following criteria:
batch rename & copy into alternative (flatter) structure
hash / encrypt filename and ensure uniqueness
sequentially name folders and limit file count
log each file's original name and path, and new (encrypted) name and path
I have several Bash scripts I have used in the past to migrate hand-made file repositories to hashed repositories to be accessed and managed from a web application (mostly PHP apps). In these repositories filenames are hashed (to avoid collisions with files with the same content/name) and files are distributed evenly (in a deterministic fashion or randomly) to keep files-per-dir count low for performance reasons. The following is one fully-working example:
#!/bin/bash
MAXFILESPERDIR=500
TARGETROOTDIR="./newrepository"
RANDOMDISTRIBUTION=1
if [ -d "$1" ]; then
LOGFILE=$(basename $0).$(date +"_%Y%m%d_%H%M").${$}.log
SQLFILE=$(basename $0).$(date +"_%Y%m%d_%H%M").${$}.sql
SOURCEDIR="$1"
TOTALSOURCEFILES=$(find "$1" -type f | wc -l)
let "TOTALTARGETDIRS=$TOTALSOURCEFILES / $MAXFILESPERDIR"
PADLENTARGETDIRS=${#TOTALTARGETDIRS}
PADLENTARGETFILE=${#TOTALSOURCEFILES}
echo "We will create $TOTALTARGETDIRS directories to hold $MAXFILESPERDIR files per directory."
if [ "$RANDOMDISTRIBUTION" == "1" ] ; then
echo "We will rename and distribute each file randomly."
else
echo "We will rename and distribute each file uniformly."
fi
echo "Do you want to continue?"
select choice in yes no ; do
if [ "$choice" == "yes" ] ; then
COUNTER=1
find "$1" -type f | while read SOURCEFILE ; do {
CHECKSUMFILE=$(sha1sum "$SOURCEFILE" | cut -d " " -f 1)
CHECKSUMNAME=$(echo "$SOURCEFILE" | sha1sum | cut -d " " -f 1)
DETERMINISTICNONCE=$(printf "%0${PADLENTARGETFILE}d\n" $COUNTER)
if [ "$RANDOMDISTRIBUTION" == "1" ] ; then
PROBABILISTICNONCE=$(let "XX=$RANDOM % $TOTALTARGETDIRS + 1" ; printf "%0${PADLENTARGETDIRS}d\n" $XX;)
else
PROBABILISTICNONCE=$(let "XX=$COUNTER % $TOTALTARGETDIRS + 1" ; printf "%0${PADLENTARGETDIRS}d\n" $XX;)
fi
FILEDATE=$(stat -c %z "$SOURCEFILE" | cut -d "." -f 1)
FILESIZE=$(stat -c %s "$SOURCEFILE")
echo "Source file $SOURCEFILE" >> $LOGFILE
echo "Target file $TARGETROOTDIR/$PROBABILISTICNONCE/$PROBABILISTICNONCE$CHECKSUMFILE$DETERMINISTICNONCE" >> $LOGFILE
echo "INSERT INTO files (Filename, Location, Checksum, CDate, Size) VALUES ('$PROBABILISTICNONCE$CHECKSUMFILE$DETERMINISTICNONCE', '$PROBABILISTICNONCE', '$CHECKSUMFILE', '$FILEDATE', $FILESIZE);" >> $SQLFILE
mkdir -p $TARGETROOTDIR/$PROBABILISTICNONCE
cp -v "$SOURCEFILE" $TARGETROOTDIR/$PROBABILISTICNONCE/$PROBABILISTICNONCE$CHECKSUMFILE$DETERMINISTICNONCE
let "COUNTER+=1"
} ; done
echo "Done."
echo
break
fi
if [ "$choice" == "no" ] ; then
echo
echo "Operation cancelled"
echo
break
fi
done
else
echo
echo "Missing source directory"
echo
fi
Just run it from the root of your new repository. You can configure it modifying the first variables: MAXFILESPERDIR defines how many files to store per-directory, TARGETROOTDIR is the name of the first-level directory to create the first level directory (it uses only two levels, the first one is really a single root), and RANDOMDISTRIBUTION defines if the files will be distributed randomly (it may look uneven, specially for small runs) or deterministically (just counting).
How it works (FYI, just in case this is not what you are looking for but maybe you can get some ideas):
Count the source files.
Calculate how many target directories will create.
Ask for confirmation.
For each file:
Calculate the SHA1 hash for the file content.
Create a deterministic nonce.
Create a probabilistic nonce (if RANDOMDISTRIBUTION is 1, otherwise just a counter).
Get the size and modification date.
Combine the values of the random value with the hash and the counter to get the new file name (the path will be the random value).
Log the source and target full paths.
Create and log a SQL insert query.
Create the target directory (if it does not exist).
Copy the file. (You can move it if you want but I'm playing safe).
Finish
If you set RANDOMDISTRIBUTION to 1 and run the script several times, you'll get duplicates of your source files, as each file will get different target filename/path each time you run it. If RANDOMDISTRIBUTION is set to something else, everytime you run the script the files will be renamed the same way (for the same file set, if you add or remove files, they will get different names/paths).
The objective of using a random value + hash + counter is to be sure we can handle duplicates (won't collide thanks to the counter) while still distributing the files randomly (for long enough runs, this will distribute the files evenly).
Also, the preffix of the generated file name is the name of the directory too, so that if you have the file name and the directory name length, you can calculate the directory name (just in case you don't store that in your database table).
Finally, this is a one-time migration script, it was not really written to be executed regularly over the same set of files.

Resources