# [Bash] Move files/movies from a single directory to their own sub-directory!

After spending a couple hours moving all my movies from a single massive folder to their own sub-directories, I decided “SCREW THIS” and decided to create a bash script.

I got my solution from trawling through various Stack Overflow posts on Bash posts. Apologies, I should have noted the references but didn’t think this would be so easy! I also don’t have enough rep to vote up either (rather annoying!).

As you might have noticed from my other scripts, I like to log things, so everything gets logged :D

Brief
Move all movies and their associated files into their own sub-directory, taking into account multi-part files (e.g. “Movie CD01/CD02”).

Pre-Requisites
Sounds a bit naff but this script relies on one thing: The movie name is consistent across each movie, i.e. the movies are already neatly named. If you use SABNzbd/Couchpotato, this shouldn’t be a problem. To give you an example, all my movies are named as follows: “Winnie The Pooh (2011) (720p).mkv” or in the case of multi-parts “Annie (1982) CD1.avi” and “Annie (1982) CD2.avi”.

How To Use
Pretty simple, start up a shell (be it via PuTTY or KiTTYTerm as I now use), “cd” to the Movie folder and paste this sucker in.

echo "Folder creation stated" > folders_created.log
find ./ -maxdepth 1 $$-name "*.avi" -o -name "*.mkv" -o -name "*.wmv" -o -name "*.mp4" -o -name "*.ts" -o -name "*.mpg"$$  -type f -print0 | while IFS= read -d '' file
do
# extract the name of the directory to create
dirName=echo ${file%.*} | sed -r 's/\sCD[0-9]*//g' # create the directory if it doesn't exist [[ ! -d "$dirName" ]] && mkdir "$dirName" && echo "Making directory$dirName" >> folders_created.log

mv -v "$dirName"* "$dirName" 2>&1 | tee -a folders_created.log
done

Script Rundown

• Basically it goes through the current directory, finds every AVI, MKV, WMV, MP4, TS and MPG file.
• For each file it finds, it …
• Strips the multi-part identifier. This can be either CD1 or CD02 or CD99, and combination of single or double digit from 0 to 99;
• Checks to see if the folder exists, if it doesn’t create it using the movie name excluding the multi-part number (and logs it);
• Moves (verbosely) all files that start with the movie title into the new directory (and logs it!).

and that’s it!

Next I have to re-scan using XBMC then clean my library to remove the old references and I’m all done! YIPPIE KAI YAY!

If you like this, then go donate to some kitten patting charity.

# [Powershell] Invoke-WebRequest and Twitter Bootstrap

This is a post to share the love …

I created a form in PHP using Twitter’s Bootstrap. I use Bootstrap because it’s so damn sexy and I seriously suck at design. The form was created to accept a dump from a table with possibly 1000’s of lines of delimited text (which was then inserted into a MySQL table).

To make it super fancy, I decided to use Powershell to call the CLI extract tool to do the dump and then someone suggested using Invoke-WebRequest to send it automatically.

CHALLENGE ACCEPTED!

After figuring out how to install Powershell v3 on my Windows 7 machine I embarked in attempting to send data via this fancy new cmdlet. Except … it wasn’t working! GAH! #rage #smash (hashtags are cool in blog posts, right? RIGHT!)

After a bit of screwing around I’ve discovered that Invoke-WebRequest didn’t pick up any of the forms fields. Considering I only had two, <button> and <textarea>, I was in serious “WTF” territory!

After some more trial and error and using my Google-fu, I started trialling different methods of creating forms from examples around the net. The one thing in common, they use .

And a solution was born. I had to use <input type=”text”> and <input type=”submit”>.

You know what really sucks? Pasting 1000 lines of text into a single row text box.

I ended up getting around it by setting it to hidden. That way, if you manually cut and paste, you see a nicely spaced <textarea> box but if you use Invoke-WebRequest, then it still works.

The final form looked like this:


Paste the contents of file here…



# [Powershell] Copy and rename files as defined in a CSV mapping file

Ha! What a title. I have no idea how else to describe it.

This is really a Part 2 to my earlier blog post, Powershell: Create files in a directory from a CSV list

In that post, I created a massive list of documents. Now the script I really needed was to rename them based on a mapping in a CSV document.

Brief:

• Copy a file from one directory to another, renaming them based on a mapping CSV document;
• Actions get logged;
• Check file exists before copying, log then skip
• Log if the file is being overwritten or not

SourceID,DestinationID

The Powershell Code:

$mappinglist = ".\mapping.csv"$sourcedocs = "\path\to\source\folder"
$destdocs = "\path\to\destination\folder"$date = get-date -format yyyyMMddHHmm
$logfile = ".\$date-documentcopy.log"

Add-Content $logfile "###$date Starting Document Copying ###"
Import-Csv $mappinglist | Foreach-Object {$srcfile = $_.SourceFile ;$destfile= $_.DestinationFile; If (!(Test-Path$destdocs\$destfile)) { Add-Content$logfile "$srcfile ($destfile) not found in destination directory"
Write-Host "$srcfile ($destfile) not found in destination directory"
}
if (Test-Path $sourcedocs\$srcfile) {
Write-Host "$srcfile copying to$destdocs\$destfile" Copy-Item$sourcedocs\$srcfile$destdocs\$destfile" Add-Content$logfile "$srcfile copied to$destdocs\$destfile" } else { Write-Host "$srcfile source document not found ... skipping"
Add-Content $logfile "$srcfile source document not found ... skipping"
}
}
Add-Content $logfile "###$date Finished Document Copying  ###"

Notes:
Most of the code is done in a few lines, however for my purpose, I really needed to have everything logged as we generate formal reports on basically everything we do!
I’m a big fan of timestamping my log files, hence the $date-documentcopy.log line. Also, I like being able to see what’s going AND having it log to file, so I duplicate the Add-Content and the Write-Host lines. I rather like the Write-Host “copying” and Add-Content “copied” line. The reason for this is I have some large files (>2GB) so *if* it fails I can see on-screen which record it was but the log has it once it’s finished. # [Powershell] Create files in a directory from a CSV list I’ve recently been using Powershell more and more to replace either broken and/or ageing ‘batch’ files at work plus also come personal projects I’ve been working on. Some of them took a bit of digging around between various sites/blogs to put together, so I’ve decided to re-post them with a final solution. Warning: I’m a pretty crap blogger, so don’t expect regular updates! Anyway, this one is I wanted to test a copy and rename script but needed some testing data first. So this creates a file in a folder with the name taken from a list in a CSV. The ID didn’t have a file extension so I’m randomly creating one from a small list. Brief: Take a list of ID’s from a CSV and create a file in a folder using the ID as the filename with a random file extension. $docext = @("msg","docx","doc","xlsx","xls","pdf")

$path = ".\filename_list.csv" Import-Csv$path | Foreach-Object {
$filename =$_.filename;
$ext =$docext | Get-Random -Count 1;
if (!(Test-Path "C:\path\to\files\$filename.*")) {$filename | Out-File "C:\path\to\files\$filename.$ext";
Write-host "creating file $filename"; } else { Write-Host "file already exists"; } }  A brief run down. The @docext is an array of possible extensions, as mine list only has an ID, not the extension. I wanted to make one up for my test. The$ext = $docext | Get-Random -Count 1; grabs one of these random extensions. I run a Test-Path to see if the filename exists so it doesn’t create two files with different extensions. I use a wildcard to check. Lastly I Write-Host so I can see what’s going on. # grepWin & Notepad++: Opening files at the line number. Worked out a cool tip to combine grepWin & Notepad++ to open files at the line number of the search. In grepWin, click on the settings button in the middle of the screen (next to the “Only one instance” checkbox. Add this to the Editor box: [plain]C:\Program Files (x86)\Notepad++\notepad++.exe -n%line% “%path%”[/plain] You’ll need to update the location of your Notepad++ if it’s not installed in the default location. UPDATE: I discovered you need to add quotes around the %path% to open documents with spaces in the full path name! # A Few Useful Utilities for Windows I thought I’d list down a few useful utilities I use for Windows on almost a daily basis! • Notepad++ : Single-handly the best Notepad tool on Windows. • PathCopyCopy : Copy the path of the folder/file in short/long/UNC plus a raft of other options. • Hashtab : Quickly check the checksums (md5/sha/etc) for files/folders within a shell extensions. • grepWin: Use text or regex based expression to mass edit files/documents etc. Also has the ability to open a file directly to the line you want! See this post here. • FreeFileSync : I use this all the time to keep folders/files in sync between work laptop, home PC, network etc. Can be scheduled too for extra backup goodness! The best bit, they are all ‘free to use’ for private use and most are even GPL licensed or similar! I’ll add more as/when I think of them. # Hacking jQueryFileTree to display specified file extensions Up until now, I’ve used the linux utility ‘tree’ to display the contents of a directory and it’s contents. Yesterday, I decided to make it prettier. The hunt began for a utility that displayed file system contents in a tree view. I stumbled across a few, but the one that I picked, mainly because of it’s ease was jQuery File Tree. A couple hours playing around I had it playing nicely with Twitter Bootstrap site, but one thing that it didn’t do was to only display specific file types, so here is my quick and dirty hack to make it work. I used the PHP connector jqueryFileTree.php and did a few things: I added a new variable: [php]$fileextvideo = array(“.avi”, “.mkv”, “.mp4”, “.m4v”, “.wmv”);[/php]

Updated the if statement for files to check for the file types:
[php]if( file_exists($root .$_POST[‘dir’] . $file) &&$file != ‘.’ && $file != ‘..’ && !is_dir($root . $_POST[‘dir’] .$file) && infiletype($file,$fileextvideo) ) { [/php]

Then added a new function to do the check:
[php]function infiletype($file,$extensions) {
foreach ($extensions as$ext) {
if (strpos($file,$ext)) {
return true;
}
}
}[/php]

All up it turned it into this:
[php]
if( file_exists($root .$_POST[‘dir’]) ) {
$files = scandir($root . $_POST[‘dir’]);$fileextvideo = array(“.avi”, “.mkv”, “.mp4”, “.m4v”, “.wmv”);

natcasesort($files); if( count($files) > 2 ) { /* The 2 accounts for . and .. */
echo “

“;
// All dirs
foreach( $files as$file ) {
if( file_exists($root .$_POST[‘dir’] . $file) &&$file != ‘.’ && $file != ‘..’ && is_dir($root . $_POST[‘dir’] .$file) ) {
echo “

• ” . htmlentities($file) . “ • “; } } // All files foreach($files as $file ) { if( file_exists($root . $_POST[‘dir’] .$file) && $file != ‘.’ &&$file != ‘..’ && !is_dir($root .$_POST[‘dir’] . $file) && infiletype($file, $fileextvideo) ) {$ext = preg_replace(‘/^.*\./’, ”, $file); echo “ • ” . htmlentities($file) . “
• “;
}
}
echo “

“;
}
}

function infiletype($file,$extensions) {
foreach ($extensions as$ext) {
if (strpos($file,$ext)) {
return true;
}
}
}
[/php]

Quite happy with the results!

Feel free to comment on improvements, as I’m a n00b to PHP and programming in general!

# Regex to create list of values for SQL statement

In my day I’m currently undertaking a lot of data migrations. One thing I have to do frequently is find a list of records that do and/or don’t exist in one or more tables. This usually involves getting a list of ID’s from a log output text files then doing SQL queries to attempt to find them!

Unfortunately our clients machines are generally Windows based. Thankfully most are kind enough to allow us to use/install Notepad++ (IMO  the best notepad software available on Windows).

As such I utilise Regex a lot to find/strip/edit the contents to get the ID’s out and today I had a brainwave I thought I’d share on how to get it out quickly and turn it into a list that you can cut and paste into a WHERE clause of a SQL statement.

In our case the format of the logs are:
[plain][/plain]

What I want is:
[plain]’‘,[/plain]

And I get there with:
[plain]s/^(.*)/’\1’,/[/plain]