A Simple Python Interface for Gmail

The Problem:
I monitor processes, files, and tasks on my computer. Some monitors are for security others are just to know when a task has finished. Whatever the case, I would like to be notified vie email/text message when something has occurred.

The Solution:
As always... write a script. I wanted something that could easily be used to send an email with or without an attachment. I also wanted to be able to just call a mail function and pass in a recipient, subject, message and attachment.

The Script:
The following script is a pretty simple script that can be imported from another python file. I did not provide command line options because I usually use it in conjunction with another script, so it is always an import. I provide a main part so you can input your username, password, and some email to test the script out. Just run it from the command line to test it.
send_mail.py#!/usr/bin/python<br />import smtplib<br />from email.MIMEMultipart import MIMEMultipart<br />from email.MIMEBase import MIMEBase<br />from email.MIMEText import MIMEText<br />from email import Encoders<br />import os<br /><br />class Gmail:<br /> def __init__(self, username, password):<br /> if '@gmail.com' not in username:<br /> username += '@gmail.com'<br /> <br /> self.user = username<br /> self.password = password<br /> self.smtp = "smtp.gmail.com"<br /> self.smtp_port = 587<br /> <br /> def mail(self, to, subject, text, attach = None):<br /> msg = MIMEMultipart()<br /> msg['From'] = self.user<br /> msg['To'] = to<br /> msg['Subject'] = subject<br /> msg.attach(MIMEText(text))<br /> if attach:<br /> part = MIMEBase('application', 'octet-stream')<br /> part.set_payload(open(attach, 'rb').read())<br /> Encoders.encode_base64(part)<br /> part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(attach))<br /> msg.attach(part)<br /> <br /> mailServer = smtplib.SMTP(self.smtp, self.smtp_port)<br /> mailServer.ehlo()<br /> mailServer.starttls()<br /> mailServer.ehlo()<br /> mailServer.login(self.user = username, self.password)<br /> mailServer.sendmail(self.user = username, to, msg.as_string())<br /> mailServer.close()<br /><br />if __name__ == "__main__":<br /> username = 'username'<br /> username = 'password'<br /> mobile_email = "YouPhoneNumber@YourProvider.com"<br /> gmail = Gmail(username, username)<br /> gmail.mail(mobile_email,<br /> "Testing Testing 123",<br /> "This is a email sent to you using python!")

Further Explanation:
This could easily extend a general class Mail that would have the same functionality, but you could tailor it to your individual email provider. Contact me if you want such help.

Create Optimized DVD/CD Images of Folder for Backup

The Problem:
I have music, videos, pictures, and files that all need backups. I have them organized how I want, but they don't always fit perfectly on to a DVD. I want a simple program that I point to a directory and it will create optimized image files for burning to a dvd. I would also like to do a few quick 'dry runs' to be sure that I like which files go in each image and that I don't have an image that is only 10% full!

The Solution:
Well, if you can't find one... make one! I decided to write a script that linked up with PowerISO's commandline tool (piso) to create images of files. You will need several Gigs of space to store the images... This is nice because you don't have to go through all your files and figure out how to divide them among images... You just do a few dry runs and then when you are happy, let it go all night!

The Script
This script uses the command line utility of PowerISO to create disc images of a folder and tries to maximize the disc usage while still preserving the file structure of the provided folder. You will need to add the PowerISO install path to your path environment variable
file_dvd_backup.pyfrom optparse import OptionParser<br />import time, logging, os, sys, string, shutil, tempfile<br />from os import sep<br /><br />class ImageTooLargeException(Exception):<br /> def __init__(self, value):<br /> self.value = value<br /> def __str__(self):<br /> return repr(self.value)<br /><br />class DiscImage:<br /> def __init__(self, file_path, base_dir, max = 4.37, password = None):<br /> self.file_name = '.'.join(file_path.split(sep)[-1].split('.')[:-1])<br /> self.logger = logging.getLogger('DiscImage:%s'%self.file_name)<br /> self.temp = None<br /> self.files = []<br /> self.dirs = []<br /> self.command = 'piso'<br /> if base_dir[-1] != sep:<br /> base_dir += sep<br /> self.base_dir = base_dir<br /> self.file_path = file_path<br /> self.max = max *1000*1000*1000 # Size in bytes<br /> self.password = password<br /> self.size = 0<br /> <br /> def setCommand(self, command):<br /> self.command = command<br /> <br /> def add(self, file):<br /> file_size = os.stat(file).st_size<br /> if self.size + file_size > self.max:<br /> raise ImageTooLargeException('File could not be added because it was too large: %s' %file)<br /> self.size += file_size<br /> self.files.append(file)<br /> self.logger.log(5, 'Adding File to Image: %s' %file)<br /> rel_dir = sep.join(file.split(sep)[:-1]).replace(self.base_dir, '')<br /> if rel_dir not in self.dirs:<br /> self.dirs.append(rel_dir)<br /> <br /> def append(self, file):<br /> self.add(file)<br /> <br /> def percent_full(self):<br /> return self.size/self.max*100<br /> <br /> def create(self):<br /> # I copy all to a temp directory because Power ISO does not have an add/append command and will not take more than a few -add command line arguments<br /> self.copyFiles()<br /> piso = self.command<br /> if self.command != 'piso':<br /> cmd = '"%s"' %self.command<br /> else:<br /> cmd = [self.command]<br /> cmd = [self.command]<br /> cmd.append('create')<br /> cmd.append('-o %s' % self.file_path)<br /> <br /> if self.password is not None:<br /> cmd.append('-setpassword %s' %self.password)<br /> <br /> cmd.append('-add \"%s\" /' %self.temp)<br /> <br /> cmd_str = ' '.join(cmd)<br /> if self.password is not None:<br /> cmd_str = cmd_str.replace(self.password, '*'*len(self.password))<br /> <br /> self.logger.debug('Creating Image with Command: \n%s '% (cmd_str))<br /> os.system(' '.join(cmd))<br /> <br /> self.cleanTemp()<br /> <br /> def copyFiles(self):<br /> if self.temp is None:<br /> self.temp = tempfile.mkdtemp()<br /> for file in self.files:<br /> file_dir = os.path.join(self.temp, sep.join(file.replace(self.base_dir, '').split(sep)[:-1]))<br /> if not os.access(file_dir, os.F_OK):<br /> os.makedirs(file_dir)<br /> self.logger.log(3, "Creating Temp File: %s" %os.path.join(file_dir, file.split(sep)[-1]))<br /> shutil.copy(file, os.path.join(file_dir, file.split(sep)[-1]))<br /> <br /> <br /> def cleanTemp(self):<br /> for root, dirs, files in os.walk(self.temp, topdown=False):<br /> for name in files:<br /> try:<br /> os.remove(os.path.join(root, name))<br /> except:<br /> self.logger.error('Could not remove temp file: %s' %os.path.join(root, name))<br /> for name in dirs:<br /> try:<br /> os.rmdir(os.path.join(root, name))<br /> except:<br /> self.logger.error('Could not remove temp dir: %s' %os.path.join(root, name))<br /> try:<br /> os.rmdir(self.temp)<br /> except:<br /> self.logger.error('Could not remove temp dir: %s' %os.path.join(root, name))<br /> <br /> self.temp = None<br /><br /><br />class Dir2Img:<br /> def __init__(self, src, dest, name_base, img_type, max = 4.37, password = None):<br /> self.logger = logging.getLogger('Dir2Img')<br /> self.command = None<br /> self.src = src<br /> self.dest = dest<br /> self.name_base = name_base<br /> self.img_type = img_type<br /> self.max = max<br /> self.password = password<br /> self.images = []<br /> self.parseImages()<br /> <br /> def setCommand(self, command):<br /> self.command = command<br /> for image in self.images:<br /> image.setCommand(command)<br /> <br /> def parseImages(self):<br /> img = self.getNextImage()<br /> img2 = None<br /> for root, dirs, files in os.walk(self.src, topdown=True):<br /> for name in files:<br /> full_path = os.path.join(root, name)<br /> img, img2 = self.addToImage(full_path, img, img2)<br /> <br /> def addToImage(self, full_path, img, img2 = None):<br /> # Try and add it to the current image<br /> try:<br /> img.append(full_path)<br /> except ImageTooLargeException:<br /> # See if the current image is less than 75 percent full<br /> if img.percent_full()< file_size =" os.stat(full_path).st_size/(1000*1000*1000)"> self.max:<br /> raise ImageTooLargeException('File Too Large! File: %s, Size: %fMB' % (full_path, file_size*1000))<br /> <br /> if img2 is not None:<br /> # If we are already dealing with a second image, add the file to that and make it the primary image<br /> return self.addToImage(full_path, img2, None)<br /> else:<br /> # If not, then create a secondary image and add the large file to it<br /> img2 = self.getNextImage()<br /> img2.append(full_path)<br /> else:<br /> # Image is pretty much full<br /> if img2 is not None:<br /> # If dealing with a secondary image, start adding smaller files to it<br /> return self.addToImage(full_path, img2, None)<br /> else:<br /> # Create a new image<br /> img = self.getNextImage()<br /> img.append(full_path)<br /> return img, img2<br /> <br /> def getNextImage(self):<br /> i = ''<br /> if len(self.images) > 0:<br /> i = str(len(self.images))<br /> file_path = os.path.join(self.dest, '%s%s.%s' %(self.name_base, i, self.img_type))<br /> img = DiscImage(file_path, self.src, self.max, self.password)<br /> self.images.append(img)<br /> self.logger.log(1, 'New Image: %s' %file_path)<br /> if self.command is not None:<br /> img.setCommand(self.command)<br /> return img<br /> <br /> def create(self):<br /> for image in self.images:<br /> image.create()<br /> <br /> def usageReport(self):<br /> s = ''<br /> for image in self.images:<br /> s += '%s: %f%% Full\n' %(image.file_name, image.percent_full())<br /> return s<br /> <br /> def fullReport(self):<br /> s = ''<br /> for image in self.images:<br /> s += '\n%s\n'%(image.file_name)<br /> s += '\tUsage: %f%% Full\n' %(image.percent_full())<br /> s += '\tDirectories:\n'<br /> for dir in image.dirs:<br /> s += '\t\t%s\n' %dir<br /> s += '\n'*2<br /> return s<br /> <br /> <br />def setupLogging(level):<br /> if not os.access('log', os.F_OK):<br /> os.mkdir('log')<br /> logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/file_dvd_backup.log'),<br /> filemode='w')<br /> # define a Handler which writes INFO messages or higher to the sys.stderr<br /> console = logging.StreamHandler()<br /> console.setLevel(level)<br /> # set a format which is simpler for console use<br /> formatter = logging.Formatter('%(name)-5s: [%(levelname)-5s] %(message)s')<br /> # tell the handler to use this format<br /> console.setFormatter(formatter)<br /> # add the handler to the root logger<br /> logging.getLogger('').addHandler(console)<br /> logger = logging.getLogger('DVDBackup')<br /> return logger<br /> <br />def main():<br /> # This method parses the options set at the command line and initializes the logger.<br /> parser = OptionParser()<br /> parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br /> parser.add_option("-s", "--src", type='string', dest="src", help="Source drive or DIRECTORY. Script will create images from this directory/drive.", metavar="DIRECTORY", default=None)<br /> parser.add_option("-d", "--dest", type='string', dest="dest", help="Create images in destination DIRECTORY. ", metavar="DIRECTORY", default=None)<br /> parser.add_option("-f", "--file", type='string', dest="file", help="Create image files using base FILENAME. Must be an iso, daa, or bin. Example: Test.iso. NOTE: If not specified, script will use the source directory as a base and create iso images.", metavar="FILENAME", default=None)<br /> parser.add_option("-p", "--password", type='string', dest="password", help="Password for the image. This is optional.", metavar="DIRECTORY", default=None)<br /> parser.add_option("-m", "--max", type='float', dest="max", help="Max disc size. Default is a 4.7 gig disc (4.37 max size).", metavar="SIZE(GB)", default=4.37)<br /> parser.add_option("", "--piso", type='string', dest="piso", help="PowerISO command. Usually piso. If the PowerISO folder is not in your environment path, you will need to specify the full COMMAND to piso.exe here. Default: piso", metavar="COMMAND", default=None)<br /> parser.add_option("", "--dry", action="store_true", dest='dry', help="Dry run only.", default=False)<br /> <br /> opts, args = parser.parse_args()<br /> logger = setupLogging(opts.debug)<br /> return opts, args, logger<br /><br />if __name__ == "__main__":<br /> opts, args, logger = main()<br /> if opts.dest is None:<br /> logger.critical('Destination not provided. Please specify a destination via the --dest=DIR command line option')<br /> sys.exit(1)<br /> if opts.src is None:<br /> logger.critical('Source not provided. Please specify a source via the --src=DIR command line option')<br /> sys.exit(1)<br /> <br /> if opts.file is None:<br /> name_base = opts.src.split(sep)[-1].replace(' ','')<br /> img_type = 'iso'<br /> else:<br /> name_base = '.'.join(opts.file.split('.')[:-1])<br /> img_type = opts.file.split('.')[-1]<br /> <br /> images = Dir2Img(opts.src, opts.dest, name_base, img_type, opts.max, opts.password)<br /> if opts.piso:<br /> images.setCommand(opts.piso)<br /> report = images.fullReport()<br /> logger.info('Image Report: \n%s' %report)<br /> <br /> if opts.dry == False:<br /> images.create()<br />
Further Explanation:
OK, so as an example, I want to backup all of my pictures. I keep them (as seen in a previous post) in the structure of YYYY\MM month and when I burn a backup image of all of my pictures. I would prefer to keep as many pictures of the same year together as possible, however, if there is a large file (say a video clip) in one folder that doesn't fit on a dvd that is only 50% full, I would prefer it to be on a new dvd image, and then continue to fill first dvd, and then fill the new dvd image. Confusing I know, it creates dvd's that are usually about 99% full, even with a few large files mixed in.

I also hate searching for which dvd has what folder. The script creates a list of directories contained on each disc and add that file to each image. It makes it very easy for figuring out which disc has what pictures. The script could easily be altered to contain a list of all folders AND files, but I thought that was overkill.

Blogger Syntax Highlighting

The Problem:
I am still using Blogger.com to host my blog, which works pretty good, you just don't have too many options and no server control. This means I can't just include my scripts on the fly (for now). I have been working with my layout, client side syntax highlighting, and general look and feel of the blog and I am getting somewhat comfortable with it. For syntax highlighting I decided to go with syntaxhighlighter, a google code project written in javascript. It works well, it's just slow. The problem with doing using plugins and including the script right into your post is that I have to generate the html for each blog post... and I want them all to be consistent.

The Solution:
Wanting things to be as simple as possible for me to post and edit my new scripts I wanted a simple code generator. I wanted it to create the html for adding a collapsed code section with a link to the script on my server on the top. I like this format because you can copy it, read it or just download it. Maybe I'll edit the syntaxhighlighter so that it takes a link for downloading the code... but for now this works. Just for fun, here's the code for my code generator:
convert_script.pyfrom optparse import OptionParser<br />import time, logging, os, sys, string, shutil<br />import win32clipboard, win32con, random<br />from urllib2 import urlopen<br />from lxml import etree<br />from os import sep<br /><br />class SyntaxHighlight:<br /> def __init__(self, code, language):<br /> self.syntax = self.script_path = self.script = None<br /> self.code = code<br /> self.lang = language<br /> self.options = [language]<br /> <br /> def setOption(self, option):<br /> if option not in self.options:<br /> self.options.append(option)<br /> <br /> def removeOption(self, option):<br /> if option in self.options:<br /> self.options.remove(option)<br /> <br /> def setScript(self, script):<br /> if '/' in script:<br /> s = '/'<br /> elif '\\' in script:<br /> s = '\\'<br /> else:<br /> s = sep<br /> <br /> self.script = script.split(s)[-1]<br /> <br /> if 'http://' in script:<br /> self.script_path = script<br /> <br /> <br /> def getSyntax(self):<br /> options = ':'.join(self.options)<br /> <br /> self.syntax = ''<br /> if self.script_path is not None:<br /> self.syntax += '<a href="%s">%s</a>' %(self.script_path, self.script)<br /> elif self.script is not None:<br /> self.syntax += self.script<br /> <br /> # I had to break these up or it killed my generated script!<br /> self.syntax += '<' self.syntax += 'textarea name="code" class="%s" cols="60" rows="10">' %(options)<br /> self.syntax += self.code<br /> self.syntax += '</' self.syntax += 'textarea>' <br /> <br /> return self.syntax<br /><br />def getLanguage(script):<br /> ext_dict = {'py':'python', 'java':'java', 'groovy':'java', 'cs':'csharp', 'css':'css', 'php':'php', 'rb':'ruby', 'sql':'sql'}<br /> ext = script.split('.')[-1]<br /> if ext.lower() not in ext_dict.keys():<br /> return ext<br /> else:<br /> return ext_dict.get(ext)<br /><br />def setupLogging(level):<br /> if not os.access('log', os.F_OK):<br /> os.mkdir('log')<br /> logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/convert_script.log'),<br /> filemode='w')<br /> # define a Handler which writes INFO messages or higher to the sys.stderr<br /> console = logging.StreamHandler()<br /> console.setLevel(level)<br /> # set a format which is simpler for console use<br /> formatter = logging.Formatter('%(name)-5s: [%(levelname)-5s] %(message)s')<br /> # tell the handler to use this format<br /> console.setFormatter(formatter)<br /> # add the handler to the root logger<br /> logging.getLogger('').addHandler(console)<br /> logger = logging.getLogger('ConvertScript')<br /> return logger<br /> <br />def main():<br /> # This method parses the options set at the command line and initializes the logger.<br /> parser = OptionParser()<br /> parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br /> parser.add_option("-s", "--script", type='string', dest="script", help="PATH to the script you want to include.", metavar="PATH", default=None)<br /> parser.add_option("", "--lang", type='string', dest="lang", help="Override the language the script is encoded in. Normally taken from the extension. Default: python", metavar="LANGUAGE", default='python')<br /> parser.add_option("", "--code", type='string', dest="code", help="Simple Code. If used you must also specify the --lang", metavar="CODE", default=None)<br /> parser.add_option('-g', '--no-gutter', action="store_true", dest="nogutter")<br /> parser.add_option('-c', '--no-controls', action="store_true", dest="nocontrols")<br /> parser.add_option('-f', '--first-line', type='int', dest="firstline", help="First line to start counting from.", metavar="CODE", default=None)<br /> parser.add_option('-n', '--show-columns', action="store_true", dest="showcolumns")<br /> opts, args = parser.parse_args()<br /> opts.collapse = None<br /> logger = setupLogging(opts.debug)<br /> return opts, args, logger<br /><br />if __name__ == "__main__":<br /> opts, args, logger = main()<br /> if opts.script is None and opts.code is None:<br /> logger.critical('Neither Script nor Code provided. Please specify a script (--script=PATH) or code (--code=CODE) command line option')<br /> sys.exit(1)<br /> <br /> if opts.code is not None and opts.lang is None:<br /> logger.critical('You must specify the language (--lang=LANGUAGE) when specifying code on the command line.')<br /> <br /> if opts.lang is None:<br /> opts.lang = getLanguage(opts.script)<br /> logger.debug('Setting Language Extension to: %s' %opts.lang)<br /> <br /> if opts.code:<br /> opts.nogutter = True<br /> opts.nocontrols = True<br /> code = opts.code<br /> else:<br /> opts.collapse = True<br /> if 'http://' in opts.script:<br /> f = urlopen(opts.script)<br /> else:<br /> f = open(opts.script)<br /> code = f.read()<br /> f.close()<br /> <br /> syntax = SyntaxHighlight(code, opts.lang)<br /> <br /> if opts.script is not None:<br /> syntax.setScript(opts.script)<br /> <br /> if opts.nogutter:<br /> syntax.setOption('nogutter')<br /> <br /> if opts.nocontrols:<br /> syntax.setOption('nocontrols')<br /> <br /> if opts.collapse:<br /> syntax.setOption('collapse')<br /> <br /> if opts.firstline is not None:<br /> syntax.setOption('firstline[%i]' %opts.firstline)<br /> <br /> if opts.showcolumns:<br /> syntax.setOption('showcolumns')<br /> <br /> syntax = syntax.getSyntax()<br /> logger.debug('Syntax: \n%s' %syntax)<br /> win32clipboard.OpenClipboard()<br /> win32clipboard.EmptyClipboard()<br /> win32clipboard.SetClipboardText(syntax)<br /> win32clipboard.CloseClipboard()<br />
Note: The html for the presentation of my code generator on my blog was created by this script...

Further Explanation:
So in essence, all I have to do is point the script at a file on my server and bam! the code is in the clipboard! I love it. It supports code snippits and local or web script files. You can also add options for the syntaxhighlighter javascript. Yea... it's a bit rough, but what do you expect in under an hour...

Redundant MySQL backup with Python

The Story:
Ok so we lost our entire company wiki the day of launch... luckily I was able to recover over 60% of it from the cache of MANY people's computers (I just wrote a script that filtered their cache and spit out the wiki content pages, then a bunch of us went through the pages)... but that is a whole different subject. (Little Note: I was not responsible for backing up or transferring the database)

The Solution:
Since then I have been pretty paranoid about running backups and have volunteered for the task (because I have done a LOT of work on the wiki). I understand MySQL Replication, which is very powerful, but sometimes you just want the sql... so I created a redundant mysqldump script that will create zipped backups in the following structure:

<root>
<year>
January.zip
February.zip
...
Week1.zip
Week2.zip
Monday.zip
Tuesday.zip
...

The Script:
The script is pretty short (under 50 lines long). I put the parameters right into the script, but you could easily add them as options. I also have it backing up all databases. This could be run as a cron job/Scheduled Task and should run on Linux, Mac and Windows.
sql_backup.py<br />import os, time, sys, zipfile, datetime,calendar<br />from os import sep<br /><br /># Script Parameters<br /># Target Directory for final Zip files<br />target_dir = r'target' <br /># The name of the sql file (this will be the same for all backups)<br />file_name = 'databases.sql' <br /># Authentication<br />user = 'username' <br />password = 'password'<br /># The command for the mysql dump. This could include the entire path to the executable if mysqldump is not on the path environment variable<br />dumpcmd = r'mysqldump' <br /># The host you want to connect to<br />host = 'example.com' <br /><br />args = [dumpcmd, '-C', '-f', '--host=%s' %host, '--user=%s' %user, '--password=%s' %password, '--result-file=%s' %file_name, '--all-databases']<br /><br />os.chdir(target_dir)<br />cmd = ' '.join(args)<br />print 'Running Command: %s' %cmd.replace(password, '*'*len(password))<br />os.system(cmd)<br /><br /># Below is for zipping the files and creating a redundant backup<br />def getZipFileName():<br /> # Set time variables (<br /> # Note: you don't have to use some of them, <br /> # but they make the script easier to understand<br /> t = time.time(); g = time.gmtime(t)<br /> year = g[0]; month = g[1]; day = g[2]<br /> <br /> try:<br /> datetime.date(year, month, day+1)<br /> except:<br /> # Last day of the month, return month name<br /> root = time.strftime('%Y', g)<br /> if not os.access(root, os.F_OK):<br /> os.mkdir(root)<br /> return os.path.join(root, '%s.zip' %time.strftime('%b'))<br /> <br /> if int(time.strftime('%w', g)) == 0:<br /> # Last day of week, return week number<br /> cal_month = calendar.monthcalendar(year, month)<br /> for week in cal_month:<br /> if day in week:<br /> return 'Week%i.zip' %month.index(week)<br /> # Just a regular weekday...<br /> return '%s.zip' %time.strftime('%A')<br /><br /># Write the zip file<br />zip_name = getZipFileName()<br />zip_file = os.path.abspath(zip_name)<br />zip = zipfile.ZipFile(zip_file, 'w', zipfile.ZIP_DEFLATED)<br />zip.write(file_name)<br />zip.close()<br /><br /># Remove uncompressed sql file<br />os.remove(file_name)<br />to_path = os.path.join(copy_dir, zip_name)<br />to_dir = sep.join(to_path.split(sep)[:-1])<br />if not os.access(to_dir, os.F_OK):<br /> os.makedirs(to_dir)<br />if os.access(to_path, os.F_OK):<br /> os.remove(to_path)<br />shutil.copy(zip_file, to_path)<br />Further Explanation:
The script is five parts:

1. Parameters: This is really all you need to alter for the script to work for you. You will need to specify:
• Target- The directory for all redundant backups
• File Name- The name for all .sql files
• Username/password- The authentication data used for signing into mysql
• Dump command- ok this is just the command used to call mysqldump. You may need to include the full path to the executable if the mysql bin directory is not on your environment path variable
• Host- this is your host- example.com
2. The command for the dump
   
3. Name Resolver: This just resolves the name of the backup (Monday, Tuesday, Week 1, Week 2, etc.)
4. Writing the Zip file
5. Cleanup- remove uncompressed sql file and move the zip file to the specified target
   

Automatically Copy Your Pictures from a Digital Camera

The Problem
Ok... so who gets sick of all of the stupid pop-ups from the media detectors? I personally hate all media detectors because even the best are slow, don't organize your photos by date and month, and are always in your face!

The Solution
I believe the best structure for organizing pictures is: root\YYYY\MM month. This structure is by far easiest to navigate (using Picasa). Each folder does not get too big (I take a lot of pictures) and it is easy at least to remember, within a couple of months, when you took a picture. Also, it is nice because the years AND months actually sort correctly.


The Script
Anyways, I wrote a python script that reads your removable drives and finds DCIM folders and copies them to a specified root folder in the structure specified above.
This script works great for me, but you could add a bit more checking and error logging... It's just over a hundred lines and I wrote it in the last hour and a half (I got sick of copying all the pictures over manually...) You might want to add the -h option once for usage. So here it is:
copy_pics.py<br />from optparse import OptionParser<br />import time, logging, os, sys, win32file, string, shutil<br />from os import sep<br /><br />class DCIM_Mover:<br />def __init__(self, src, dest, logger = None):<br /># Setup Logger<br />if logger is None:<br /> logger = logging.getLogger('DCIM_Mover')<br />self.logger = logger<br />self.src = src<br />self.dest = dest<br /><br />def transfer(self):<br /># The method called to transfer files from source to destination<br />self.logger.warning('Moving All Pictures from %s to %s.' %(self.src, self.dest))<br /># Walk through the root source directory<br />for root, dirs, files in os.walk(self.src, topdown=False):<br /> for name in files:<br /> src_path = os.path.join(root,name)<br /> dest_dir, dest_name = self.getDest(src_path)<br /> dest_dir = os.path.join(self.dest, dest_dir)<br /> if not os.access(dest_dir, os.F_OK):<br /> os.makedirs(dest_dir)<br /> dest_path = os.path.join(dest_dir, dest_name)<br /> self.logger.info('Moving %s to %s' %(src_path, dest_path))<br /> try:<br /> # Move them or log an error.<br /> # By copying first you at least copy it over, it just still exists on the disk.<br /> shutil.copy(src_path, dest_path)<br /> os.remove(src_path)<br /> except:<br /> self.logger.error('Could not move file: %s' %src_path)<br /> for name in dirs:<br /> src_dir = os.path.join(root,name)<br /> try:<br /> os.rmdir(src_dir)<br /> except:<br /> self.logger.error('Could not remove folder: %s' %src_dir)<br /><br />def getDest(self, src_path):<br /># This is where we format the folder structure<br />ctime = time.gmtime(os.path.getctime(src_path))<br />dir = time.strftime('%Y', ctime) + sep<br />dir += time.strftime('%m %b', ctime)<br />name = src_path.split(sep)[-1]<br />return dir, name<br /><br /><br /><br />def removabledisks():<br /># Find and return all drives that are removable<br />driveletters=[]<br />for drive in string.letters[len(string.letters)/2:]:<br />if win32file.GetDriveType(drive+":")==win32file.DRIVE_REMOVABLE:<br /> driveletters.append(drive+":")<br />return driveletters<br /><br /><br />def getDCIMDirs():<br /># Find removable drives with a DCIM directory<br />drives = []<br />for drive in removabledisks():<br />path = '%s\\DCIM' %drive<br />if os.access(path, os.F_OK):<br /> drives.append(path)<br />return drives<br /><br />def setupLogging(level):<br />if not os.access('log', os.F_OK):<br />os.mkdir('log')<br />logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/copy_pics.log'),<br /> filemode='w')<br /># define a Handler which writes INFO messages or higher to the sys.stderr<br />console = logging.StreamHandler()<br />console.setLevel(level)<br /># set a format which is simpler for console use<br />formatter = logging.Formatter('%(name)-5s: %(asctime)-3s [%(levelname)-5s] %(message)s')<br /># tell the handler to use this format<br />console.setFormatter(formatter)<br /># add the handler to the root logger<br />logging.getLogger('').addHandler(console)<br />logger = logging.getLogger('TestMonitor')<br />return logger<br /><br />def main():<br /># This method parses the options set at the command line and initializes the logger.<br />parser = OptionParser()<br />parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br />parser.add_option("-s", "--src", type='string', dest="src", help="Source drive or DIRECTORY. Script will search for all DCIM directories if not specified.", metavar="DIRECTORY", default=None)<br />parser.add_option("-d", "--dest", type='string', dest="dest", help="Copy pictures to DIRECTORY(not including year or month). This is required.", metavar="DIRECTORY", default=None)<br />opts, args = parser.parse_args()<br />logger = setupLogging(opts.debug)<br />return opts, args, logger<br /><br />if __name__ == "__main__":<br />opts, args, logger = main()<br />if opts.dest is None:<br />logger.critical('Destination not provided. Please specify a destination via the --dest=DIR command line option')<br />sys.exit(1)<br />if opts.src is None:<br />dirs = getDCIMDirs()<br />for dir in dirs:<br /> logger.debug('Moving Pictures from: %s' %(dir))<br /> mover = DCIM_Mover(dir, opts.dest)<br /> mover.transfer()<br />else:<br />mover = DCIM_Mover(opts.src, opts.dest)<br />mover.transfer()<br />So, if you're like me and have an all-in-one internal card reader, your drive letter basically never changes. I just create a batch file and put a link to it on my quick launch. This is my batch file:

copy_pics.bat
<br />$(DIRECTORY)\copy_pics.py -d E:\Pictures -s L:\DCIM<br />

I'll probably create a registry key that will run every time I mount a removable drive and check for a DCIM folder... but that can wait till tomorrow... I told my wife I'd come to bed over an hour ago. Oops.

First post

Hello world!