django-ui

Ok, so django comes with a pretty default widgets... but it does allow you to use and develop your own. The problem is I haven't really found any good widget packs. There are some template packs for YUI, which is ok, but I really just want to have unintrusive, clean widgets, components and forms/model forms that you can use that automatically let you use the ajax widgets (so you don't have to write custom code for each dialog, tab, panel, etc) but still allows for customization.

Well after a good bit of searching I found nothing... there was a django-ui project on google code... with nothing in it. So... I started my own project (django-ui) on sourceforge! My goal is to have my own set of widgets, forms, views, etc that allow you to easily integrate ajax and ajax components into django. I'm using jQuery/jQuery UI as my ajax framework and ui library. Another one I liked was dhtmlx, but it really like the demo + documentation style of jQuery UI. Eventually it would be nice to have something like that for this project.

Django + Ajax

I have been doing a lot of looking into which Ajax (ui) framework to use for cronosa and if those frameworks have any django integration projects. I need a simple but highly customizable framework that allows me to do things in a very pythonic / django way... I found a few tools and the standard frameworks, but I really didn't see any integrations with python/django - at least none that I liked or that were complete enough. What I really want is something like JIDE for Java.

The one thing I did see (and haven't really looked into completely) is Maxime Haineault's jQuery integration with django. This sounds pretty promising (not to mention I already use jQuery and it is pretty much the standard when it comes to JavaScript). Source can be checked out here. It seems like he is on the right path, but said he is "not sure it would be a great added value nor that it would be optimal". As for me, I'm all about developing a widget pack that you would use as you would any other widget on a django form.

So, for now I think I'll develop the wigets myself and then submit them to Maxime. If he decides not to use them I'll create a new project that either forks or depends on his.

If anyone has any ideas or would like to help I am open to suggestions and I can give you the subversion link to check out my code.

Python Setuptools "Uprade All"

Ok... so I have found an awesome way to update all python packages here:

Basically you just:

1) install yolk (easy_install yolk)

2) run: yolk -U | cut -d ' ' -f 2 | xargs easy_install

I just don't want to forget this... maybe I should put it in a bash script...

Greg

Cronosa.com... about

Well I finally got a simple "Welcome" page up about cronosa.com. I have been working hard and it is coming along very quickly. I put an email interest form up and man it was tough to go from the dev environment to production! Anyways, I got a form up and I'm about a third of the way to the first "beta" release. I really think cronosa will make scheduling much less tedious for managers.

Plone

Wow... I mean, wow. I went to a Utah Ruby Users Group tonight as I have been looking into Ruby/Rails, Groovy/Grails, and other CMS/frameworks. Ok... so one of the guys there (I thank him deeply) introduced me to Plone. He just said: "Go check it out". So I went and I actually hyperventilated durring the presentation (NEVER happened before). I was jumping up and down on the couch I was so excited. As you can probably tell, I LOVE python. Love it. I was not looking forward to learning grails and rails (I have ready read 400-500 page books on each, but to really learn them it takes a lot of practice and a good learning curve)... but now I DON'T HAVE TOO! I am buying a book tomorrow so I can digest as much as I can on Plone. Just watch the videos at http://www.plone.org. I will have much more to say about plone, I'm sure, in the near future. Development starts tomorrow!

Simple Motherboard Beep Script for Windows

Problem:
So you want to be notified by sound when something is triggered (process starts/stops, file accessed, etc). Well, you could just open a file and let it play, but what if you have headphones plugged in? Then you won't hear anything (unless the volume is extremely loud... but if you listen to it that loud I'm sure you wouldn't hear it anyways). So... how do you make your motherboard beep at the frequency, duration, spacing, and count you desire? Python.

Solution:
So Python has a wonderful module to do just this: winsound. I personally just wanted something that I could import or call from the command line to notify me of an event. So I wrote a little script to do this. I also wrote a script for py2exe.

The Script:
bell.pyfrom optparse import OptionParser<br />import winsound<br />import time<br /><br />def ring(number = 9, frequency = 500, durration = 2000, spacing = 3000):<br /> for x in range(number):<br /> winsound.Beep(frequency, durration)<br /> time.sleep(spacing/1000)<br /><br />def main():<br /> parser = OptionParser()<br /> parser.add_option("-n", "--number", type='int', dest="number", help="NUMBER of Motherboard beeps", metavar="NUMBER", default=9)<br /> parser.add_option("-f", "--frequency", type='int', dest="frequency", help="FREQUENCY of Motherboard beeps", metavar="FREQUENCY", default=500)<br /> parser.add_option("-d", "--durration", type='int', dest="durration", help="DURRATION of Motherboard beeps in milliseconds", metavar="DURRATION", default=2000)<br /> parser.add_option("-s", "--spacing", type='int', dest="spacing", help="SPACING of Motherboard beeps in milliseconds", metavar="SPACING", default=3000)<br /> opts, args = parser.parse_args()<br /> return opts, args<br />if __name__ == "__main__":<br /> opts, args = main()<br /> ring(opts.number, opts.frequency, opts.durration, opts.spacing)<br />
So as you can see the script is only about 20 lines, but you can specify from the command line the number of beeps, duration, frequency, and spacing. I use this all the time for process monitoring, etc. It is dependable because unlike printing the \a character or chr(7) you have control over the beep AND it is dependable (doesn't depend on your terminal settings). To convert this python script to exe I use py2exe
bellexe.pyfrom distutils.core import setup<br />import py2exe<br /><br />setup(console=['bell.py'])

This simply creates an exe that can be called from any program within any language. I use it a lot in automated testing.

Just run:
bellexe.py py2exe
from the command line to create the exe.

A Simple Python Interface for Gmail

The Problem:
I monitor processes, files, and tasks on my computer. Some monitors are for security others are just to know when a task has finished. Whatever the case, I would like to be notified vie email/text message when something has occurred.

The Solution:
As always... write a script. I wanted something that could easily be used to send an email with or without an attachment. I also wanted to be able to just call a mail function and pass in a recipient, subject, message and attachment.

The Script:
The following script is a pretty simple script that can be imported from another python file. I did not provide command line options because I usually use it in conjunction with another script, so it is always an import. I provide a main part so you can input your username, password, and some email to test the script out. Just run it from the command line to test it.
send_mail.py#!/usr/bin/python<br />import smtplib<br />from email.MIMEMultipart import MIMEMultipart<br />from email.MIMEBase import MIMEBase<br />from email.MIMEText import MIMEText<br />from email import Encoders<br />import os<br /><br />class Gmail:<br /> def __init__(self, username, password):<br /> if '@gmail.com' not in username:<br /> username += '@gmail.com'<br /> <br /> self.user = username<br /> self.password = password<br /> self.smtp = "smtp.gmail.com"<br /> self.smtp_port = 587<br /> <br /> def mail(self, to, subject, text, attach = None):<br /> msg = MIMEMultipart()<br /> msg['From'] = self.user<br /> msg['To'] = to<br /> msg['Subject'] = subject<br /> msg.attach(MIMEText(text))<br /> if attach:<br /> part = MIMEBase('application', 'octet-stream')<br /> part.set_payload(open(attach, 'rb').read())<br /> Encoders.encode_base64(part)<br /> part.add_header('Content-Disposition', 'attachment; filename="%s"' % os.path.basename(attach))<br /> msg.attach(part)<br /> <br /> mailServer = smtplib.SMTP(self.smtp, self.smtp_port)<br /> mailServer.ehlo()<br /> mailServer.starttls()<br /> mailServer.ehlo()<br /> mailServer.login(self.user = username, self.password)<br /> mailServer.sendmail(self.user = username, to, msg.as_string())<br /> mailServer.close()<br /><br />if __name__ == "__main__":<br /> username = 'username'<br /> username = 'password'<br /> mobile_email = "YouPhoneNumber@YourProvider.com"<br /> gmail = Gmail(username, username)<br /> gmail.mail(mobile_email,<br /> "Testing Testing 123",<br /> "This is a email sent to you using python!")

Further Explanation:
This could easily extend a general class Mail that would have the same functionality, but you could tailor it to your individual email provider. Contact me if you want such help.

Create Optimized DVD/CD Images of Folder for Backup

The Problem:
I have music, videos, pictures, and files that all need backups. I have them organized how I want, but they don't always fit perfectly on to a DVD. I want a simple program that I point to a directory and it will create optimized image files for burning to a dvd. I would also like to do a few quick 'dry runs' to be sure that I like which files go in each image and that I don't have an image that is only 10% full!

The Solution:
Well, if you can't find one... make one! I decided to write a script that linked up with PowerISO's commandline tool (piso) to create images of files. You will need several Gigs of space to store the images... This is nice because you don't have to go through all your files and figure out how to divide them among images... You just do a few dry runs and then when you are happy, let it go all night!

The Script
This script uses the command line utility of PowerISO to create disc images of a folder and tries to maximize the disc usage while still preserving the file structure of the provided folder. You will need to add the PowerISO install path to your path environment variable
file_dvd_backup.pyfrom optparse import OptionParser<br />import time, logging, os, sys, string, shutil, tempfile<br />from os import sep<br /><br />class ImageTooLargeException(Exception):<br /> def __init__(self, value):<br /> self.value = value<br /> def __str__(self):<br /> return repr(self.value)<br /><br />class DiscImage:<br /> def __init__(self, file_path, base_dir, max = 4.37, password = None):<br /> self.file_name = '.'.join(file_path.split(sep)[-1].split('.')[:-1])<br /> self.logger = logging.getLogger('DiscImage:%s'%self.file_name)<br /> self.temp = None<br /> self.files = []<br /> self.dirs = []<br /> self.command = 'piso'<br /> if base_dir[-1] != sep:<br /> base_dir += sep<br /> self.base_dir = base_dir<br /> self.file_path = file_path<br /> self.max = max *1000*1000*1000 # Size in bytes<br /> self.password = password<br /> self.size = 0<br /> <br /> def setCommand(self, command):<br /> self.command = command<br /> <br /> def add(self, file):<br /> file_size = os.stat(file).st_size<br /> if self.size + file_size > self.max:<br /> raise ImageTooLargeException('File could not be added because it was too large: %s' %file)<br /> self.size += file_size<br /> self.files.append(file)<br /> self.logger.log(5, 'Adding File to Image: %s' %file)<br /> rel_dir = sep.join(file.split(sep)[:-1]).replace(self.base_dir, '')<br /> if rel_dir not in self.dirs:<br /> self.dirs.append(rel_dir)<br /> <br /> def append(self, file):<br /> self.add(file)<br /> <br /> def percent_full(self):<br /> return self.size/self.max*100<br /> <br /> def create(self):<br /> # I copy all to a temp directory because Power ISO does not have an add/append command and will not take more than a few -add command line arguments<br /> self.copyFiles()<br /> piso = self.command<br /> if self.command != 'piso':<br /> cmd = '"%s"' %self.command<br /> else:<br /> cmd = [self.command]<br /> cmd = [self.command]<br /> cmd.append('create')<br /> cmd.append('-o %s' % self.file_path)<br /> <br /> if self.password is not None:<br /> cmd.append('-setpassword %s' %self.password)<br /> <br /> cmd.append('-add \"%s\" /' %self.temp)<br /> <br /> cmd_str = ' '.join(cmd)<br /> if self.password is not None:<br /> cmd_str = cmd_str.replace(self.password, '*'*len(self.password))<br /> <br /> self.logger.debug('Creating Image with Command: \n%s '% (cmd_str))<br /> os.system(' '.join(cmd))<br /> <br /> self.cleanTemp()<br /> <br /> def copyFiles(self):<br /> if self.temp is None:<br /> self.temp = tempfile.mkdtemp()<br /> for file in self.files:<br /> file_dir = os.path.join(self.temp, sep.join(file.replace(self.base_dir, '').split(sep)[:-1]))<br /> if not os.access(file_dir, os.F_OK):<br /> os.makedirs(file_dir)<br /> self.logger.log(3, "Creating Temp File: %s" %os.path.join(file_dir, file.split(sep)[-1]))<br /> shutil.copy(file, os.path.join(file_dir, file.split(sep)[-1]))<br /> <br /> <br /> def cleanTemp(self):<br /> for root, dirs, files in os.walk(self.temp, topdown=False):<br /> for name in files:<br /> try:<br /> os.remove(os.path.join(root, name))<br /> except:<br /> self.logger.error('Could not remove temp file: %s' %os.path.join(root, name))<br /> for name in dirs:<br /> try:<br /> os.rmdir(os.path.join(root, name))<br /> except:<br /> self.logger.error('Could not remove temp dir: %s' %os.path.join(root, name))<br /> try:<br /> os.rmdir(self.temp)<br /> except:<br /> self.logger.error('Could not remove temp dir: %s' %os.path.join(root, name))<br /> <br /> self.temp = None<br /><br /><br />class Dir2Img:<br /> def __init__(self, src, dest, name_base, img_type, max = 4.37, password = None):<br /> self.logger = logging.getLogger('Dir2Img')<br /> self.command = None<br /> self.src = src<br /> self.dest = dest<br /> self.name_base = name_base<br /> self.img_type = img_type<br /> self.max = max<br /> self.password = password<br /> self.images = []<br /> self.parseImages()<br /> <br /> def setCommand(self, command):<br /> self.command = command<br /> for image in self.images:<br /> image.setCommand(command)<br /> <br /> def parseImages(self):<br /> img = self.getNextImage()<br /> img2 = None<br /> for root, dirs, files in os.walk(self.src, topdown=True):<br /> for name in files:<br /> full_path = os.path.join(root, name)<br /> img, img2 = self.addToImage(full_path, img, img2)<br /> <br /> def addToImage(self, full_path, img, img2 = None):<br /> # Try and add it to the current image<br /> try:<br /> img.append(full_path)<br /> except ImageTooLargeException:<br /> # See if the current image is less than 75 percent full<br /> if img.percent_full()< file_size =" os.stat(full_path).st_size/(1000*1000*1000)"> self.max:<br /> raise ImageTooLargeException('File Too Large! File: %s, Size: %fMB' % (full_path, file_size*1000))<br /> <br /> if img2 is not None:<br /> # If we are already dealing with a second image, add the file to that and make it the primary image<br /> return self.addToImage(full_path, img2, None)<br /> else:<br /> # If not, then create a secondary image and add the large file to it<br /> img2 = self.getNextImage()<br /> img2.append(full_path)<br /> else:<br /> # Image is pretty much full<br /> if img2 is not None:<br /> # If dealing with a secondary image, start adding smaller files to it<br /> return self.addToImage(full_path, img2, None)<br /> else:<br /> # Create a new image<br /> img = self.getNextImage()<br /> img.append(full_path)<br /> return img, img2<br /> <br /> def getNextImage(self):<br /> i = ''<br /> if len(self.images) > 0:<br /> i = str(len(self.images))<br /> file_path = os.path.join(self.dest, '%s%s.%s' %(self.name_base, i, self.img_type))<br /> img = DiscImage(file_path, self.src, self.max, self.password)<br /> self.images.append(img)<br /> self.logger.log(1, 'New Image: %s' %file_path)<br /> if self.command is not None:<br /> img.setCommand(self.command)<br /> return img<br /> <br /> def create(self):<br /> for image in self.images:<br /> image.create()<br /> <br /> def usageReport(self):<br /> s = ''<br /> for image in self.images:<br /> s += '%s: %f%% Full\n' %(image.file_name, image.percent_full())<br /> return s<br /> <br /> def fullReport(self):<br /> s = ''<br /> for image in self.images:<br /> s += '\n%s\n'%(image.file_name)<br /> s += '\tUsage: %f%% Full\n' %(image.percent_full())<br /> s += '\tDirectories:\n'<br /> for dir in image.dirs:<br /> s += '\t\t%s\n' %dir<br /> s += '\n'*2<br /> return s<br /> <br /> <br />def setupLogging(level):<br /> if not os.access('log', os.F_OK):<br /> os.mkdir('log')<br /> logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/file_dvd_backup.log'),<br /> filemode='w')<br /> # define a Handler which writes INFO messages or higher to the sys.stderr<br /> console = logging.StreamHandler()<br /> console.setLevel(level)<br /> # set a format which is simpler for console use<br /> formatter = logging.Formatter('%(name)-5s: [%(levelname)-5s] %(message)s')<br /> # tell the handler to use this format<br /> console.setFormatter(formatter)<br /> # add the handler to the root logger<br /> logging.getLogger('').addHandler(console)<br /> logger = logging.getLogger('DVDBackup')<br /> return logger<br /> <br />def main():<br /> # This method parses the options set at the command line and initializes the logger.<br /> parser = OptionParser()<br /> parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br /> parser.add_option("-s", "--src", type='string', dest="src", help="Source drive or DIRECTORY. Script will create images from this directory/drive.", metavar="DIRECTORY", default=None)<br /> parser.add_option("-d", "--dest", type='string', dest="dest", help="Create images in destination DIRECTORY. ", metavar="DIRECTORY", default=None)<br /> parser.add_option("-f", "--file", type='string', dest="file", help="Create image files using base FILENAME. Must be an iso, daa, or bin. Example: Test.iso. NOTE: If not specified, script will use the source directory as a base and create iso images.", metavar="FILENAME", default=None)<br /> parser.add_option("-p", "--password", type='string', dest="password", help="Password for the image. This is optional.", metavar="DIRECTORY", default=None)<br /> parser.add_option("-m", "--max", type='float', dest="max", help="Max disc size. Default is a 4.7 gig disc (4.37 max size).", metavar="SIZE(GB)", default=4.37)<br /> parser.add_option("", "--piso", type='string', dest="piso", help="PowerISO command. Usually piso. If the PowerISO folder is not in your environment path, you will need to specify the full COMMAND to piso.exe here. Default: piso", metavar="COMMAND", default=None)<br /> parser.add_option("", "--dry", action="store_true", dest='dry', help="Dry run only.", default=False)<br /> <br /> opts, args = parser.parse_args()<br /> logger = setupLogging(opts.debug)<br /> return opts, args, logger<br /><br />if __name__ == "__main__":<br /> opts, args, logger = main()<br /> if opts.dest is None:<br /> logger.critical('Destination not provided. Please specify a destination via the --dest=DIR command line option')<br /> sys.exit(1)<br /> if opts.src is None:<br /> logger.critical('Source not provided. Please specify a source via the --src=DIR command line option')<br /> sys.exit(1)<br /> <br /> if opts.file is None:<br /> name_base = opts.src.split(sep)[-1].replace(' ','')<br /> img_type = 'iso'<br /> else:<br /> name_base = '.'.join(opts.file.split('.')[:-1])<br /> img_type = opts.file.split('.')[-1]<br /> <br /> images = Dir2Img(opts.src, opts.dest, name_base, img_type, opts.max, opts.password)<br /> if opts.piso:<br /> images.setCommand(opts.piso)<br /> report = images.fullReport()<br /> logger.info('Image Report: \n%s' %report)<br /> <br /> if opts.dry == False:<br /> images.create()<br />
Further Explanation:
OK, so as an example, I want to backup all of my pictures. I keep them (as seen in a previous post) in the structure of YYYY\MM month and when I burn a backup image of all of my pictures. I would prefer to keep as many pictures of the same year together as possible, however, if there is a large file (say a video clip) in one folder that doesn't fit on a dvd that is only 50% full, I would prefer it to be on a new dvd image, and then continue to fill first dvd, and then fill the new dvd image. Confusing I know, it creates dvd's that are usually about 99% full, even with a few large files mixed in.

I also hate searching for which dvd has what folder. The script creates a list of directories contained on each disc and add that file to each image. It makes it very easy for figuring out which disc has what pictures. The script could easily be altered to contain a list of all folders AND files, but I thought that was overkill.

Blogger Syntax Highlighting

The Problem:
I am still using Blogger.com to host my blog, which works pretty good, you just don't have too many options and no server control. This means I can't just include my scripts on the fly (for now). I have been working with my layout, client side syntax highlighting, and general look and feel of the blog and I am getting somewhat comfortable with it. For syntax highlighting I decided to go with syntaxhighlighter, a google code project written in javascript. It works well, it's just slow. The problem with doing using plugins and including the script right into your post is that I have to generate the html for each blog post... and I want them all to be consistent.

The Solution:
Wanting things to be as simple as possible for me to post and edit my new scripts I wanted a simple code generator. I wanted it to create the html for adding a collapsed code section with a link to the script on my server on the top. I like this format because you can copy it, read it or just download it. Maybe I'll edit the syntaxhighlighter so that it takes a link for downloading the code... but for now this works. Just for fun, here's the code for my code generator:
convert_script.pyfrom optparse import OptionParser<br />import time, logging, os, sys, string, shutil<br />import win32clipboard, win32con, random<br />from urllib2 import urlopen<br />from lxml import etree<br />from os import sep<br /><br />class SyntaxHighlight:<br /> def __init__(self, code, language):<br /> self.syntax = self.script_path = self.script = None<br /> self.code = code<br /> self.lang = language<br /> self.options = [language]<br /> <br /> def setOption(self, option):<br /> if option not in self.options:<br /> self.options.append(option)<br /> <br /> def removeOption(self, option):<br /> if option in self.options:<br /> self.options.remove(option)<br /> <br /> def setScript(self, script):<br /> if '/' in script:<br /> s = '/'<br /> elif '\\' in script:<br /> s = '\\'<br /> else:<br /> s = sep<br /> <br /> self.script = script.split(s)[-1]<br /> <br /> if 'http://' in script:<br /> self.script_path = script<br /> <br /> <br /> def getSyntax(self):<br /> options = ':'.join(self.options)<br /> <br /> self.syntax = ''<br /> if self.script_path is not None:<br /> self.syntax += '<a href="%s">%s</a>' %(self.script_path, self.script)<br /> elif self.script is not None:<br /> self.syntax += self.script<br /> <br /> # I had to break these up or it killed my generated script!<br /> self.syntax += '<' self.syntax += 'textarea name="code" class="%s" cols="60" rows="10">' %(options)<br /> self.syntax += self.code<br /> self.syntax += '</' self.syntax += 'textarea>' <br /> <br /> return self.syntax<br /><br />def getLanguage(script):<br /> ext_dict = {'py':'python', 'java':'java', 'groovy':'java', 'cs':'csharp', 'css':'css', 'php':'php', 'rb':'ruby', 'sql':'sql'}<br /> ext = script.split('.')[-1]<br /> if ext.lower() not in ext_dict.keys():<br /> return ext<br /> else:<br /> return ext_dict.get(ext)<br /><br />def setupLogging(level):<br /> if not os.access('log', os.F_OK):<br /> os.mkdir('log')<br /> logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/convert_script.log'),<br /> filemode='w')<br /> # define a Handler which writes INFO messages or higher to the sys.stderr<br /> console = logging.StreamHandler()<br /> console.setLevel(level)<br /> # set a format which is simpler for console use<br /> formatter = logging.Formatter('%(name)-5s: [%(levelname)-5s] %(message)s')<br /> # tell the handler to use this format<br /> console.setFormatter(formatter)<br /> # add the handler to the root logger<br /> logging.getLogger('').addHandler(console)<br /> logger = logging.getLogger('ConvertScript')<br /> return logger<br /> <br />def main():<br /> # This method parses the options set at the command line and initializes the logger.<br /> parser = OptionParser()<br /> parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br /> parser.add_option("-s", "--script", type='string', dest="script", help="PATH to the script you want to include.", metavar="PATH", default=None)<br /> parser.add_option("", "--lang", type='string', dest="lang", help="Override the language the script is encoded in. Normally taken from the extension. Default: python", metavar="LANGUAGE", default='python')<br /> parser.add_option("", "--code", type='string', dest="code", help="Simple Code. If used you must also specify the --lang", metavar="CODE", default=None)<br /> parser.add_option('-g', '--no-gutter', action="store_true", dest="nogutter")<br /> parser.add_option('-c', '--no-controls', action="store_true", dest="nocontrols")<br /> parser.add_option('-f', '--first-line', type='int', dest="firstline", help="First line to start counting from.", metavar="CODE", default=None)<br /> parser.add_option('-n', '--show-columns', action="store_true", dest="showcolumns")<br /> opts, args = parser.parse_args()<br /> opts.collapse = None<br /> logger = setupLogging(opts.debug)<br /> return opts, args, logger<br /><br />if __name__ == "__main__":<br /> opts, args, logger = main()<br /> if opts.script is None and opts.code is None:<br /> logger.critical('Neither Script nor Code provided. Please specify a script (--script=PATH) or code (--code=CODE) command line option')<br /> sys.exit(1)<br /> <br /> if opts.code is not None and opts.lang is None:<br /> logger.critical('You must specify the language (--lang=LANGUAGE) when specifying code on the command line.')<br /> <br /> if opts.lang is None:<br /> opts.lang = getLanguage(opts.script)<br /> logger.debug('Setting Language Extension to: %s' %opts.lang)<br /> <br /> if opts.code:<br /> opts.nogutter = True<br /> opts.nocontrols = True<br /> code = opts.code<br /> else:<br /> opts.collapse = True<br /> if 'http://' in opts.script:<br /> f = urlopen(opts.script)<br /> else:<br /> f = open(opts.script)<br /> code = f.read()<br /> f.close()<br /> <br /> syntax = SyntaxHighlight(code, opts.lang)<br /> <br /> if opts.script is not None:<br /> syntax.setScript(opts.script)<br /> <br /> if opts.nogutter:<br /> syntax.setOption('nogutter')<br /> <br /> if opts.nocontrols:<br /> syntax.setOption('nocontrols')<br /> <br /> if opts.collapse:<br /> syntax.setOption('collapse')<br /> <br /> if opts.firstline is not None:<br /> syntax.setOption('firstline[%i]' %opts.firstline)<br /> <br /> if opts.showcolumns:<br /> syntax.setOption('showcolumns')<br /> <br /> syntax = syntax.getSyntax()<br /> logger.debug('Syntax: \n%s' %syntax)<br /> win32clipboard.OpenClipboard()<br /> win32clipboard.EmptyClipboard()<br /> win32clipboard.SetClipboardText(syntax)<br /> win32clipboard.CloseClipboard()<br />
Note: The html for the presentation of my code generator on my blog was created by this script...

Further Explanation:
So in essence, all I have to do is point the script at a file on my server and bam! the code is in the clipboard! I love it. It supports code snippits and local or web script files. You can also add options for the syntaxhighlighter javascript. Yea... it's a bit rough, but what do you expect in under an hour...

Redundant MySQL backup with Python

The Story:
Ok so we lost our entire company wiki the day of launch... luckily I was able to recover over 60% of it from the cache of MANY people's computers (I just wrote a script that filtered their cache and spit out the wiki content pages, then a bunch of us went through the pages)... but that is a whole different subject. (Little Note: I was not responsible for backing up or transferring the database)

The Solution:
Since then I have been pretty paranoid about running backups and have volunteered for the task (because I have done a LOT of work on the wiki). I understand MySQL Replication, which is very powerful, but sometimes you just want the sql... so I created a redundant mysqldump script that will create zipped backups in the following structure:

<root>
<year>
January.zip
February.zip
...
Week1.zip
Week2.zip
Monday.zip
Tuesday.zip
...

The Script:
The script is pretty short (under 50 lines long). I put the parameters right into the script, but you could easily add them as options. I also have it backing up all databases. This could be run as a cron job/Scheduled Task and should run on Linux, Mac and Windows.
sql_backup.py<br />import os, time, sys, zipfile, datetime,calendar<br />from os import sep<br /><br /># Script Parameters<br /># Target Directory for final Zip files<br />target_dir = r'target' <br /># The name of the sql file (this will be the same for all backups)<br />file_name = 'databases.sql' <br /># Authentication<br />user = 'username' <br />password = 'password'<br /># The command for the mysql dump. This could include the entire path to the executable if mysqldump is not on the path environment variable<br />dumpcmd = r'mysqldump' <br /># The host you want to connect to<br />host = 'example.com' <br /><br />args = [dumpcmd, '-C', '-f', '--host=%s' %host, '--user=%s' %user, '--password=%s' %password, '--result-file=%s' %file_name, '--all-databases']<br /><br />os.chdir(target_dir)<br />cmd = ' '.join(args)<br />print 'Running Command: %s' %cmd.replace(password, '*'*len(password))<br />os.system(cmd)<br /><br /># Below is for zipping the files and creating a redundant backup<br />def getZipFileName():<br /> # Set time variables (<br /> # Note: you don't have to use some of them, <br /> # but they make the script easier to understand<br /> t = time.time(); g = time.gmtime(t)<br /> year = g[0]; month = g[1]; day = g[2]<br /> <br /> try:<br /> datetime.date(year, month, day+1)<br /> except:<br /> # Last day of the month, return month name<br /> root = time.strftime('%Y', g)<br /> if not os.access(root, os.F_OK):<br /> os.mkdir(root)<br /> return os.path.join(root, '%s.zip' %time.strftime('%b'))<br /> <br /> if int(time.strftime('%w', g)) == 0:<br /> # Last day of week, return week number<br /> cal_month = calendar.monthcalendar(year, month)<br /> for week in cal_month:<br /> if day in week:<br /> return 'Week%i.zip' %month.index(week)<br /> # Just a regular weekday...<br /> return '%s.zip' %time.strftime('%A')<br /><br /># Write the zip file<br />zip_name = getZipFileName()<br />zip_file = os.path.abspath(zip_name)<br />zip = zipfile.ZipFile(zip_file, 'w', zipfile.ZIP_DEFLATED)<br />zip.write(file_name)<br />zip.close()<br /><br /># Remove uncompressed sql file<br />os.remove(file_name)<br />to_path = os.path.join(copy_dir, zip_name)<br />to_dir = sep.join(to_path.split(sep)[:-1])<br />if not os.access(to_dir, os.F_OK):<br /> os.makedirs(to_dir)<br />if os.access(to_path, os.F_OK):<br /> os.remove(to_path)<br />shutil.copy(zip_file, to_path)<br />Further Explanation:
The script is five parts:

1. Parameters: This is really all you need to alter for the script to work for you. You will need to specify:
• Target- The directory for all redundant backups
• File Name- The name for all .sql files
• Username/password- The authentication data used for signing into mysql
• Dump command- ok this is just the command used to call mysqldump. You may need to include the full path to the executable if the mysql bin directory is not on your environment path variable
• Host- this is your host- example.com
2. The command for the dump
   
3. Name Resolver: This just resolves the name of the backup (Monday, Tuesday, Week 1, Week 2, etc.)
4. Writing the Zip file
5. Cleanup- remove uncompressed sql file and move the zip file to the specified target
   

Automatically Copy Your Pictures from a Digital Camera

The Problem
Ok... so who gets sick of all of the stupid pop-ups from the media detectors? I personally hate all media detectors because even the best are slow, don't organize your photos by date and month, and are always in your face!

The Solution
I believe the best structure for organizing pictures is: root\YYYY\MM month. This structure is by far easiest to navigate (using Picasa). Each folder does not get too big (I take a lot of pictures) and it is easy at least to remember, within a couple of months, when you took a picture. Also, it is nice because the years AND months actually sort correctly.


The Script
Anyways, I wrote a python script that reads your removable drives and finds DCIM folders and copies them to a specified root folder in the structure specified above.
This script works great for me, but you could add a bit more checking and error logging... It's just over a hundred lines and I wrote it in the last hour and a half (I got sick of copying all the pictures over manually...) You might want to add the -h option once for usage. So here it is:
copy_pics.py<br />from optparse import OptionParser<br />import time, logging, os, sys, win32file, string, shutil<br />from os import sep<br /><br />class DCIM_Mover:<br />def __init__(self, src, dest, logger = None):<br /># Setup Logger<br />if logger is None:<br /> logger = logging.getLogger('DCIM_Mover')<br />self.logger = logger<br />self.src = src<br />self.dest = dest<br /><br />def transfer(self):<br /># The method called to transfer files from source to destination<br />self.logger.warning('Moving All Pictures from %s to %s.' %(self.src, self.dest))<br /># Walk through the root source directory<br />for root, dirs, files in os.walk(self.src, topdown=False):<br /> for name in files:<br /> src_path = os.path.join(root,name)<br /> dest_dir, dest_name = self.getDest(src_path)<br /> dest_dir = os.path.join(self.dest, dest_dir)<br /> if not os.access(dest_dir, os.F_OK):<br /> os.makedirs(dest_dir)<br /> dest_path = os.path.join(dest_dir, dest_name)<br /> self.logger.info('Moving %s to %s' %(src_path, dest_path))<br /> try:<br /> # Move them or log an error.<br /> # By copying first you at least copy it over, it just still exists on the disk.<br /> shutil.copy(src_path, dest_path)<br /> os.remove(src_path)<br /> except:<br /> self.logger.error('Could not move file: %s' %src_path)<br /> for name in dirs:<br /> src_dir = os.path.join(root,name)<br /> try:<br /> os.rmdir(src_dir)<br /> except:<br /> self.logger.error('Could not remove folder: %s' %src_dir)<br /><br />def getDest(self, src_path):<br /># This is where we format the folder structure<br />ctime = time.gmtime(os.path.getctime(src_path))<br />dir = time.strftime('%Y', ctime) + sep<br />dir += time.strftime('%m %b', ctime)<br />name = src_path.split(sep)[-1]<br />return dir, name<br /><br /><br /><br />def removabledisks():<br /># Find and return all drives that are removable<br />driveletters=[]<br />for drive in string.letters[len(string.letters)/2:]:<br />if win32file.GetDriveType(drive+":")==win32file.DRIVE_REMOVABLE:<br /> driveletters.append(drive+":")<br />return driveletters<br /><br /><br />def getDCIMDirs():<br /># Find removable drives with a DCIM directory<br />drives = []<br />for drive in removabledisks():<br />path = '%s\\DCIM' %drive<br />if os.access(path, os.F_OK):<br /> drives.append(path)<br />return drives<br /><br />def setupLogging(level):<br />if not os.access('log', os.F_OK):<br />os.mkdir('log')<br />logging.basicConfig(level=level,<br /> format='%(asctime)s [%(levelname)s] %(message)s',<br /> filename=os.path.abspath('log/copy_pics.log'),<br /> filemode='w')<br /># define a Handler which writes INFO messages or higher to the sys.stderr<br />console = logging.StreamHandler()<br />console.setLevel(level)<br /># set a format which is simpler for console use<br />formatter = logging.Formatter('%(name)-5s: %(asctime)-3s [%(levelname)-5s] %(message)s')<br /># tell the handler to use this format<br />console.setFormatter(formatter)<br /># add the handler to the root logger<br />logging.getLogger('').addHandler(console)<br />logger = logging.getLogger('TestMonitor')<br />return logger<br /><br />def main():<br /># This method parses the options set at the command line and initializes the logger.<br />parser = OptionParser()<br />parser.add_option("-l", "--logging", type='int', dest="debug", help="LOG level. See http://docs.python.org/library/logging.html. Default: INFO", metavar="LOG", default=logging.INFO)<br />parser.add_option("-s", "--src", type='string', dest="src", help="Source drive or DIRECTORY. Script will search for all DCIM directories if not specified.", metavar="DIRECTORY", default=None)<br />parser.add_option("-d", "--dest", type='string', dest="dest", help="Copy pictures to DIRECTORY(not including year or month). This is required.", metavar="DIRECTORY", default=None)<br />opts, args = parser.parse_args()<br />logger = setupLogging(opts.debug)<br />return opts, args, logger<br /><br />if __name__ == "__main__":<br />opts, args, logger = main()<br />if opts.dest is None:<br />logger.critical('Destination not provided. Please specify a destination via the --dest=DIR command line option')<br />sys.exit(1)<br />if opts.src is None:<br />dirs = getDCIMDirs()<br />for dir in dirs:<br /> logger.debug('Moving Pictures from: %s' %(dir))<br /> mover = DCIM_Mover(dir, opts.dest)<br /> mover.transfer()<br />else:<br />mover = DCIM_Mover(opts.src, opts.dest)<br />mover.transfer()<br />So, if you're like me and have an all-in-one internal card reader, your drive letter basically never changes. I just create a batch file and put a link to it on my quick launch. This is my batch file:

copy_pics.bat
<br />$(DIRECTORY)\copy_pics.py -d E:\Pictures -s L:\DCIM<br />

I'll probably create a registry key that will run every time I mount a removable drive and check for a DCIM folder... but that can wait till tomorrow... I told my wife I'd come to bed over an hour ago. Oops.

First post

Hello world!